DSPy GEPA Optimizer

Compile and tune DSPy 3 programs with GEPA — the reflective prompt optimizer that mutates instructions using natural-language feedback and keeps a Pareto frontier of candidates. This skill is for developers who already have a DSPy pipeline (classifier, RAG answerer, extraction module, agent) and want measurably better prompts without hand-tuning wording or hopping to a separate fine-tuning workflow.

When to use it

• Your dspy.ChainOfThought or dspy.ReAct module works but accuracy plateaus on a held-out set.
• You can express quality as a metric that returns more than a bare score — e.g. why an answer was wrong.
• You want optimization that improves the instruction text itself (readable, inspectable prompts), not few-shot demo bootstrapping alone.
• You have a small-to-medium labeled set (dozens to a few hundred examples) and a budget to spend on reflection calls.

How it works

GEPA runs a reflective evolutionary loop. It executes your program on the trainset, reads the per-example feedback your metric returns, and asks a separate reflection_lm to propose improved instructions for the underperforming predictors. New candidates are evaluated, scored against the valset, and retained on a Pareto frontier so distinct strengths aren't discarded.

Key pieces:

• Signature — typed I/O contract (question -> answer or a class with InputField/OutputField).
• Metric — signature metric(gold, pred, trace=None, pred_name=None, pred_trace=None); return a float or a dspy.Prediction(score=..., feedback="..."). The text feedback is what GEPA learns from.
• Budget — set auto="light"|"medium"|"heavy" OR a hard max_metric_calls. Use one, not both.
• reflection_lm — usually a stronger model than your task LM; pass it explicitly.

Usage

curl -fsSL --create-dirs -o ~/.claude/skills/dspy-gepa-optimizer/SKILL.md https://claudeskil.com/skills/dspy-gepa-optimizer/SKILL.md

import dspy
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

program = dspy.ChainOfThought("question -> answer")

def metric(gold, pred, trace=None, pred_name=None, pred_trace=None):
    ok = pred.answer.strip().lower() == gold.answer.strip().lower()
    fb = "Correct." if ok else f"Expected '{gold.answer}', got '{pred.answer}'. Be more literal."
    return dspy.Prediction(score=float(ok), feedback=fb)

gepa = dspy.GEPA(
    metric=metric,
    auto="light",
    reflection_lm=dspy.LM("openai/gpt-4o", temperature=1.0, max_tokens=8000),
)
optimized = gepa.compile(program, trainset=trainset, valset=valset)
optimized.save("optimized.json")

Tips & gotchas

• Feedback quality dominates results. A metric that only returns 0.0/1.0 gives GEPA nothing to reason about — return diagnostic text.
• reflection_lm typically needs a high max_tokens (several thousand); reflection prompts are long.
• Keep a genuinely held-out test set GEPA never sees; it optimizes against trainset+valset and can overfit.
• Set track_stats=True to inspect candidate lineage and the frontier afterward via the result's detailed results.

FAQ

GEPA vs. MIPROv2 / BootstrapFewShot? Those optimize demos and instructions via search/Bayesian methods. GEPA evolves instructions from textual feedback and often wins when you can articulate why outputs fail.

Does it change my code? No — it returns a new compiled program with rewritten predictor instructions (and demos). Load it with program.load("optimized.json").

Roughly how many LM calls? auto="light" is cheapest; max_metric_calls gives exact control. Reflection calls add on top of evaluation calls.