DSPy

Part I: The mental model

you write a program        signatures + modules (Predict, ChainOfThought, ReAct)
      |                    each module is a declarative LM call, PROMPT LEFT BLANK
      v
Module.forward()           ordinary Python control flow composes the calls
      |
      v
Adapter                    turns (signature, demos, inputs) -> chat messages,
      |                    then parses the completion back into named fields
      v
dspy.LM (LiteLLM)          one provider-agnostic, cached model call
      |
      v
Prediction                 typed output fields, e.g. pred.answer
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
optimizer.compile(program, trainset, metric)
      |                    bootstrap few-shot demos + propose instructions,
      v                    search the combination against your metric
same program, tuned        demos and instruction strings are now filled in

The one-sentence identity. DSPy separates what a language-model pipeline should do from the exact prompt text that makes it happen, so you specify behavior as typed signatures wired together by modules, and an optimizer compiles that specification into working prompts by searching demonstrations and instructions against a metric. Ordinary prompt engineering fuses those two concerns into one brittle string. You write the task description, the format rules, the few-shot examples, and the phrasing all in one place, and every change to the model, the data, or one downstream step forces you to rewrite it by hand. DSPy pulls them apart. The program says only that a question flows into an answer, or that a context and a question flow into a grounded answer, and the concrete prompt that carries that flow becomes a parameter the framework fills in.

Two consequences follow. First, the prompt stops being source code you maintain and becomes an artifact you compile, the same way a training loop turns data plus a loss into weights rather than into hand-written activations. The few-shot examples in the prompt are learned, the instruction wording is proposed and selected, and the only thing you author is the information flow and a way to score outputs. Second, because a module is provider-agnostic, the same program runs on any model DSPy can reach, and swapping the backing model becomes a one-line change with a fresh recompile rather than a rewrite. The design descends from the 2022 Demonstrate-Search-Predict work and the 2023 DSPy paper on compiling declarative LM calls into self-improving pipelines, and it sits squarely in the lineage of self-improving agent systems. Everything here is verified against DSPy 2.5 and later in mid 2026. The project moves quickly, so where an exact symbol is likely to have shifted I say so and stay at the level of the idea.

Part II: Using it

DSPy is a pure-Python package. It installs anywhere Python runs, including macOS, because the heavy compute lives behind whatever model endpoint you point it at rather than inside the library.

pip install dspy
# older releases used the PyPI name dspy-ai; the import is always `import dspy`

The first thing any program does is configure a language model. DSPy talks to providers through LiteLLM, so a model is named by a provider prefix and an identifier, and one call registers it as the default:

import dspy

lm = dspy.LM("openai/gpt-4o-mini", api_key="sk-...")
dspy.configure(lm=lm)

You are not tied to a hosted API. Point the same client at a local server that speaks the OpenAI protocol, which is exactly what vLLM and SGLang expose, or at a local Ollama daemon:

# a local vLLM or SGLang OpenAI-compatible endpoint
lm = dspy.LM("openai/meta-llama/Llama-3.1-8B-Instruct",
             api_base="http://localhost:8000/v1", api_key="EMPTY")

# a local Ollama model
lm = dspy.LM("ollama_chat/llama3.2", api_base="http://localhost:11434")

Now the smallest possible program. A module is constructed from a signature, and calling it with keyword arguments named after the input fields returns a Prediction whose attributes are the output fields:

qa = dspy.Predict("question -> answer")
pred = qa(question="What is the boiling point of water at sea level in Celsius?")
print(pred.answer)        # -> "100 degrees Celsius"

You never wrote a prompt. DSPy built one from the signature, sent it, and parsed the reply back into pred.answer. To see the prompt it actually sent, ask for the history, which is the single most useful habit when learning the framework:

dspy.inspect_history(n=1)   # prints the exact messages and the raw completion

Real programs are classes. Subclass dspy.Module, build sub-modules in __init__, and wire them together in forward, exactly the shape of a PyTorch module. Here is retrieval-augmented generation in nine lines:

class RAG(dspy.Module):
    def __init__(self):
        super().__init__()
        self.retrieve = dspy.Retrieve(k=3)
        self.generate = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question):
        context = self.retrieve(question).passages
        return self.generate(context=context, question=question)

rag = RAG()
print(rag(question="Who wrote the paper that introduced DSPy?").answer)

This runs immediately with zero-shot prompts. The point of DSPy, though, is to compile it. You provide a training set of examples and a metric, and an optimizer fills in the prompts. An example carries fields and a declaration of which of them are inputs:

trainset = [
    dspy.Example(question=q, answer=a).with_inputs("question")
    for q, a in raw_pairs
]

def exact_match(example, pred, trace=None):
    return example.answer.lower() == pred.answer.lower()

from dspy.teleprompt import BootstrapFewShot
optimizer = BootstrapFewShot(metric=exact_match, max_bootstrapped_demos=4)
compiled_rag = optimizer.compile(RAG(), trainset=trainset)

After compile, run dspy.inspect_history again and the prompt now carries worked examples that DSPy generated and kept because they passed your metric. Save the result and the demonstrations and instructions travel with it, not the code:

compiled_rag.save("rag_v1.json")   # demos + instructions, as data
loaded = RAG()
loaded.load("rag_v1.json")

Now the mistakes beginners make. First, forgetting .with_inputs(...) on examples, which leaves DSPy unable to tell which fields are given versus predicted, so evaluation and bootstrapping behave strangely. Second, writing a metric that returns something other than a clean boolean or float when a trace is passed. During optimization the metric is called with the trace argument set, and optimizers read its truthiness to decide whether a run's demonstrations are worth keeping, so a metric must stay well defined in that mode. Third, configuring a retrieval model. dspy.Retrieve queries whatever retriever you registered with dspy.configure(rm=...), and current DSPy also encourages passing retrieval as a plain Python function or a tool, so if retrieve returns nothing, check that a retriever exists. Fourth, and most fundamental, do not reach back into the module and start editing prompt strings by hand. The moment you hard-code the phrasing you have forfeited the compile step that the whole framework exists to provide.

Part III: When it is the right tool

DSPy is the right tool when your language-model system has more than one step, when you can measure quality with a metric even a loose one, and when you would rather improve the system by optimizing against that metric than by staring at prompt strings. Multi-hop retrieval pipelines, classification and extraction with strict output formats, agent loops with tools, and LLM-as-judge evaluators are all natural fits, because each has a clear input-output contract and a scorable objective. It is also the right tool when you expect to change models often, since the program is written once and recompiled per model rather than rewritten.

The honest cases for alternatives. LangChain and LangGraph when you mainly want orchestration, prebuilt integrations, and graph-shaped control flow, and you are content to author the prompt strings yourself. LlamaIndex when the center of gravity is data ingestion and indexing for retrieval rather than program optimization. Constrained-decoding libraries like Outlines and the guidance project, or a provider's own structured-output and function-calling features, when your problem is really forcing a grammar or JSON schema onto the tokens, DSPy composes with those through its adapters rather than replacing them. Other prompt-optimization efforts, TextGrad with its textual gradients, AdalFlow, and the evolutionary promptbreeder line, when you want a different optimization philosophy, DSPy's answer is bootstrapping plus grounded instruction proposal plus search. And plain prompt engineering remains right for a genuinely one-shot task where there is nothing to compile and no metric to optimize.

One category mistake worth stating plainly. DSPy is not an inference server and not a fine-tuning trainer, it is the layer above both. It calls a serving stack like vLLM, and its weight-tuning optimizer hands work off to a real finetune. Treating DSPy as a replacement for either is the wrong shape, the same way treating an ORM as a database is the wrong shape:

wrong picture:  DSPy competes with vLLM / a finetuning trainer

right picture:  DSPy program  (signatures, modules, optimizer)
                     |  provider-agnostic LM call
                     v
                serving stack   vLLM / SGLang / Ollama / a hosted API
                     |
                     v
                the model weights (which BootstrapFinetune can also tune)

The other subtle trap is treating the metric as an afterthought. An optimizer is only as good as the signal it optimizes, so a noisy or misaligned metric compiles a program that is confidently wrong. The discipline DSPy asks for is the same one that evaluation harnesses teach, decide how you will measure quality before you try to improve it.

Part IV: The full life of one compile

The specimen is one call to BootstrapFewShot.compile on a program that has one or more predictors inside it. This is the operation that gives DSPy its name, and understanding it end to end is understanding the framework. I trace BootstrapFewShot because it is the simplest optimizer that still shows the whole idea, then note what the heavier optimizers add.

optimizer = BootstrapFewShot(metric=exact_match, max_bootstrapped_demos=4)
compiled  = optimizer.compile(student=program, trainset=trainset)
      |
      v
1. prepare        student = program.reset_copy()   fresh copy, demos cleared
      |           teacher = student unless one is supplied
      v
2. labeled seed   sample raw (input, label) pairs from trainset and attach
      |           up to max_labeled_demos of them to each predictor
      v
3. run teacher    for example in trainset:
      |               with dspy.context(trace=[]):
      |                   pred = teacher(**example.inputs())
      |               trace = [(predictor, inputs, outputs), ...]
      v
4. keep winners   if metric(example, pred, trace):
      |               for (predictor, inputs, outputs) in trace:
      |                   predictor.demos.append(Example(**inputs, **outputs))
      |               (up to max_bootstrapped_demos per predictor)
      v
5. stop           when every predictor is full or the trainset is exhausted
      |           (repeat up to max_rounds, re-running the failures)
      v
compiled program  identical code, demonstrations now populated

Stage 1: reset and choose a teacher

compile starts by making a clean copy of the program with reset_copy, so the demonstrations it is about to collect land on a fresh student rather than mutating your object. It then picks a teacher, which is the program that will generate the demonstrations. By default the teacher is the student itself, which is why bootstrapping is a form of self-improvement, the program teaches itself from its own successful runs. You can pass a stronger or differently configured teacher through teacher_settings, for example a larger model that produces better traces which a smaller student then learns to imitate.

Stage 2: seed with labeled demonstrations

Before any bootstrapping, the optimizer runs the equivalent of the LabeledFewShot strategy, sampling raw input-label pairs straight from the training set and attaching up to max_labeled_demos of them to each predictor. These are honest gold examples with no model reasoning attached, and they give the student a floor to stand on. Bootstrapped demonstrations, which carry the intermediate fields the program produces, are layered on top.

Stage 3: run the teacher under a trace

This is the heart of the mechanism. For each training example the optimizer runs the teacher inside a context that installs a fresh trace, roughly with dspy.context(trace=[]). Every time any predictor inside the program makes a call while that context is active, DSPy appends a tuple of the predictor, the inputs it received, and the outputs it produced to the trace. So after one run of a two-stage RAG program the trace holds two entries, one for the retrieval-conditioned generation and one for anything else the program invoked, in execution order. The trace is the program's own record of what each step did on this example, and it is what makes multi-stage optimization possible without the user labeling the intermediate steps.

Stage 4: keep the demonstrations from successful runs

The optimizer now calls your metric as metric(example, pred, trace). Because the trace is present, the metric runs in its gating mode and its truthiness decides the fate of this run. If the run succeeded, every entry in the trace becomes a demonstration. For each predictor the optimizer builds a dspy.Example from that step's inputs and outputs and appends it to that predictor's demo pool, up to max_bootstrapped_demos per predictor. The crucial point is what a demonstration contains. It is not just the final answer, it is the full set of fields that step produced, including a ChainOfThought step's reasoning field. So the compiled prompt for the generator will show the model complete worked examples of the exact intermediate behavior you want, all generated by the program itself and filtered by your metric. Runs that fail the metric contribute nothing, which is the filter that keeps quality up.

Stage 5: stop, and what the heavier optimizers add

Bootstrapping stops when every predictor has collected its quota of demonstrations or the training set is exhausted, and max_rounds can repeat the pass so that a program partly taught in round one produces better traces in round two. The returned student is your original program, unchanged in code, with demos now populated on each predictor. That is the entire compile.

The stronger optimizers wrap this loop. BootstrapFewShotWithRandomSearch runs the bootstrap several times over different random subsets and seeds, evaluates each compiled candidate on a validation set, and keeps the best. MIPROv2 goes further, adding a second axis. It bootstraps demonstration candidates as above, and it also proposes several candidate instruction strings per predictor using a proposer grounded in your program's source, a summary of your data, and a few bootstrapped examples, then it runs a Bayesian search over the joint choice of which instruction and which demonstration set to assign to each predictor, scoring combinations on the validation set with minibatches to control cost. COPRO optimizes instructions alone by coordinate ascent, repeatedly proposing improved wordings. BootstrapFinetune takes the same successful traces and, instead of stuffing them into a prompt, fine-tunes the underlying model's weights on them. Every one of these speaks the same interface, a program in, a metric and a training set as supervision, and a better program out, which is why they are interchangeable and stackable.

Part V: Internals deep dives

Deep dive: signatures as typed specs

A signature is a declarative statement of a transformation's input and output fields. The compact form is a string, "question -> answer" or "context, question -> answer", where names left of the arrow are inputs and names to the right are outputs. The rich form is a class, and it is where signatures earn their keep:

class Emotion(dspy.Signature):
    """Classify the emotion expressed in a sentence."""
    sentence: str = dspy.InputField()
    emotion: str = dspy.OutputField(desc="one of: joy, sadness, anger, fear, surprise")

Three things are doing work here. The docstring becomes the instruction that heads the prompt, so the task description lives in one obvious place and an optimizer can rewrite it. Each field is a dspy.InputField or dspy.OutputField with an optional desc and prefix, which the adapter turns into labels and hints in the rendered prompt. And the Python type annotation is a real constraint, so a field typed as float, a Literal, a bool, or a Pydantic model tells the adapter what to request and how to parse and validate what comes back. Under the hood a signature is a Pydantic model built by a metaclass in the signatures package, so the fields, their order, their descriptions, and the instruction are all structured data the rest of the system can read and modify rather than an opaque template.

Because a signature is structured, it can be transformed programmatically, and that is exactly how ChainOfThought works. It takes your signature and prepends a new output field that holds the model's step-by-step reasoning, named reasoning in current versions and rationale in older ones, then delegates to a plain Predict on that extended signature. So the reasoning is not a hidden prompt trick, it is a first-class output field you can read on the prediction, and it flows into demonstrations during compilation. Once the input-output contract is typed metadata rather than prose, the framework can insert fields, rewrite instructions, and validate outputs, and improving a program becomes editing that metadata rather than editing text.

Deep dive: modules and the Predict call path

dspy.Module is the base class every program and every built-in module subclasses, and like a PyTorch module it can hold sub-modules as attributes and be traversed to find its parameters. The parameters that matter to an optimizer are the predictors, and Module exposes them through helpers such as predictors() and named_predictors(). A Predict is the leaf. It holds a signature, a list of demos, an optional per-module lm, and generation config, and those demos and the signature's instruction are precisely the state that optimizers write.

Follow one call. When you invoke a module, its __call__ runs forward, and a Predict's forward does four things in order:

Predict.forward(**inputs)
   1. resolve the active adapter and LM (from dspy.settings or the module)
   2. adapter.format(signature, demos, inputs)  ->  a list of chat messages
   3. lm(messages)                              ->  one or more raw completions
   4. adapter.parse(signature, completion)      ->  a dict of typed fields
        |
        v
   Prediction(**fields)   and, if a trace is active, append (self, inputs, fields)

The adapter is the piece that turns typed intent into text and back. The default ChatAdapter renders a system message from the instruction and the field descriptions, then the demonstrations as prior turns, then the current input, and it marks every field with a header like [[ ## answer ## ]] so that parsing the reply is a matter of finding those sections rather than guessing at free text. JSONAdapter instead asks the model for JSON and can lean on a provider's structured-output mode, which is sturdier when the model supports it. The same signature drives either adapter, so the format is a swappable concern and not baked into your program. The LM call itself goes through dspy.LM, which is built on LiteLLM, so one code path reaches every provider, and DSPy caches calls by default, which makes recompiling and re-evaluating far cheaper than the raw token count suggests. The trace append in step four is the same hook Part IV depends on, so tracing, prompting, and optimization all meet at this one method.

The other built-in modules are compositions over this path. ReAct takes a signature and a list of tools and runs the ReAct agent loop, at each turn asking the model for a thought and a tool choice with arguments, executing the tool, appending the observation to a running trajectory, and repeating up to max_iters before a final step extracts the signature's outputs from the trajectory. Tools are ordinary Python callables wrapped as dspy.Tool, and a finish tool lets the model end the loop. The exact internal field names have changed across releases, so learn the loop shape rather than the current spelling. ProgramOfThought has the model write and run code, MultiChainComparison samples several reasoning chains and compares them, and Refine and BestOfN run a module several times and keep the best by a reward, which is also where the older assertion-and-backtrack machinery has largely migrated.

Deep dive: the optimizer family

Every optimizer lives in the dspy.teleprompt package, still named for the original word teleprompter even though the docs now say optimizer, and every one exposes the same compile(student, trainset=..., ...) entry point. They differ in which knobs of the program they turn:

Three ideas unify the table. The first is that few-shot demonstrations are learnable parameters, not fixtures, so the bulk of the family simply searches for good demonstrations, and the cheapest way to get good ones is to let the program generate them and filter by the metric, which is a form of self-distillation. The second is that instructions are also learnable, but proposing good instructions blindly is hopeless, so MIPROv2 grounds its proposer in the actual program code, a summary of the data, and example traces, which is what lets the proposed wording be about your task rather than generic advice. The third is that the metric is the one interface, the loss function of the whole enterprise, and it can be as simple as exact match or as rich as an LLM judge you build from a signature and a Predict. Because optimizers share the interface, they stack, and a common recipe is to bootstrap demonstrations, then run MIPROv2 for instructions, then optionally finetune, each step measured against the same metric on the same validation split.

A caution on cost and expectations. Compilation makes many model calls, since it runs the program across the training set, sometimes several times, and evaluates candidates on a validation set. MIPROv2 exposes auto settings of "light", "medium", and "heavy" that trade search budget for quality, and evaluation runs many calls in parallel through num_threads, which is where a local high-throughput server and an understanding of parallel execution pay off. The honest expectation is that optimization helps most when the program has real structure and a faithful metric, and helps least when the task is trivial or the metric is noisy. It is optimization, not magic, and a bad objective compiles a bad program.

Deep dive: evaluation and the metric contract

A metric is a plain function with the signature metric(example, prediction, trace=None). It returns a number or a boolean in the normal case, which dspy.Evaluate averages over a dataset, and it returns a boolean gate when trace is not None, which is the mode optimizers use to decide whether a run's demonstrations survive. That dual role is the one subtlety, and it is why the same function serves both scoring and bootstrapping. Evaluation itself is a thin harness:

evaluator = dspy.Evaluate(devset=devset, metric=exact_match,
                          num_threads=8, display_progress=True, display_table=5)
score = evaluator(compiled_rag)

When exact match is too brittle, the metric can itself be a small DSPy program. A common pattern defines an assessment signature and runs a ChainOfThought judge inside the metric, so the objective you optimize against is itself a compiled, model-graded check. That is the flexibility the metric contract buys, the objective is code, so it can be as smart as the task needs.

Part VI: Reading the repository

The package under dspy/ is organized by concept and reads cleanly in dependency order. Names below are accurate to recent releases, and where a file is likely to have moved I name the component by its role.

Stage 0, orientation. Read the top-level README.md and the concept pages on the docs site, then open dspy/__init__.py to see exactly what the public surface is, which classes are exported, and where each lives. Questions to hold. What is the difference between a signature, a module, and an optimizer, and which of the three do you author by hand?

Stage 1, signatures. Read the signatures package, the signature class and its metaclass in dspy/signatures/signature.py and the field definitions in dspy/signatures/field.py. Questions. How does a docstring become an instruction, how are input and output fields distinguished, and how does the string form "a, b -> c" get parsed into fields?

Stage 2, modules. Read dspy/predict/predict.py first, then chain_of_thought.py and react.py in the same package, and the primitives they rest on, dspy/primitives/module.py for the base class, example.py for Example, and prediction.py for Prediction. Questions. What are the four steps of Predict.forward, how does ChainOfThought extend a signature, and how is the trace appended?

Stage 3, adapters and clients. Read the adapters package, dspy/adapters/chat_adapter.py and json_adapter.py against their shared base, then the client layer in dspy/clients/, especially lm.py and the base LM, plus the cache. Questions. What does the ChatAdapter field-marker format buy for parsing, when is the JSONAdapter the better choice, and what exactly does DSPy cache?

Stage 4, optimizers. Read dspy/teleprompt/bootstrap.py as the anchor, then the LabeledFewShot optimizer and the random-search wrapper in the same package, then mipro_optimizer_v2.py and copro_optimizer.py, then bootstrap_finetune.py. Pair the MIPRO reading with the proposer in dspy/propose/. Questions. Where does the trace become demonstrations, how does random search pick a winner, and what grounds the instruction proposer?

Stage 5, evaluation and retrieval. Read dspy/evaluate/, the evaluator and the built-in metrics, and the retrieval package for Retrieve and ColBERTv2. Questions. How does the metric's dual role show up in the code, and how does a program call out to a retriever?

Where not to start. The many provider and retriever integrations are breadth, not depth, and reading one is enough to see the pattern. The streaming and async paths, the experimental optimizers such as the reflective and introspective ones, and the weight-finetuning provider plumbing are all worth meeting only after the Predict-and-Bootstrap core is solid.

Part VII: Hands-on labs

Every lab needs only a configured model, either a hosted API key or a local vLLM, SGLang, or Ollama endpoint, and no GPU of your own. Log and prompt formats shift with releases, so read what your version prints.

Lab 1: watch DSPy write a prompt. Concept: the Predict call path of Part V.

import dspy
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

qa = dspy.Predict("question -> answer")
print(qa(question="How many moons does Mars have?").answer)
dspy.inspect_history(n=1)   # read the exact messages DSPy sent

Read the printed messages closely. Identify the instruction, the field markers, and where your input landed. You wrote none of it. Then swap Predict for dspy.ChainOfThought("question -> answer"), rerun, and confirm a reasoning field now appears both in the prompt structure and on the prediction.

Lab 2: a class signature with types. Concept: typed fields drive requesting and parsing.

from typing import Literal

class Classify(dspy.Signature):
    """Classify the sentiment of a product review."""
    review: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

clf = dspy.Predict(Classify)
out = clf(review="The battery dies in an hour but the screen is gorgeous.")
print(out.sentiment, out.confidence)

Note that out.confidence comes back as a float and out.sentiment is constrained to the three labels. Change a type or add a field and rerun to see the prompt and the parsing follow the signature.

Lab 3: build a metric and evaluate a baseline. Concept: the metric contract and Evaluate.

trainset = [dspy.Example(question=q, answer=a).with_inputs("question")
            for q, a in pairs[:50]]
devset   = [dspy.Example(question=q, answer=a).with_inputs("question")
            for q, a in pairs[50:100]]

def em(example, pred, trace=None):
    return example.answer.strip().lower() == pred.answer.strip().lower()

program = dspy.ChainOfThought("question -> answer")
base = dspy.Evaluate(devset=devset, metric=em, num_threads=8)(program)
print("baseline:", base)

Record the baseline number. This is the reference every later lab compares against, and building the metric first is the discipline Part III argues for.

Lab 4: compile with BootstrapFewShot. Concept: demonstrations as learned parameters.

from dspy.teleprompt import BootstrapFewShot
opt = BootstrapFewShot(metric=em, max_bootstrapped_demos=4, max_labeled_demos=4)
compiled = opt.compile(program, trainset=trainset)

after = dspy.Evaluate(devset=devset, metric=em, num_threads=8)(compiled)
print("compiled:", after)
compiled(question=devset[0].question)
dspy.inspect_history(n=1)   # the prompt now carries self-generated worked examples

Compare after to the Lab 3 baseline, then read the history and find the bootstrapped demonstrations DSPy injected. Notice they include the reasoning field, generated by the program and kept only because they passed em.

Lab 5: move to MIPROv2 and inspect the artifact. Concept: joint instruction and demo search, and configs as data.

from dspy.teleprompt import MIPROv2
mipro = MIPROv2(metric=em, auto="light")
optimized = mipro.compile(program, trainset=trainset, valset=devset)

print("mipro:", dspy.Evaluate(devset=devset, metric=em, num_threads=8)(optimized))
optimized.save("qa_mipro.json")   # then open the JSON and read it

Open qa_mipro.json in a text editor. It contains the proposed instruction and the selected demonstrations as plain data, which is the whole point, your compiled program is a document, not a rewritten prompt. Try auto="medium" and weigh the extra calls against the gain.

Lab 6: an agent with a tool. Concept: ReAct and tools.

def word_count(text: str) -> int:
    """Return the number of whitespace-separated words in text."""
    return len(text.split())

agent = dspy.ReAct("request -> answer", tools=[word_count])
print(agent(request="How many words are in the phrase 'programming not prompting'?").answer)
dspy.inspect_history(n=3)   # watch thought, tool call, observation, repeat

Follow the trajectory in the history, the model's thought, its choice of the tool and arguments, the observation you returned, and the final extraction of the answer field. Add a second tool and pose a question that needs both.

Part VIII: Questions and model answers

Understanding checks. Answer aloud before reading on.

1. What is DSPy in one sentence?

A framework that separates a language-model pipeline's behavior from its prompt text, letting you declare typed signatures wired by modules and then compile that program into working prompts by bootstrapping demonstrations and tuning instructions against a metric.

2. What are the three core abstractions and which do you write by hand?

Signatures, which specify typed input and output fields, modules, which are strategies like Predict, ChainOfThought, and ReAct that turn a signature into behavior, and optimizers, which compile a program against data and a metric. You author signatures, the composition of modules, and the metric. The optimizer authors the prompts.

3. What does a signature actually contain?

An instruction, taken from the class docstring, and an ordered set of input and output fields, each with a name, an optional description and prefix, and a Python type. It is a Pydantic model built by a metaclass, so all of that is structured data the framework can read, rewrite, and validate against.

4. Walk the four steps of a Predict call.

Resolve the adapter and LM, have the adapter format the signature, demonstrations, and inputs into chat messages, send them through the LM, and have the adapter parse the completion back into typed fields. The result becomes a Prediction, and if a trace is active the call is recorded for optimization.

5. How does ChainOfThought differ from Predict?

It prepends a reasoning output field to your signature and then delegates to a plain Predict on that extended signature. The reasoning is a real output field you can read on the prediction and that flows into demonstrations, not a hidden prompt trick.

6. What is the trace, and why does bootstrapping need it?

The trace is the ordered list of every predictor call made during one run, each recorded as the predictor plus its inputs and outputs. Bootstrapping needs it because it turns a whole-program success into per-step supervision, each step's inputs and outputs become a demonstration for that step, so intermediate stages get examples without the user labeling them.

7. What exactly does BootstrapFewShot produce?

The same program with each predictor's demos filled in. It seeds with labeled gold pairs, then runs the teacher across the training set, keeps the runs whose metric passes, and turns each successful run's trace entries into demonstrations, up to a quota per predictor.

8. Why is bootstrapping a form of self-improvement?

By default the teacher is the student itself, so the program generates its own candidate demonstrations, and the metric filters them, keeping only the ones from successful runs. The program learns from its own best behavior, which is self-distillation.

9. What does MIPROv2 add over BootstrapFewShot?

A second axis. Alongside bootstrapped demonstration candidates it proposes candidate instructions with a proposer grounded in the program's code, a data summary, and example traces, then runs a Bayesian search over the joint assignment of instruction and demo set per predictor, scoring combinations on a validation set with minibatches.

10. Why is the metric the most important thing you write?

It is the objective every optimizer maximizes, the loss function of the system. A faithful metric compiles a good program and a noisy or misaligned one compiles a confidently wrong program, so quality of the metric bounds quality of the result.

11. Why does the metric take a trace argument?

Because it plays two roles. With no trace it returns a score that Evaluate averages over a dataset. With a trace present it returns a boolean gate that optimizers use to decide whether a run's demonstrations are kept. The same function serves scoring and bootstrapping.

12. When would you pick LangChain or LlamaIndex instead?

LangChain or LangGraph when you mainly want orchestration and integrations and are content to write the prompts yourself, LlamaIndex when the job is data ingestion and indexing for retrieval. DSPy wins when you have a scorable objective and want to optimize the prompts against it rather than hand-tune them.

13. Is DSPy a replacement for vLLM or a finetuning trainer?

No. It is the layer above both. It issues provider-agnostic model calls that a server like vLLM answers, and its BootstrapFinetune optimizer hands weight tuning to a real finetuning path. DSPy orchestrates and optimizes, it does not serve or train by itself.

14. Your compiled program scores worse than the baseline. Name three suspects.

A metric that is noisy or misaligned with what you actually want, so the optimizer chased the wrong signal. Too little or unrepresentative training data, so the bootstrapped demonstrations do not generalize. Or a validation set that overlaps the training set, so the search overfit. Fix the objective and the splits before touching the optimizer.

Part IX: Design lessons

Separate the program from the prompt. DSPy's founding move is to split what a step should do, the signature, from the exact text that makes it happen, the compiled prompt. This is mechanism-and-policy separation, the same instinct that keeps business logic out of persistence code, and it is what lets the prompt be optimized without touching the program.

Make the objective a first-class function. By insisting on a metric, DSPy gives the whole system a loss function, and once quality is measurable it can be optimized. Compilers, training loops, and search all win the same way, name the objective and the machinery can improve against it.

Treat examples as learned parameters. Few-shot demonstrations are not fixtures you write, they are weights the optimizer sets, and the cheapest good weights come from the program's own successful runs filtered by the metric. Reframing a hand-authored artifact as a learnable one is the move that unlocks automation.

Ground generation in the real context. Proposing instructions blindly produces platitudes, so MIPROv2 feeds its proposer the program's code, a data summary, and example traces. Retrieval-augmented generation, grounded tool use, and this proposer all share the lesson, a model asked to produce something useful needs the specifics of the situation in front of it.

Put the boundary at a provider-agnostic call. Routing every model call through one LiteLLM-backed client makes the program portable across providers and local servers, so the same code runs on a hosted API today and a local vLLM deployment tomorrow. A single narrow interface at the volatile boundary is what keeps everything above it stable.

Compile to data, not to code. A compiled DSPy program is a JSON document of instructions and demonstrations, so it is diffable, reviewable, and shippable without regenerating source. Keeping the output of optimization as inspectable data rather than rewritten text is what makes the whole process auditable.

Part X: Memorization framework

The one-sentence summary. DSPy turns a language-model pipeline into signatures wired by modules, records every step's behavior in a trace, and lets an optimizer compile the program by keeping demonstrations from successful runs and proposing grounded instructions, all scored against a metric you define.

write         signatures (typed in/out) + modules (Predict, ChainOfThought, ReAct)
run           Module.forward -> Adapter formats -> dspy.LM calls -> Adapter parses
supervise     a metric(example, prediction, trace) scores outputs and gates demos
compile       optimizer bootstraps demos + proposes instructions, searches the mix
ship          save the tuned program as JSON (instructions + demos), reload later

The chain mapped to the package:

signatures    dspy/signatures/ (signature.py, field.py)
modules       dspy/predict/ (predict.py, chain_of_thought.py, react.py)
primitives    dspy/primitives/ (module.py, example.py, prediction.py)
rendering     dspy/adapters/ (chat_adapter.py, json_adapter.py)
model call    dspy/clients/ (lm.py, base LM, cache) over LiteLLM
optimizers    dspy/teleprompt/ (bootstrap.py, mipro_optimizer_v2.py, copro, finetune)
proposal      dspy/propose/ (grounded instruction proposer for MIPRO)
scoring       dspy/evaluate/ (evaluate.py, metrics.py)

Memorize these blocks:

• Three abstractions: signatures specify typed input and output, modules turn a signature into behavior, optimizers compile a program against data and a metric.
• Predict path: resolve adapter and LM, format to messages, call the model, parse to typed fields, record to the trace.
• Bootstrapping: run the teacher under a trace, keep runs the metric passes, turn each step's inputs and outputs into demonstrations for that step.
• Optimizer axes: few-shot demos (LabeledFewShot, BootstrapFewShot, random search, KNN), instructions (COPRO, MIPROv2), and weights (BootstrapFinetune), all sharing compile().
• Metric contract: metric(example, prediction, trace=None), a score for Evaluate and a boolean gate for optimizers, the loss function of the system.

Part XI: Papers and further reading

The ideas in this walkthrough come from a small set of papers, and each one rewards a direct read. Where this site treats the same idea in depth, the companion link points there.

1. Khattab et al., DSPy, Compiling Declarative Language Model Calls into Self-Improving Pipelines, 2023. The paper this repository implements, signatures and modules compiled by optimizers that bootstrap their own demonstrations. The broader family of systems that improve from their own runs is surveyed in the self-improving agents class.
2. Khattab et al., Demonstrate-Search-Predict, Composing retrieval and language models for knowledge-intensive NLP, 2022. The predecessor framework whose pipeline-aware demonstration bootstrapping became the compile step traced in Part IV.
3. Opsahl-Ong et al., Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs, 2024. The MIPRO paper, grounded instruction proposal plus Bayesian search over the joint choice, which MIPROv2 implements. The discipline of scoring candidates against a held-out set is the subject of the data pipelines and evaluation class.
4. Singhvi et al., DSPy Assertions, Computational Constraints for Self-Refining Language Model Pipelines, 2023. The constraint-and-backtrack machinery whose role has largely migrated into Refine and BestOfN.
5. Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, 2022. The prompting result that dspy.ChainOfThought turns into a reusable module with a typed reasoning field.
6. Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models, 2022. Sampling several reasoning paths and keeping the agreement, the idea behind MultiChainComparison.
7. Yao et al., ReAct, Synergizing Reasoning and Acting in Language Models, 2022. The thought-action-observation loop that dspy.ReAct packages as a module over tools.
8. Chen et al., Program of Thoughts Prompting, Disentangling Computation from Reasoning for Numerical Reasoning Tasks, 2022. Having the model write code and hand the arithmetic to an interpreter, the pattern behind ProgramOfThought.
9. Santhanam et al., ColBERTv2, Effective and Efficient Retrieval via Lightweight Late Interaction, 2021. The late-interaction retriever behind dspy.ColBERTv2 and the classic retrieval examples. The data-and-indexing side of retrieval is the focus of the LlamaIndex walkthrough.
10. Yuksekgonul et al., TextGrad, Automatic Differentiation via Text, 2024. The textual-gradient alternative that Part III weighs against DSPy's bootstrap-and-search philosophy.
11. Fernando et al., Promptbreeder, Self-Referential Self-Improvement Via Prompt Evolution, 2023. The evolutionary line of prompt optimization named among the alternatives in Part III.

Part XII: Final takeaway

If the model side underneath DSPy is the gap, the language models from scratch and applied generative AI material builds the intuition for what the calls do, and the vLLM chapter shows what answers those calls at scale. Then come back and write one small program, give it an honest metric, and run compile. The moment you read the history and see prompts you never wrote outscore prompts you labored over, the thesis lands.