SWE-agent

Part I: The mental model

GitHub issue text            "TypeError when calling foo() with a list"
      |
      v
sweagent run                 resolve config, model, env, problem statement
      |
      v
SWEEnv  -->  SWE-ReX         start a sandbox (Docker), clone repo @ base commit
      |                       upload tool bundles, add bin/ to PATH
      v
Agent loop (thought + action)   the model reads the ACI docs and the issue
      |
      |   observation <-- one shell command runs in the sandbox
      v
Agent-Computer Interface     open / goto / scroll / search_* / edit / submit
      |                       windowed file view, linted edits, concise output
      v
edit is applied only if flake8 finds no NEW errors
      |
      v
submit                       git diff of the working tree becomes the patch
      |
      v
.traj (full history) + .patch (the fix)
      |
      v
SWE-bench harness            apply patch, run FAIL_TO_PASS and PASS_TO_PASS tests

The one-sentence identity: SWE-agent is an autonomous issue-fixing agent whose central finding is that a purpose-built Agent-Computer Interface, not the language model behind it, drives the pass rate, so the repository is really a carefully-designed set of commands and feedback rules wrapped around an ordinary thought-action loop. A frontier model handed a raw Linux shell is a poor software engineer. It floods its own context with cat of a two thousand line file, loses track of where it is, applies edits that silently break indentation, and never notices until a test fails much later. SWE-agent's thesis is that most of that failure is an interface problem, and that redesigning the interface to be compact, stateful, and guarded recovers most of the lost performance without touching the model at all.

Two consequences follow. First, the agent and the interface are deliberately separable. The loop that queries the model and parses one action per turn is a few hundred lines, and every command the model can issue lives outside the Python, as a small bash or Python script in a tool bundle that gets uploaded into the sandbox. Redesigning the interface means editing those scripts and a YAML file, not the agent. Second, because SWE-agent was built by the same group that built SWE-bench, it doubles as the reference harness for that benchmark, so reading it is the shortest path to understanding how automated software engineering is actually measured. This chapter is written against SWE-agent 1.0, which rewrote the original research code around a new execution backend called SWE-ReX. The project moves quickly, so where a file path or an exact command is likely to have shifted I say so and stay at concept level.

Part II: Using it

SWE-agent is a Python package that drives models through litellm, so it speaks to OpenAI, Anthropic, and most hosted or local model endpoints with one interface. Execution happens in a sandbox, and the default sandbox is Docker, so you want Docker running locally and an API key for whichever model you choose:

git clone https://github.com/SWE-agent/SWE-agent
cd SWE-agent
pip install --editable .
# a model key for whatever you point it at, for example:
export OPENAI_API_KEY=...        # or ANTHROPIC_API_KEY, etc.
docker --version                 # the default runtime pulls a sandbox image

The core action is to point the agent at one GitHub issue and a repository, pick a model, and cap the spend. The 1.0 command line is a nested flag system, where dotted flags set fields on a config object, so --agent.model.name chooses the model and --env.repo.github_url chooses the repo:

sweagent run \
  --agent.model.name=gpt-4o \
  --agent.model.per_instance_cost_limit=2.00 \
  --env.repo.github_url=https://github.com/SWE-agent/test-repo \
  --problem_statement.github_url=https://github.com/SWE-agent/test-repo/issues/1

The agent clones the repository into a fresh sandbox, reads the issue as its problem statement, and starts issuing commands. You watch a stream of turns scroll by, each one a short natural language thought followed by a single command such as open, search_dir, or edit, with the sandbox's response printed underneath. When the model decides it is done it runs submit, and the run writes two files you care about, a .traj file that is the complete history in JSON and a .patch file that is the fix as a git diff. If you have no issue in mind, the problem statement can also be plain text or a local file, which is the easiest way to try it:

sweagent run \
  --agent.model.name=claude-sonnet-4-20250514 \
  --agent.model.per_instance_cost_limit=2.00 \
  --env.repo.path=. \
  --problem_statement.text="fix the off-by-one in paginate()" \
  --problem_statement.id=local-1

To evaluate on a whole dataset rather than one issue, the sibling subcommand is sweagent run-batch, which pulls a SWE-bench split from Hugging Face, starts many sandboxes in parallel, and writes one prediction per instance. This is the command you use to reproduce a leaderboard number:

sweagent run-batch \
  --agent.model.name=gpt-4o \
  --instances.type=swe_bench \
  --instances.subset=lite \
  --instances.split=dev \
  --instances.slice=:5 \
  --num_workers=4

Everything above can live in a YAML config instead of the command line, and the two can be mixed, with the command line overriding the file. A config file is where you set the agent's behavior, the templates that build each prompt, and the list of tool bundles the model is allowed to use. A trimmed sketch of what one looks like:

agent:
  templates:
    system_template: |
      You are an autonomous software engineer working in a shell.
      You have access to the following commands: {command_docs}
    instance_template: |
      Here is the issue to solve:
      {problem_statement}
    next_step_template: |
      {observation}
      What is your next command?
  tools:
    bundles:
      - path: tools/registry
      - path: tools/search
      - path: tools/windowed
      - path: tools/windowed_edit_linting
      - path: tools/submit
    parse_function:
      type: thought_action

Now the mistakes beginners make. First, expecting speed. A single issue is many model turns inside a container, so it takes minutes and costs real money, which is exactly why per_instance_cost_limit exists and why you should set it low while learning. Second, forgetting the sandbox is ephemeral. The agent works on a clone, and only the final submit diff leaves the container, so nothing touches your real working tree unless you point env.repo.path at it. Third, confusing the two families of documentation. The original research code used a single run.py with flags like --model_name and --config_file, while 1.0 uses the sweagent run subcommands and the dotted config flags above, so old blog posts and the current tool disagree on syntax. Fourth, and most fundamental, do not reach for a bigger model as your first fix when the agent flails. The whole point of the project is that the interface is usually the lever, so a run that thrashes is more often a signal to look at the command set and the feedback than at the model.

Part III: When it is the right tool

SWE-agent is the right tool when you want a transparent, configurable, reproducible agent scaffold for fixing issues in a code repository, and especially when you want to study or measure agent behavior rather than ship a product. It is the reference implementation researchers fork to test a new interface idea, the harness people use to produce SWE-bench numbers they can defend, and the cleanest available codebase for learning what an issue-fixing agent actually is under the marketing. Because every command is an external script and every prompt is a template, it is unusually easy to instrument, ablate, and extend, which is the opposite of a closed product.

The honest cases for alternatives. If your goal is a pair-programming tool that edits your local repo interactively with a human in the loop, Aider is a better fit, since it is built around git commits, repository maps, and a chat you steer, not around a benchmark. If you want a broad, production-leaning agent platform with a full sandbox, a browser, and general task execution beyond issue fixing, OpenHands (formerly OpenDevin) is the larger system. And if you doubt the agent framing entirely, the honest foil is Agentless, a deliberately non-agentic three phase pipeline of localize, repair, and validate that posted competitive SWE-bench numbers with no autonomous loop at all, and whose whole argument is that a fixed pipeline can beat a wandering agent. Moatless Tools and various commercial systems like Devin, Cursor, and Windsurf sit elsewhere on the same map. SWE-agent's niche is deliberate. It is a research instrument for the interface question, not a general-purpose coding assistant with a thousand options.

The design-shaped warning is about what actually moves the number. The intuitive belief is that a smarter model is the way to a higher pass rate, and the more you spend on the model the better you do. SWE-agent's ablations argue the curve is flatter than that near the top and steeper than that near the interface.

tempting belief:  pass rate is a function of the model
   weak model ----------------------------------- strong model
        low pass ----------------------------------- high pass

what the ablations show:  the interface sets the ceiling
   plain shell + strong model      ~10.7 points BELOW
   good ACI    + same model                        the ACI run
        <----- the interface gap can exceed a model generation ----->

In the original paper, swapping the ACI for a plain Linux shell while holding the model fixed cost roughly 10.7 percentage points on a 300 issue subset, a gap comparable to a whole step in model capability. The practical reading is that if you are trying to improve results, profiling the interface, where the agent gets confused, what output floods the context, which edits silently corrupt files, is usually a better investment than upgrading the model, and the tool is built to make exactly that investigation cheap.

Part IV: The full life of one issue resolution

The specimen: a single GitHub issue describing a bug, resolved by sweagent run against a Python repository, then scored by the SWE-bench harness. Most of the machinery below is identical whether the model is GPT-4o or Claude and whether the sandbox is local Docker or a cloud runtime, and where the path forks I follow the common case.

Stage 1: sweagent run and config resolution

sweagent run is the single-instance entry point in the sweagent/run/ package. Its first job is boring and important, assemble one config object out of any YAML files plus the dotted command-line flags, with the command line winning on conflicts. That config names four things that matter for the rest of the run, the model (agent.model), the environment and repository (env), the problem statement (problem_statement), and the agent's templates and tool bundles (agent). Nothing has run in a container yet. This stage is pure configuration, and it is where the cost limit, the step limit, and the choice of tools are fixed before a single model token is spent.

Stage 2: SWEEnv and SWE-ReX bring up a sandbox

The environment object, SWEEnv in sweagent/environment/, does not run commands itself. It delegates to SWE-ReX, an external package (swerex) whose entire job is to run shell commands in a sandbox and abstract over where that sandbox lives, local process, Docker container, Modal, AWS Fargate. SWE-ReX starts the deployment, exposes a small runtime API the host talks to, and opens a persistent interactive shell session inside it, which is what lets a later cd still be in effect on the next command. SWEEnv then clones the target repository into the sandbox at the issue's base commit and installs the tool bundles, and this installation is the clever part, each bundle directory is copied in, its bin/ is prepended to PATH, and its install.sh is sourced. After this stage the sandbox is a normal Linux environment where typing open or edit invokes one of the agent's scripts, and the repository is sitting at the exact commit the bug was reported against.

Stage 3: the first prompt is built from templates

The agent, a stateless DefaultAgent in sweagent/agent/, now builds its opening message from the templates in the config. The system_template explains the rules and, critically, splices in {command_docs}, the auto-generated documentation for every command the tool bundles provide, so the model learns the interface from the same source of truth that implements it. The instance_template injects the issue text as the problem statement. Many configs also insert a demonstration, a full example trajectory of a solved issue, so the model sees the command format in action rather than only in the abstract. This is few-shot teaching of the ACI, and it is why the agent almost never has to guess the syntax of edit.

Stage 4: the thought-action loop turns

Now the loop, which is the heart of the repository. On each turn the agent sends its message history to the model and gets back a response that contains a short natural language thought and a single command. A ToolHandler in sweagent/tools/ parses that response into an action, and it supports several formats behind a parse_function setting, the classic thought-action format where the command sits in a fenced block after the thought, plus JSON, XML, and native function calling for models that support tool use. The parsed command is a single line of shell, and SWEEnv runs it in the persistent session. Whatever the command prints, trimmed to a length limit so a runaway output cannot swallow the context window, comes back as the observation. The next_step_template wraps that observation and asks for the next command, and the loop turns again. One thought, one command, one observation, repeat, and every pair is appended to the trajectory.

Stage 5: navigation and search before any edit

A good run spends its early turns locating the bug, and this is where the ACI earns its keep. Rather than cat or grep, whose raw output is enormous and stateless, the model uses the interface's own commands. search_dir "foo" returns a concise list of files and match counts rather than every matching line, find_file locates a filename, and open path/to/file.py shows a window of roughly a hundred lines with line numbers and a marker of where in the file the window sits. goto, scroll_down, and scroll_up move that window without reprinting the whole file. The interface holds the state, which file is open and which lines are visible, so the model does not have to reconstruct it from a growing pile of raw dumps. This is the difference between a model that knows where it is and one that is lost, and it costs the agent almost no context to maintain.

Stage 6: the guarded edit, and why it can be refused

When the model has found the offending lines it edits them. In the classic interface the command names a line range and supplies the replacement text, which in the original ACI looked like a multi-line block terminated by a sentinel:

edit 218:224
    if len(items) > 0:
        return items[0]
    return None
end_of_edit

The edit is not applied blindly. Before writing, the editor runs a linter, flake8, and compares the errors in the file after the proposed change against the errors before it. If the edit would introduce a new error the file did not already have, a syntax error, a bad indent, an undefined name, the edit is refused, and the model is shown the flake8 output and the relevant lines rather than a corrupted file. This single guardrail is the most cited piece of the ACI, because it converts a class of errors that used to surface much later, as a mysterious test failure, into an immediate, local, recoverable message at the moment of the mistake. The comparison is against the file's prior state, not against zero, so a file that already had lint warnings can still be edited. SWE-agent 1.0 ships several editors behind this same idea, a windowed linting editor, a search-and-replace editor, and an Anthropic-style string-replacement editor, so the exact spelling of the edit command has moved, but the guardrail philosophy is constant.

Stage 7: reproduce, run tests, iterate

Because the sandbox is a real shell, the model can and often does write a small reproduction script and run it, or run the project's own tests, using an ordinary command whose output comes back as the next observation. A capable run tends to follow a loop, reproduce the bug, localize it, edit, rerun the reproduction, and only move on when the symptom is gone. Two limits keep this from running forever, a per-instance cost limit in dollars and a step or context limit, and when either is hit the agent is forced to stop. This is also where a run can quietly go wrong, chasing the wrong file, editing in circles, or fixing the symptom rather than the cause, and the trajectory file is the record you read afterward to see exactly which turn it went off the rails.

Stage 8: submit becomes a patch

When the model runs submit, the interface takes the git diff of the working tree against the base commit and emits it as the prediction. That diff, plus the model's own reasoning that it is finished, ends the loop. The run writes the full history as a .traj JSON file and the fix as a .patch, and for batch runs it also writes a predictions file keyed by instance id in exactly the shape the SWE-bench harness expects. Notice what the agent's output actually is, not an explanation, not a description of the fix, but a unified diff that either applies and passes tests or does not. That closes the agent's half of the loop, issue in, patch out.

Stage 9: the SWE-bench harness scores it

Scoring is a separate program, the swebench harness, and it is deliberately not the agent. It takes the predicted patch, checks out the repository at the same base commit in its own Docker image, applies the patch, then applies the instance's hidden test patch and runs two sets of tests. The FAIL_TO_PASS set are the tests that were failing because of the bug and must now pass, and the PASS_TO_PASS set are tests that were already passing and must not regress. An instance counts as resolved only if every FAIL_TO_PASS test passes and every PASS_TO_PASS test still passes. There is no partial credit and no model in the loop, which is what makes the number trustworthy. That closes the full loop of one issue, natural language bug report in, autonomous edits in a sandbox, a unified diff out, and a hard pass or fail from a test suite the agent never saw.

Part V: Internals deep dives

Deep dive: the Agent-Computer Interface

The ACI is the whole idea, so it deserves to be stated as a design stance rather than a feature list. A Human-Computer Interface is tuned for eyes and hands, scrollbars, syntax highlighting, mouse selection. A model has none of those. It has a context window it reads linearly and a single action per turn, and it cannot see state it did not print. The ACI asks what interface a program would want if it read text and acted one line at a time, and the paper distills the answer into a few principles worth memorizing. Actions should be simple and few, so the model faces a small, learnable vocabulary rather than the entire surface of bash. Actions should be compact, so a whole edit is one action instead of a fragile sequence of cursor moves. Feedback should be informative but concise, so search returns a summary and file views return a window, never a flood. And guardrails should prevent and help recover from errors, so a bad edit is caught at the source. Every concrete piece of SWE-agent, the windowed viewer, the summarizing search, the output length cap, the linting editor, is one of these principles made real, and the pass rate is the sum of them.

The commands themselves fall into three small families. Navigation is open, goto, scroll_down, and scroll_up, which move a stateful window over one file. Search is search_dir, search_file, and find_file, which answer where questions with counts and paths rather than raw matches. Editing is create, edit, and submit, which change files and finish the task. That is close to the entire interface, and its smallness is the point.

Deep dive: tool bundles, the interface as data

The elegant structural choice in 1.0 is that commands are not Python inside the agent. They are files in a tool bundle, a directory with a config.yaml that declares each command's name, arguments, and documentation, a bin/ of the actual executables, and an optional install.sh. At sandbox startup the bundle is uploaded, its bin/ joins PATH, and its install script runs. From then on a command like edit is just a program on the path that the model invokes by typing its name. The bundles you meet most on current main include registry, search, windowed, windowed_edit_linting, submit, review_on_submit, edit_anthropic, and filemap, and swapping which bundles are active is how you reconfigure the interface. A command definition is small and legible:

# tools/search/config.yaml (shape, not verbatim)
tools:
  search_dir:
    signature: "search_dir <search_term> [<dir>]"
    docstring: >-
      searches for search_term in all files in dir; if dir is not
      provided, searches in the current directory
    arguments:
      search_term:
        type: string
        required: true
      dir:
        type: string
        required: false

The docstring is not decoration. It is the exact text spliced into the system prompt as {command_docs}, so the interface the model reads and the interface the sandbox runs are generated from one declaration and cannot drift apart. Representing the command set as uploadable data rather than as agent code is what makes the interface a hypothesis you can edit, which is the machinery behind the paper's central claim. The underlying bin/ script does the work in ordinary shell, printing a concise result:

# tools/search/bin/search_dir  (illustrative)
term="$1"; dir="${2:-.}"
matches=$(grep -rIl -- "$term" "$dir" 2>/dev/null)
n=$(printf '%s\n' "$matches" | grep -c . )
if [ "$n" -eq 0 ]; then echo "No matches for \"$term\" in $dir"; exit 0; fi
echo "Found matches in $n files:"
printf '%s\n' "$matches" | sed 's/^/  /'
echo "(use search_file to see matches within a file)"

Deep dive: the windowed viewer and the state registry

A model cannot hold a scrollbar, so the ACI holds one for it. The windowed viewer shows a fixed window of the open file, roughly a hundred lines by default and configurable, with line numbers and a note of how many lines lie above and below the window. scroll_down and scroll_up move by a window, goto jumps to a line, and each of these reprints only the new window, never the whole file. For this to work across separate command invocations the interface needs memory, because each command is a fresh process, so the registry bundle keeps a small piece of state in the sandbox, the currently open file and the current line, written to a file that the next command reads. That registry is the unglamorous trick that makes the viewer feel stateful even though every command is a stateless one-shot script, and without it scroll and goto would have nothing to move relative to. The payoff in context economy is large. Reading a file in windows keeps the model oriented while spending a bounded amount of context, where a naive cat would spend the file's full length and leave the model no better anchored.

Deep dive: the linting editor

The editor is the sharpest example of a guardrail. Its contract is not merely to change text but to refuse changes that make the file worse by a mechanical measure. Concretely it runs flake8 on the file with the edit applied, compares those errors against flake8 on the file before the edit, and only if the change introduces no new errors does it commit the write. Otherwise it reverts and returns the linter's complaint with the offending lines shown. The comparison against the prior state, rather than against a clean file, is what keeps the editor usable on real repositories that already have style issues, since it forbids only regressions the edit itself caused.

edit request  -->  write to a temp copy
                     |
                     v
              flake8(after)  vs  flake8(before)
                     |
        new errors? --+-- no --> commit the write, show the new window
                     |
                    yes
                     |
                     v
              revert, show flake8 output + the lines, ask again

The reason this matters so much is timing. A syntax error introduced by an edit, if allowed to land, does not announce itself, it surfaces much later as an import failure or a broken test, far from the turn that caused it, where the model has lost the thread. The linting editor collapses that distance to zero, so the error and its fix live on adjacent turns. SWE-agent 1.0 generalizes the idea into a family of editors, the windowed linting editor, a search-and-replace editor, and an Anthropic-style string-replacement editor that matches a unique substring and swaps it, and a config picks which one is on the path. The spelling changes, the principle, never let a bad edit become invisible, does not.

Deep dive: the trajectory loop, parsers, and history

The loop is small enough to hold in your head. Build a message, query the model, parse one action, run it, observe, append, repeat until submit or a limit. Two pieces are worth understanding. First, the parser. The ToolHandler turns a model response into a concrete action, and the parse_function chooses how, thought-action for a fenced command after prose, JSON or XML for models coaxed into structure, or native function calling for models with real tool use. If the model emits something unparseable, the loop can return a format error and let it retry, which is itself a guardrail around the interface. Second, history management. A long run would overflow any context window if every observation stayed verbatim, so history processors shape what the model re-reads, typically keeping recent observations in full and collapsing older ones. The whole run is serialized to a .traj file, a JSON list of the messages and the action-observation pairs, which is the object you replay to understand or debug a run, and the object the project's viewer renders.

Deep dive: SWE-bench, the benchmark that grounds it all

SWE-bench is why the whole enterprise can claim a number. Each instance is a real, merged pull request scraped from a popular open-source Python project, torn back into its parts, a base commit, the issue text as the problem statement, the human's code change as a hidden gold patch, and a hidden test patch carrying the tests that the fix made pass. An agent sees only the repo at the base commit and the issue. Grading applies the agent's patch, then the test patch, and runs the two test sets from Stage 9, FAIL_TO_PASS must flip to passing and PASS_TO_PASS must stay passing, all inside Docker for reproducibility. The full test set has 2,294 instances across a dozen projects like Django, sympy, scikit-learn, and matplotlib. SWE-bench Lite is a 300 instance subset for cheaper iteration, and SWE-bench Verified is a 500 instance subset that humans vetted for solvability and clear specification. The benchmark's realism is exactly what makes the interface argument land, because a raw shell fails on real repositories in ways a toy task would never reveal, and the ACI's value is measured against genuine engineering rather than puzzles.

The headline results, stated carefully. In the original paper, SWE-agent with GPT-4 Turbo resolved 12.47% of the full SWE-bench test set and 18.00%, that is 54 of 300, of SWE-bench Lite, against roughly 3.8% for the best non-interactive retrieval-augmented baseline at the time. Those numbers are from 2024 and are now far below the state of the art, since frontier models and improved scaffolds have pushed SWE-bench Verified well past those figures, and the leaderboard is a fast-moving target. Cite the ablation, not the absolute number, when you want the durable lesson, holding the model fixed and replacing the ACI with a plain shell cost about 10.7 points on a 300 issue subset, which is the whole thesis in one measurement.

Part VI: Reading the repository

The codebase is compact and readable, which is part of its value. Paths below reflect SWE-agent 1.0 on recent main, and the fast pace means a file may have moved, so treat these as roles first and paths second.

Stage 0, orientation. Read the README.md and the online docs, then the tools/ directory listing. Before any Python, open one bundle end to end, its config.yaml and one script in its bin/. Questions: what is a command, physically, in this project, and how does its documentation reach the model?

Stage 1, the entry points. Read the sweagent/run/ package, the run and run-batch paths. Questions: how is a config assembled from YAML plus flags, what does run-batch parallelize over, and where do the .traj, .patch, and predictions files get written?

Stage 2, the agent loop. Read the agent in sweagent/agent/, the DefaultAgent and its step method, with the thought-action loop as the destination. Questions: what exactly is one turn, where does the model get queried, and how does a response become a single action?

Stage 3, tools and parsing. Read sweagent/tools/, the ToolHandler and the parse functions, alongside the tools/ bundle directories they load. Questions: which action formats are supported, how is an unparseable response handled, and how does a bundle's config.yaml become both runnable commands and prompt documentation?

Stage 4, the environment. Read sweagent/environment/, the SWEEnv, and follow it into the swerex package it calls. Questions: what does SWE-ReX abstract, why is a persistent shell session necessary, and where does the repository get cloned at the base commit?

Stage 5, the interface itself. Return to tools/ and read the load-bearing bundles as code, the windowed viewer, the registry that stores current file and line, the linting editor, and the search commands. Questions: how does the viewer stay stateful across one-shot scripts, and what exactly does the editor compare before it commits a write?

Stage 6, the frontier and the grader. Skim the configs under config/, then read the separate swebench harness to see how a patch is scored, and note the SWE-ReX and EnIGMA lines of work, the latter reusing the ACI for cybersecurity capture-the-flag tasks. Questions: what is FAIL_TO_PASS versus PASS_TO_PASS, and what does the harness run in Docker that the agent never sees?

Where not to start: do not begin in the model wrappers or the history processors, which are plumbing that makes more sense once the loop is clear, and do not try to read every config in config/, since one default plus one custom bundle teaches the pattern and the rest are variations.

Part VII: Hands-on labs

Labs 1 through 3 need only a model key and Docker. Labs 4 and 5 are reading and instrumentation and need no model spend. Costs are small if you keep the cost limit low and the slices short, and log formats vary with the fast pace of main.

Lab 1: fix one issue and read the trajectory. Concept: the whole loop of Part IV.

sweagent run \
  --agent.model.name=gpt-4o \
  --agent.model.per_instance_cost_limit=1.50 \
  --env.repo.github_url=https://github.com/SWE-agent/test-repo \
  --problem_statement.github_url=https://github.com/SWE-agent/test-repo/issues/1

Watch the turns scroll by and name each command as navigation, search, or edit. When it finishes, open the written .traj file and match its turns to Stages 4 through 8. Find the first edit and confirm the observation that followed it, was the edit accepted, or did the linter push back?

Lab 2: watch the linter refuse an edit. Concept: the guardrail of Stage 6.

# give it a task likely to tempt a broken edit, keep the budget tiny,
# then grep the trajectory for the linter's refusals
sweagent run --agent.model.name=gpt-4o \
  --agent.model.per_instance_cost_limit=1.00 \
  --env.repo.path=. \
  --problem_statement.text="reindent and simplify the parse() function" \
  --problem_statement.id=lint-demo
grep -i "flake8\|end_of_edit\|not been applied\|errors" *.traj

Look for a turn where the editor reports new flake8 errors and the edit is not applied, and read how the model recovers on the next turn. This is the 10.7 point difference made visible, one turn at a time.

Lab 3: reproduce a SWE-bench Lite slice. Concept: batch evaluation and the harness.

sweagent run-batch \
  --agent.model.name=gpt-4o \
  --instances.type=swe_bench \
  --instances.subset=lite \
  --instances.split=dev \
  --instances.slice=:3 \
  --num_workers=2

You get one prediction per instance in a predictions file. Feed that file to the separate swebench harness to score it, and read the per-instance report, which tells you for each instance whether the patch applied and which FAIL_TO_PASS tests passed. Even three instances teach the shape of the loop and how often a plausible-looking patch fails a hidden test.

Lab 4: read one tool bundle end to end. Concept: the interface as data.

ls tools/windowed_edit_linting
cat tools/windowed_edit_linting/config.yaml
ls tools/windowed_edit_linting/bin

Read the config.yaml to see the command's declared signature and docstring, then read the bin/ script to see the flake8 comparison in the shell. Confirm for yourself that the same docstring is what the model reads as documentation. Predict what the editor would do to an edit that fixes one bug but leaves an undefined name, then find where the script would catch it.

Lab 5: shrink the interface and predict the damage. Concept: the ACI ablation of Part III.

# in a copy of a config, remove the windowed viewer and search bundles,
# leaving the model to fend for itself with plain shell
# then rerun Lab 1 and compare the trajectories

Do not run this to win, run it to lose on purpose. With navigation and search removed, watch the model fall back to cat and grep, watch its context fill with raw output, and watch it lose track of where it is. This is the fastest way to feel why the interface, not the model, sets the ceiling, and it costs one cheap run.

Part VIII: Questions and model answers

Understanding checks. Answer aloud before reading.

1. What is SWE-agent, in one sentence?

An autonomous agent that fixes GitHub issues by driving a language model through a purpose-built Agent-Computer Interface inside a sandboxed repository, whose central claim is that the interface, not the model, is what drives the pass rate.

2. What is the Agent-Computer Interface and why does it exist?

It is the set of commands and feedback rules a model uses to operate a computer, designed for a reader that consumes text linearly and acts one line at a time rather than for eyes and hands. It exists because a raw human-oriented shell floods the model's context, hides state, and lets errors pass silently, and a compact, stateful, guarded interface recovers most of the lost performance without changing the model.

3. Name the ACI design principles.

Actions should be simple and few so the vocabulary is learnable, actions should be compact so a whole change is one action, feedback should be informative but concise so search summarizes and file views window, and guardrails should prevent and help recover from errors so a bad edit is caught at the source.

4. Why does the linting editor sometimes refuse an edit?

Because it runs flake8 after the proposed edit and compares against flake8 before it, and if the change introduces a new error the file did not already have, it reverts and shows the linter's complaint instead of writing a broken file. It compares against the prior state, not a clean file, so it forbids only regressions the edit itself caused.

5. Why is refusing a bad edit worth so much?

Because it collapses the distance in time between an error and its discovery. An unguarded bad edit surfaces much later as a broken import or a failing test, far from the turn that caused it, where the model has lost the thread, while the guardrail puts the error and its fix on adjacent turns.

6. What does the windowed viewer do that cat does not?

It shows a fixed window with line numbers and a note of how much lies above and below, and it keeps the current file and line as state via the registry, so scroll and goto move a cursor the model never has to reconstruct. It keeps the model oriented while spending a bounded amount of context, where cat spends the file's full length and anchors nothing.

7. How is the interface represented, and why does that matter?

As tool bundles, directories of a config.yaml plus bin scripts that are uploaded into the sandbox and added to PATH, not as Python inside the agent. That matters because the command set becomes data you can edit and swap, which is precisely what makes the interface a testable hypothesis rather than fixed code.

8. Trace one turn of the loop.

Build a message from the templates and history, query the model, parse the response into a single action with the ToolHandler, run that command in the persistent sandbox session, capture the trimmed output as the observation, append the action-observation pair to the trajectory, and repeat until submit or a cost or step limit.

9. What is the agent's actual output on submit?

A unified git diff of the working tree against the base commit, not an explanation. That patch is the prediction, and it either applies and passes the hidden tests or it does not.

10. How does SWE-bench decide an issue is resolved?

It checks out the repo at the base commit in Docker, applies the agent's patch and then the hidden test patch, and runs two sets of tests. The FAIL_TO_PASS tests, which the bug was breaking, must all pass, and the PASS_TO_PASS tests, which were already passing, must not regress. Only then does the instance count as resolved, with no partial credit.

11. What are SWE-bench Lite and SWE-bench Verified?

Lite is a 300 instance subset of the 2,294 instance test set for cheaper iteration, and Verified is a 500 instance subset that humans vetted so that each task is actually solvable and clearly specified. Verified is now the more trusted target because it removes underspecified or impossible instances.

12. What role does SWE-ReX play?

It is the execution backend that runs shell commands in a sandbox and abstracts over where that sandbox lives, local, Docker, Modal, or Fargate, and it holds a persistent interactive session so state like the working directory carries across commands. Splitting it out let the agent logic stay independent of the runtime and let many sandboxes run in parallel for batch evaluation.

13. When would you reach for Agentless, Aider, or OpenHands instead?

Agentless when you want a fixed localize-repair-validate pipeline and doubt the value of an autonomous loop, Aider when you want an interactive human-in-the-loop pair programmer built on git, and OpenHands when you want a broad production agent platform with a browser and general task execution. SWE-agent's niche is a transparent research instrument for the interface question.

14. A run keeps thrashing on real issues. What do you profile first?

The interface, not the model. Read the trajectory for where the model gets lost, what output floods its context, and which edits the linter fought, because the project's whole evidence is that the interface usually sets the ceiling and profiling it is cheaper and more often decisive than upgrading the model.

Part IX: Design lessons

Design the interface for the agent that reads it. A model is not a person with a screen, so an interface tuned for eyes and hands wastes its context and hides its state. Meeting the consumer where it actually is, text in, one action out, is the move, and it generalizes to any place we bolt a human UI onto a program and wonder why the program struggles.

Make feedback informative but concise. Search that summarizes, file views that window, output that is length capped, each keeps signal high and context spend bounded. The same instinct improves any log, any error message, any API response, say what changed and where, not everything that happened.

Put guardrails at the moment of the mistake. The linting editor turns a class of errors that used to surface late and mysteriously into an immediate, local, recoverable message. Fail fast and fail near the cause is an old lesson, type checkers, database constraints, assertions, and it pays double when the actor is a model that forgets.

Represent the varying part as data, not code. The command set is tool bundles you upload and swap, so the interface is a hypothesis you can edit rather than a fixed program. Plugins, declarative config, and content-as-data pipelines all win the same way, put the thing you want to vary where varying it is cheap.

Separate the agent, the runtime, and the grader. The loop, SWE-ReX, and the SWE-bench harness are three programs with clean seams, so the runtime can change without touching the loop and the score is produced by code the agent never runs. Clean seams between decision, execution, and evaluation are what make a system both extensible and trustworthy.

Measure on something real, then ablate. The interface claim is only convincing because SWE-bench is genuine engineering and because the paper held the model fixed and removed the ACI to size its effect. Build the honest benchmark first, then change one thing at a time, and the design lessons come out as numbers rather than opinions.

Part X: Memorization framework

The one-sentence summary: SWE-agent wraps an ordinary thought-action loop in a compact, stateful, guarded Agent-Computer Interface built from swappable tool bundles, runs it over a sandboxed repo through SWE-ReX until it submits a git diff, and lets the separate SWE-bench harness score that diff with hidden tests, and its evidence is that the interface, not the model, sets the pass rate.

sweagent run -> config (model, env, problem, tools)
  -> SWEEnv -> SWE-ReX sandbox, clone @ base commit, upload tool bundles
  -> prompt from templates (+ {command_docs} + demonstration)
  -> loop: thought+action -> ToolHandler parse -> run -> observe -> repeat
  -> ACI: open/goto/scroll, search_*, edit (flake8-guarded), submit
  -> .traj (history) + .patch (git diff)
  -> swebench harness: apply patch, FAIL_TO_PASS + PASS_TO_PASS in Docker

The chain mapped to source:

launch            sweagent/run/ (run, run-batch)
agent loop        sweagent/agent/ (DefaultAgent, thought-action step)
parsing           sweagent/tools/ (ToolHandler, parse_function)
environment       sweagent/environment/ (SWEEnv) -> swerex package
the interface     tools/ bundles (registry, windowed, search,
                  windowed_edit_linting, submit, edit_anthropic, filemap)
config            config/ YAML (templates, tools, model)
grading           the separate swebench harness (Docker, hidden tests)

Memorize these blocks:

• The thesis: the interface, not the model, drives the pass rate, and swapping the ACI for a plain shell cost about 10.7 points on a 300-issue subset.
• ACI principles: simple and few actions, compact actions, informative but concise feedback, guardrails that prevent and help recover from errors.
• The commands: navigation (open, goto, scroll_up, scroll_down), search (search_dir, search_file, find_file), editing (create, edit, submit).
• The guardrail: the editor runs flake8 after versus before and refuses edits that introduce new errors.
• The grade: a patch resolves an instance only if all FAIL_TO_PASS tests pass and all PASS_TO_PASS tests still pass, in Docker, with no partial credit.
• The numbers to cite carefully: the 2024 paper reported 12.47% on full SWE-bench and 18.00% on Lite with GPT-4 Turbo, both long since exceeded.

Part XI: Papers and further reading

The ideas in this walkthrough come from a small set of papers, and each one rewards a direct read. Where this site covers the same ground in depth, the companion link points there.

1. Yang et al., SWE-agent, Agent-Computer Interfaces Enable Automated Software Engineering, 2024. The paper this repository implements, with the ACI principles and the plain-shell ablation behind the 10.7 point gap. The agent loop it wraps is built from first principles in the self-improving agents class on this site.
2. Jimenez et al., SWE-bench, Can Language Models Resolve Real-World GitHub Issues?, 2023. The benchmark of 2,294 real issues that grounds every number in this chapter. The craft of building evaluations like it is the subject of the data pipelines and evaluation class.
3. OpenAI and the SWE-bench team, SWE-bench Verified, 2024. The human-vetted 500 instance subset that removed underspecified tasks and became the trusted leaderboard target.
4. Yao et al., ReAct, Synergizing Reasoning and Acting in Language Models, 2022. The interleaved thought and action pattern that the agent loop runs one command at a time.
5. Shinn et al., Reflexion, Language Agents with Verbal Reinforcement Learning, 2023. The complementary lever, feedback kept in the transcript so an agent can learn from its own failed attempts within a run.
6. Xia et al., Agentless, Demystifying LLM-based Software Engineering Agents, 2024. The honest foil from Part III, a fixed localize, repair, and validate pipeline that posted competitive numbers with no autonomous loop.
7. Wang et al., OpenHands, An Open Platform for AI Software Developers as Generalist Agents, 2024. The broad generalist platform that sits at the other end of the design space from this focused research instrument.
8. Abramovich et al., EnIGMA, Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities, 2024. The same ACI ideas carried into capture the flag security tasks, evidence that the interface argument travels.

Part XII: Final takeaway

If the agent loop itself is the unfamiliar part, the self-improving agents material builds the thought-action pattern from first principles, and the model behind the loop is demystified in language models from scratch. Serving that model efficiently, so a batch of agents does not bankrupt you, is the subject of the vLLM and SGLang chapters. Then come back and read the loop in sweagent/agent/ once more, it is only a few hundred lines, and that is the point.