DevBox: RudderStack's Open Source Harness That Turns Tickets Into Pull Requests

RudderStack spent six months feeding its ordinary engineering backlog to an autonomous agent, then did the thing open source companies keep promising and not doing: it released the agent. DevBox went public under an MIT license with a launch blog on October 27, 2026, and the pitch is one sentence long: hand it a ticket, and it plans the change, writes the code, runs the tests, reviews its own diff, and opens a draft pull request. You review and merge. Comment on the pull request, and it pushes the fix to the same branch.

What makes this worth a post is not that an agent can open a pull request. Plenty of tools do that. DevBox is built as a shared org tool: one harness that picks up Linear tickets, answers on pull requests, works across dozens of repositories, and commits what it learned back into the repos it touched. RudderStack reports it now merges about 35% of the company's internal pull requests. Those are the vendor's own numbers, and this post treats them that way, but the architecture around them is open source and worth reading regardless of whether you believe the statistics.

The short version

• DevBox is a harness, not a model. A control plane, a per-task sandbox, a staged agent pipeline, and an egress proxy. OpenAI or Anthropic, your keys, your infrastructure.
• The pipeline is gated. Orientation, planning, plan review, implementation, a review-and-fix loop, distillation. No code is written until the plan survives a separate agent arguing against it.
• Credential isolation is the load-bearing design choice. Sandboxes hold placeholder tokens; an egress proxy (iron-proxy) injects the real ones and allows writes only to repos you list. If you have read our egress firewall playbook, this is that pattern shipped as a product.
• Memory is versioned and reviewed like code. Learnings are distilled into knowledge files in the repo and land as a commit on the same pull request.
• The numbers are real but self-reported. 1,500+ pull requests since April, 66 repositories touched, an 80% merge rate over the last three reported months, rising monthly from 72% in August to 91% in October.
• v1 is deliberately narrow. GitHub only, Linear plus CLI as task sources, two LLM APIs, and honest docs about what is missing.

The ticket DevBox was built for: the one nobody schedules

Every engineering org carries work that is clearly worth doing and never worth prioritizing. RudderStack's Principal Engineer Fotis Xenikoudakis opens the launch post with four examples: an API client one field short, a CI test whose timeout is too tight, a dependency pinned three minor versions back, and documentation that drifted from the code six weeks ago. The work is understood, a spec could be written in two minutes, and no engineer will ever have a free afternoon for it. So it accumulates, and the cost shows up as reruns, support tickets, and upgrades that balloon because they never got the hour they needed.

That is the scope DevBox deliberately claims: short to medium tasks that are already well understood. It is not positioned to design a complex feature, and its failure modes are named up front: vague tickets, tribal knowledge that was never written down, and sandbox images that are not built for your stack (detailed with the numbers below). Human review and steering are part of the design rather than a concession. In other words, this is not a pitch to replace engineers; it is a pitch to give the unschedulable backlog somewhere to go.

How the pieces fit

Three components, split cleanly:

• The control plane (control-plane/) is a Bun service with Postgres that stores tasks, asks OpenSandbox for a fresh sandbox per task, and relays messages and progress to Linear and GitHub. It makes no LLM calls. A Linear webhook starts a task; that is what "assigned to DevBox" triggers.
• The harness (harness/) runs inside the sandbox and is built on DeepAgents, LangChain's deep-agent scaffold. It carries the task from orientation through the pull request and reports progress back to the control plane as it goes.
• iron-proxy sits between the sandboxes and GitHub and the LLM endpoints. It swaps the sandbox's dummy tokens for real ones and refuses every host that is not on the allowlist. On Kubernetes it is the sandbox's only route out and its only name resolver: sandboxes use it as their DNS server, it answers every name with its own address, and NetworkPolicies block any other egress. On the Compose trial stack, Docker network aliases send only the GitHub and LLM hosts through it, and sandbox traffic to other hosts bypasses the proxy entirely.

Deployment matches the audience. A Docker Compose path exists so you can try it on a laptop in an afternoon (bun run quickstart, about 15 minutes, most of it the first image build). For a team it installs from a Helm chart, one pod per task, with a production example that gives sandboxes a Kata Containers runtime class, meaning each task is a real virtual machine on dedicated nodes instead of a container sharing the host kernel. The chart shipping a production example with a VM-per-task configuration is a telling choice about who they expect to run this; the launch post says of the Compose path that it "isn't the shape to run a team on."

Seven model stages, one plan gate

Inside the sandbox, an assignment moves through fixed stages. The list below follows the architecture doc, which is slightly more precise than the launch blog's six-step telling:

1. Orientation is one agent that reads the workspace and your conventions (AGENTS.md, CLAUDE.md, skills, .agents/knowledge/) and picks the repos and components the task touches.
2. Planning runs one agent per chosen repository, in parallel, each drafting that repo's part of the plan.
3. Plan review is a separate adversarial agent whose only job is to argue with that plan. No code is written until it passes.
4. Implementation runs one agent per planned repo.
5. Review puts reviewer agents over the diff at the same time, each looking for a different class of problem, while a fix agent resolves or declines each finding; the review-then-fix loop runs up to HARNESS_REVIEW_MAX_ITERATIONS passes (two by default), with six parallel reviewers as the default cap.
6. Distillation turns the notes the agents recorded into the repo's knowledge files.
7. Commit branches, pushes, and opens the draft pull request; it and the setup step before orientation call no model.

Seven of these stages drive a model, each with its own model and reasoning effort: orientation, planning, plan review, implementation, review, distillation, and the follow-up agent that answers comments on the pull request.

Per-stage model routing is the economically interesting part, and it is the same pattern we described in our token-usage field guide: review runs on a stronger model at high effort while the early reading stages run cheaply, so cost and quality get set per stage instead of once for the whole pipeline. The repo's example config shows how a model is pinned per provider, and the configuration docs cover specifying models per stage.

The closed feedback loop is what makes it a teammate rather than a batch job. Review comments on the pull request, @mentions, failed CI runs, and follow-up messages in the Linear session all route back to the running task, and new commits land on the same branch. The GitHub App adds a bot identity, self-refreshing tokens, and comments from people you trust reaching the task. One caveat is worth knowing up front: on GitHub, DevBox only picks up follow-ups on its own pull requests. It does not start work from GitHub issues.

Versioned memory, committed where you can review it

The feature that distinguishes DevBox from a pile of one-shot agents is the memory model. Each agent writes down what it is doing as it goes, into the Linear agent session: the decision it took and the alternative it passed over, the correction a reviewer gave it, the mistake it made and what the right answer turned out to be.

A distillation stage then decides which of those observations are worth keeping and files them by domain into the repository's knowledge files: architecture, conventions, patterns, entry points, the stack. Those files are committed in the same pull request as the code change, so new knowledge arrives for review next to the work that produced it. When DevBox learns something wrong, you see it in the diff and correct it the way you would fix a misleading comment. The launch post is honest about the loop the other way too: tribal knowledge (the things humans know and never wrote down) is invisible to the agent, and hand-over-the-shoulder fixes for that tend to be expensive.

Why pointing it at a private repo is defensible

Most teams hesitate before giving an autonomous process a token with write access to source code, and this is where DevBox's design overlaps most with our own writing. Five properties, from the launch post and README:

• Credentials never enter the sandbox. It receives dummy values for the GitHub token and model key. iron-proxy substitutes the real ones as requests leave.
• Egress is denied by default on Kubernetes. There the proxy is the only route out and the only resolver, and anything not on the allowlist is refused. The Docker Compose path is a trial setup with weaker plumbing: network aliases route only the GitHub and LLM hosts through the proxy, and sandbox traffic to other hosts never reaches it.
• Writes are proxy-enforced, not prompt-enforced. Only the repositories listed in the config are writable: the allowlist permits everything on /{owner}/{repo}/* and /{owner}/{repo}.git/* for each configured repo, and anything outside those paths is refused at the proxy regardless of what any text in the sandbox says. Reads are broader than writes; any repo can be cloned and fetched.
• Instruction is scoped to people, not to anyone with a comment box. DevBox reads pull request discussion only from organizations and users on the trust list, and the list is empty by default, meaning it reads no pull request discussion at all until you name people.
• One sandbox per task. Tasks share no workspace, so one cannot corrupt another's branch or state.

This is the same layered posture our Docker sandbox walkthrough and default-deny egress playbook argue for at the design level, shipped end to end. Note what it does not claim: the agent still sees your code and still sends model-bound context to whatever LLM endpoint you configure, as with any coding tool, and an allowlist cannot stop data riding inside a permitted request. No system solves that today.

Getting started in an afternoon

The quickstart assumes macOS, and wants Docker Desktop with 8 GB of memory for Docker (Compose 2.13 or newer), Bun 1.4.4 or newer, make and openssl on PATH, a GitHub token or a logged-in gh, and an OpenAI or Anthropic API key. Secrets go into the macOS Keychain (or the Linux Secret Service), never into the config file.

$ git clone https://github.com/rudderlabs/devbox.git && cd devbox
$ cp devbox.config.example.json devbox.config.json
# put your repo URL, branch, and model in devbox.config.json
$ bun run quickstart
# picks Compose or a local kind cluster, checks tools, builds images (about 10 min first time)
$ bun run devbox create "Add a /health endpoint to your-org/your-repo"
task key: DBX-API-1
$ bun run devbox logs -f DBX-API-1
# you watch it orient, plan, review the plan, implement, test, and self-review
$ bun run devbox send DBX-API-1 "Also add a test"
# pushes new commits to the same draft pull request
$ bun run down
# tasks persist; bun run reset deletes them

So how good is it, really

The claims, all company-reported: running across RudderStack's engineering org since April, DevBox has produced 1,500+ pull requests touching 66 repositories, and it merges about 35% of the company's internal pull requests with about an 80% merge rate. Over the last three reported months it merged 734 pull requests across 60 repositories, and the monthly merge rate climbed from 72% in August to 79% in September to 91% in October. RudderStack's own read of that trend is that DevBox got more capable and the engineers got better at briefing it.

Read these with the denominator in mind, because the launch blog states it for the three-month figure: the rate counts only pull requests DevBox took to a decision, merged or closed. Closed is what a person says when they read the change and judge it not worth taking, and a wide spec gap shows up there. Two footnotes from the same chart: the 80% is pooled across the whole period, not an average of the monthly rates, and October's 91% is a partial month, the chart running only to the 26th. RudderStack's own failure analysis names three modes without ranking them: vague tickets produce vague changes, tribal knowledge stays invisible until someone writes it down, and the harness needs a sandbox image built for your stack (toolchain, versions, dependency trees) before you judge the results. The one lever the post does single out: an engineer refining the requirements into the issue description before delegating moved the outcome more than any configuration change they tried.

Two anecdotes are concrete claims about process rather than metrics. A new customer hit a bug in an integration; a pull request was open 21 minutes after an engineer approved the work, and the fix was live that afternoon. And the pull request that relicensed DevBox under MIT ahead of the release was opened by DevBox itself. That last one is not publicly checkable: the public repo begins with the release commit, so the relicensing pull request lives in RudderStack's internal history.

What v1 does not do

The README has a plain "Not in v1" section, which I appreciate more than a roadmap. Everything you might assume and be wrong about is written down.

Also worth knowing: Slack receives only completion updates for Linear tasks, and that integration is Kubernetes-only for now (Compose refuses the config block outright). There is no telemetry to RudderStack. Tracing with Langfuse is optional, off by default, and pointed at your own Langfuse instance.

Against the agent at your keyboard

The most common question in the repo's FAQ is whether DevBox replaces Claude Code or Codex, and the answer the maintainers give is the one I would give after reading the design: no, those tools make one developer faster, and RudderStack says it uses them every day alongside DevBox. They occupy different slots.

The comparison that actually matters for adoption is organizational: the question is not which tool writes better code, it is whether your team has a place to delegate whole tickets, with review and audit trails, that is not one person's terminal.

FAQ

Is it safe to point at a private repository?

The design tries to make the question smaller rather than to answer it with trust. Real credentials are never inside the sandbox, egress is allowlisted and proxied, writes are refused at the proxy for anything outside your listed repos, comments are only read from people you named, and each task gets its own environment. The residual risks are the ones no tool solves: the model endpoint sees whatever code context the task needs, and an attacker allowed inside a permitted host can still carry data out in a permitted request. The strict default-deny egress posture is the Kubernetes deployment, where Kata VMs and network policy tighten the picture further; the Compose stack is a trial setup, and only its GitHub and LLM hosts are aliased through the proxy.

What is the merge rate actually measuring?

Merged pull requests divided by pull requests merged or closed, over pull requests DevBox took to a decision in the period. Over the three reported months that is 80% pooled across the period (72/79/91 by month, October counted only through the 26th), on 734 merged pull requests across 60 repositories. The 35%-of-internal-pull-requests figure is the README's current headline, not a three-month window. Company-reported, and the denominator is spelled out, which is more than most agent claims offer.

Does it replace Linear or my project workflow?

It leans on Linear: assigning an issue to DevBox, or @mentioning it on one, is the recommended team path, with progress posted into the issue's agent session so nobody on the team needs the CLI. Linear sends webhooks, so a first try on a laptop needs a stable public URL (a static ngrok domain or named tunnel, because free tunnels change URL and the webhook follows the old one). The CLI alone does everything if you just want it for yourself.

Is DevBox free?

The code is MIT licensed on GitHub. You pay for your own infrastructure and your own model API usage. There is also a public site form for joining the community list.

Can I run it on my own model?

Only through a server that speaks OpenAI's Responses API with reasoning, on port 80 or 443 under a DNS name. That is the v1 constraint, stated plainly in the README, and it is the first thing to check before planning a local-model deployment.

Further reading

• DevBox on GitHub: https://github.com/rudderlabs/devbox
• The launch blog post: https://www.rudderstack.com/blog/open-sourcing-devbox/
• The DevBox product page: https://www.rudderstack.com/devbox/
• Architecture (control plane, harness, proxy, task states): https://github.com/rudderlabs/devbox/blob/main/docs/architecture.md
• Configuration, secrets and trust settings: https://github.com/rudderlabs/devbox/blob/main/docs/configuration.md
• Running it on Kubernetes: https://github.com/rudderlabs/devbox/blob/main/docs/kubernetes/README.md
• Our default-deny egress firewall playbook for coding agents: https://systhoughts.com/posts/ai-coding-agents-egress-firewall
• Our Docker walkthrough for sandboxing agents and keeping secrets on the host: https://systhoughts.com/posts/docker-sandboxes-ai-agents
• Our coding agent field comparison (Codex, Claude Code, OpenCode and friends): https://systhoughts.com/posts/codex-vs-claude-code-vs-opencode-vs-pi-vs-oh-my-pi-vs-hermes
• Our guide to workstation isolation and its limits: https://systhoughts.com/posts/securing-developer-workstation-isolation-hardening-limits-linux-security

Would your org hand coffee-machine-level tickets (timeout tweak, API field, doc drift) to a background agent with only your conventions for socialization, and would you even trust your own ticket descriptions enough to delegate them? That is the comment this post needs.

Until next time, keep your systems thoughtful.