i told my lead agent to hand everything off. it became a switchboard, and wrote the best code anyway
· Christoph Heuwieser
tl;dr
- Six setups built the same app side by side, from the same 30 prompts by a scripted user: one solo agent, and five six-agent "meshes" with different rules.
- The rule I bet on was a lead agent told to hand work off first, plus a volunteer who keeps the plan. It didn't spread ownership at all. 98 % of handoffs went through the lead, and nobody ever passed a task on.
- That same team wrote the best code on one measure, and by a wide margin: 0.62 against 0.37–0.49 for everyone else on the hidden-test curve over the run. On the plain count of passing tests the lead is real but still inside the noise.
- "Budding", copying a working agent's session into an idle one, never got a fair test. My rig fired it too rarely, and a direct follow-up test broke too. The two pairs that ran hint that the copy brings real knowledge of the code, and also the donor's sense of who it is.
- In this setup, at this n, a single agent was still at least 2× cheaper per passing test. Also: one agent ran xargs kill and took down its own team.
Where part 3 left off
Part 3 ended with one lesson: in both meshes, one head did basically all the work. The receptionist (the "concierge") either became the app, or pushed everything into one helper. So this time I stopped hoping and tested two ideas for making ownership spread on its own:
- A: handoff-first. The concierge's instructions flip from "do it yourself unless handing off makes sense" to "hand it off first". A volunteer "caretaker" keeps the plan and an overview of who owns what. The caretaker is explicitly told it doesn't hand out work, so it can't become the new boss.
- B: budding. Every fifth step, the rig copies the session of whoever knows an area best into an idle agent, through the a4sx library. The copy then offers to co-own that area. Nothing is assigned.
That gives four meshes (neither, A, B, both). I ran the plain one twice, to see how much two identical runs differ by pure chance, plus one solo agent as a yardstick. All six ran at the same time, on the same router, with 6 agents per mesh.
The app is Horse Tinder again, cut to 30 prompts, with a "sleep" halfway: every agent writes notes and gets a fresh session before step 21. The scripted user starts vague and keeps jumping scope:
S-01: "I run a community of horse breeders and I want "Horse Tinder"…" S-16: "People miss things. Send emails: when there's a new match (both owners), a new message (to the recipient), a new booking (to the stallion's owner) and when someone wins an auction." S-19: "Premium membership: 9.90 euros for 30 days, paid by card through HorsePay…"
59 hidden tests grade the app from the outside after every step. The agents never see them.
Before running anything, I wrote down what would count as "it works" and froze the analysis code. The main measures are things no agent is told to do:
- does each agent mostly stick to one area of the app;
- after the sleep, does the same agent keep owning its area;
- do change requests reach the agent that owns that code.
Code quality and cost are guardrails.
The results
| Setup | Steps that pass the hidden checks | Hidden tests at step 30 (of 59) | Quality curve (higher is better) | Cost per passing test | |---|---|---|---|---| | Solo | 15 of 29 done | 44 | 0.38 | $3.16 | | Plain mesh, run 1 | 17 of 28 | 43 | 0.37 | $11.47 | | Plain mesh, run 2 | 19 of 28 | 48 | 0.39 | $6.43 | | A: hand off first | 19 of 28 | 53 | 0.62 | $6.35 | | B: budding | 18 of 29 | 46 | 0.40 | $6.76 | | A + B | 15 (lead restarted once) | 47 | 0.49 | $12.86 |
The quality curve is the area under the "share of hidden tests passing" line across the run, so it rewards being right early as well as at the end.
The two plain-mesh runs, identical setups, differ by 5 hidden tests and almost 2× in tokens. Keep that in mind for everything below: with one run per setup, a lot of differences are just noise.
A: the lead became a switchboard
The handoff-first lead handed off a lot. 64 of its team's 65 handoffs came from it, spread across four colleagues. A typical message:
"I've handed S-24 to w1 to add the five-second booking sweep and configurable vet deadline, with a short-deadline live QA run before reporting completion."
So the work spread out. The routing never did. Nobody passed a task on to someone else, ever. When a follow-up came in, it went back through the lead, who decided again. On the frozen measures (agents sticking to an area, ownership surviving the sleep, change requests finding their owner), A was flat or slightly worse than the plain mesh. By the rules I wrote down beforehand, that's "doesn't work" for what A was meant to do. Strictly, that verdict needs a second wave to be final.
Two things surprised me, though.
First, the plain mesh wasn't the one-head team I predicted. I'd written down that one agent would make at least 75 % of the code. It was 34 % and 49 %. Without any treatment, work spread more than in part 3. A just made the lead's switchboard role total.
Second, the switchboard team wrote the best code. Its quality curve was 0.62; everyone else sat between 0.37 and 0.49, and the two identical plain runs differed by only 0.038 on that measure. It ended with 53 of 59 hidden tests and the best final quality score.
I have to be careful here. On the plain count of passing tests, A's lead over the plain mesh is +7.5 at step 30. My own rule says an effect must be at least twice the gap between the two identical runs, which is 10 tests. So the test count is a lead, not a result. The curve is the strong signal: +0.12, against a bar of about 0.08. A+B, the other team with handoff-first rules, was also ahead early: 41 of 42 tests at step 18, against 33–37 for everyone else. That's two teams pointing the same way, still at one run each.
My best guess at why: a lead that has to write every handoff writes down what "done" means for each one. In part 3, the best moment came when an agent stopped to check everything against the running app. Here that kind of checking got built into every handoff. That's a guess, not something this experiment tested.
B: budding never got a fair test
Budding needs a donor (someone who clearly knows an area) and an idle agent at the right moment. My rig checked for that only every fifth step. Each budding team got 2 copies; my own rule needed at least 3 before saying anything. In the budding-only team, 3 of the 5 chances found no donor at all. So every budding result in the main run is a rig failure, not evidence about budding.
One small thing to note for later: the budding-only team had the least central routing of all. Only about half its handoffs came from the lead, and agents started 30 handoffs themselves. That's a hint, at one run, with a broken treatment.
A direct test, broken differently
Since the main run never really tested budding, I tried a smaller, direct test. I took the saved teams from this wave and made pairs. In each pair, one idle agent got a copy of the session of whoever knew an area best (the "bud"). Another idle agent got a fresh session. Both got the same small change request in that area, starting from the same code. Twelve pairs were planned.
Two ran. My rig broke the other ten. Before each pair, it added a few safety rules to each agent's settings. In doing so, it replaced a link with a plain file, and the a4sx CLI rightly refuses to start an agent in that state. Worse, I had put each pair's code in a folder the agents' own workspace rules don't let them write to. All four agents in the two pairs that ran were blocked on their first edit. The two buds accepted the block and stopped. The two fresh agents found a way around it. So the pass/fail result says nothing about budding.
What the two pairs did show, at n=2:
| | Bud | Fresh | |---|---|---| | Steps before the first edit (bookings) | 1 | 4 | | Steps before the first edit (auctions) | 2 | 6 | | Length of its answer to "how does this area work?" | about 410 characters, naming the actual functions and where the data lives | about 150, generic |
So the copy brings the donor's map of the code with it, and the bud uses it straight away. That's what budding is for. Two pairs is a direction, not a result.
The second thing surprised me. One bud explained the block as "this session is authorized to write only in" the lead's folder. The lead was its donor. The copy didn't just inherit what the donor knew; it inherited who the donor thought it was. A bud needs to be told clearly who it is now and what its area is, or it will act as the donor, or refuse work because of the donor's rules.
I've parked budding there for now.
A + B: the lead that swelled
The team with both rules had the opposite problem from part 3's hands. Its lead kept everything. Its prompts grew to about 473,000 tokens of context, close to the model's limit. At step 19 (premium payments) it spent more than the 35M tokens per step I'd set as a ceiling, and the watchdog killed it, exactly as the rules said. The rest of the team was fine, just idle: nobody prompts the workers except the lead.
To finish the run, I restarted the lead on a fresh session that started from its own notes, without the swollen context. It then did steps 19 to 30 in 74 minutes, all on time. The team ended at 47 of 59 hidden tests, with a quality curve of 0.49: better than every team without handoff-first rules, worse than A alone. It was also the most expensive team, at $12.86 per passing test. So the ceiling stop wasn't the end of the team, just the end of that one oversized head. A lead that keeps everything costs a lot, and it takes a restart from notes to stop it costing more.
Solo is still the cheapest (here)
The solo agent finished 29 of 30 steps, passed 44 of 59 hidden tests, and cost $3.16 per passing test. The meshes cost $6.35 to $12.86. I predicted "at least 2× cheaper" before the run, and it came in at 2.01× against the best mesh. It was also the fastest. This is one workload, one wave, one run per setup, so I'm not claiming more than "in this setup, at this n".
The rig stories
The meshes did plenty of things I didn't plan for. Four of them changed how I run these experiments.
The agent that killed its own team. While fighting a stale test server, one agent ran this, with the description "Stop stale test servers and rerun tests":
The answer was "Exit code 137". The command killed every agent process with "run" in its command line, and every agent's command line has "run" in it. That included itself and its five colleagues. Only the parts running as root survived ("Operation not permitted"). The team was dead for five and a half hours until I restarted it. Next time the agents run as a user their own shells can't touch.
The request that waited 17 hours, then took 7 minutes. The scripted user holds a new request until the lead is idle. I restarted that script once to fix a bug, and the restart threw away one waiting request: "People miss things. Send emails…". The team sat idle for 17 hours, every agent politely waiting. When I re-sent it, the team finished it in under 7 minutes.
The DONE nobody heard. The script listens for "DONE S-21" at the start of a line. One lead wrote it in bold:
"DONE S-21 — premium refunds are on main as 3f4a616."
The script didn't recognise the bold, so it kept waiting. The lead then re-checked a colleague's work and said it again, and again, an hour apart:
"I re-verified w1's hardening myself rather than taking the report on trust…"
Three hours later I found it and fixed the pattern.
Fifteen minutes per five hours. The agents ran on gpt-5.6-luna through our router, which falls back to claude-opus-5 when luna is out. Luna barely reuses cached context, so each call cost about 86,000 fresh tokens. A five-hour usage window filled after about 400 calls, roughly 15 minutes of lab time. The whole wave ran in short bursts with long pauses. For part of the run, overflow went to Opus: 38–64 % of the tokens depending on the team. The analysis treats that as part of each team's behaviour, not as noise to correct for. A team that makes more calls gets more fallback.
What I take from it
- Telling the lead to hand off made it hand off. It didn't make anyone else feel responsible. Instructions change behaviour; they don't change structure.
- The team with the most central routing wrote the best code on one strong measure. Maybe a good switchboard is not the failure I thought it was. Or maybe it's one lucky run. The next wave has to tell.
- Budding is still untested. That's on my rig. The one thing I'd keep from the two pairs that ran: a copied session needs to be told it's someone new.
- Every one of these runs taught me more about my test harness than about agents. Experiments that need days of compute need a harness that can't be killed by its own subjects.
Next: let them hill-climb
The next idea flips the question. Instead of hoping ownership spreads, give each handed-off topic something to climb. When a topic gets big enough, the agent that takes it writes a small check from the user's own examples, a colleague freezes it, and the agent keeps changes only when the score goes up. A second agent keeps asking the others what's blocking them. And between runs, the mesh proposes changes to its own rules, which get tested on runs it has never seen.
Before spending a single token on that, there's a cheap test: write those checks from the user's examples and replay them against this wave's saved code. If they don't track the hidden tests, the idea dies right there.
Disclosure: screening-grade results from one wave, one run per setup (two for the plain mesh). Agents ran in Claude Code against gpt-5.6-luna, with claude-opus-5 as router fallback for part of the run, including most of A+B's restarted final stretch. Rig deviations (the restart after the self-kill, the lost request, the DONE pattern fix, the ceiling restart) are logged, and none changes a verdict. The direct budding test ran on gpt-5.6-luna only. It was logged as incomplete: 2 of 12 pairs, and its pass/fail result is invalid.