How Is a Multi-Agent System Actually Designed?
Chanwoo Song 9 min read
In AXIT, a workspace where teams hand work to AI agents, no more than 9 tasks ran at the same time in the 30 days up to October 20. When we split 14 days of activity into minutes on October 10, nothing was running 49% of the time, and exactly one task was running 34% of the time.
“Multi-agent” usually brings to mind dozens of agents running at once, and it has become a familiar sight to see Claude Code or Codex spin up several sub-agents to work in parallel. The numbers show far fewer tasks running concurrently than that picture suggests. Open up a single task, though, and you find AI working through it in sequence, changing roles as it goes.
AXIT, which we build, is a workspace where AI agents plan and carry out work handed to them in a team channel, then post the result back into the same conversation. I'm going to walk through how it is put together over a few posts. This first one covers how many AIs work between one message and its result, and in what order.
Each task gets four AI roles, but only one execution agent works in the task folder at a time.
Counted by role, there are four. Counted by execution processes holding the task folder, there is one. Of the four roles (planning, execution, supervision and review), planning, supervision and review are short calls that run without write access in an empty temporary folder rather than the task folder, so only execution actually changes files. What we added was roles inside a single task, not agents running at the same time. Concurrency follows the same principle: a channel runs one task at a time, and only separate channels run concurrently. My September post ‘Does running more agents actually help?’ (in Korean) concluded that you should keep a single writer, split only the judgment, and reserve parallelism for work that doesn't touch shared state. AXIT has the same shape.
Every message passes through four stops: storage, planning, execution and result.
When someone hands off work in a channel, the control plane, the server that receives requests and manages tasks, first stores the message. It then calls an AI engine once to make a plan. That call decides what to do and in what order, and returns a list of steps. Once the plan is in place, the actual work moves to a worker. The progress indicator on screen only advances when the executing agent itself reports the start and end of each step.
Execution is a CLI process that runs for tens of minutes in a disposable container.
A worker is a container started fresh for every run. It connects to the control plane over a single WebSocket, receives the execution plan, runs the CLI of the engine the workspace has chosen, and sends progress events and results back over the same socket. One agent can keep running inside it for tens of minutes.
We moved execution off the server for isolation and stability. In mid-September, a single agent attempt pushed server memory from 0.56 GB to 13.22 GB. In that setup, one person's task could slow down the screens of everyone else on the same server. Workers became the default on October 16, and 98% of the tasks executed from October 17 to 28 ran on workers. While a run lasts tens of minutes, though, the server can stop or be replaced partway through. That makes deciding who owns a run at any given moment harder than it sounds.
Two AI calls that only judge keep watch on execution.
When you hand work to an agent, it sometimes stops after investigating, or reports work it never did. So separately from the executing agent, we run short calls that only make judgments. The executing agent's instructions also tell it to confirm the actual success output before saying it pushed or opened a PR.
If execution has been quiet for more than 60 seconds, the supervisor steps in. It reads the last 8 lines of the work log and leaves a single line beginning with “OK:” or “Caution:”, without stopping the run. A quiet agent may simply be thinking through something long, so the 60-second timer starts over after each check. The supervisor runs at most three times per task and only one at a time per workspace. Time spent waiting for a person to answer a question card, or for a server or worker to become available, doesn't count as silence.
Review runs when execution finished successfully but some steps were never reported as complete, or when the task has completion criteria. It looks at the request, the completion criteria and the end of the report, and answers only “Done” or “Incomplete: <reason>”. If the answer is incomplete, the task continues in the same task folder up to two more times. If the same reason comes back twice in a row, review treats that as no progress, stops, and marks the result as incomplete. We decided that flagging it was better than rerunning without end.
Neither is a new model. Each is just one more call to the workspace's main engine at a low reasoning level, and from October 17 to 28 the supervision and review calls that ran on Claude took a median of 5 seconds each. Answers that don't match the format are discarded, so instructions mixed into the execution log can't pass themselves off as a verdict. If a judgment call itself fails, the task is let through rather than blocked. Review, however, only sees the agent's report. It can't tell whether the numbers in the output match the source, and we haven't yet counted how often review actually returns “Incomplete”.
Once past review, the result goes back to where the instruction came from. In a talk channel, where people chat with each other and call in @AXIT to hand something off, the plan, progress and result appear in turn in the thread of that instruction. The result shows up immediately on any open screen, and phones get a push notification.
The unit of parallelism is the channel, not the agent.
Each channel has one task folder, so a channel executes one task at a time. When a second task arrives in the same channel, it gets as far as planning, then waits with the message “Waiting for the earlier task in this channel to finish.” An execution lease written to the database decides who holds that slot, so even with several control planes, two agents never attach to the same folder. Tasks in different channels don't wait on each other for a folder.
Running several servers did produce a bug. In late September, in production, a session copy restored by a server that wasn't the owner grabbed a channel's slot, and three times the next task in that channel failed because the folder was supposedly in use by another task. We've since fixed it so copies no longer take the slot.
Actual overlap is rare. In the same 14-day distribution, two tasks were running at once 11% of the time, three 3.5% of the time, and four or more 2% of the time, with a peak of 8. Moving the window to the month starting September 28, the busiest moment was 10 tasks, on the afternoon of October 24 during a training event.
This shape changed how we design for people, too. When the team put together a hands-on course, it had participants alternate between two work channels (channels where each post becomes an instruction for an AI task). Building one web page takes a median of 17 minutes, so while one channel was running, participants wrote the next instruction or reviewed results in the other. Their first hand-off was a reply in a talk channel, which doesn't use a worker and answers in a median of 14 seconds. Putting everything into one channel only makes the queue longer. In the end, how wide parallelism goes depends more on how people divide their channels than on the number of agents.
At Uniquegood Company, which builds this product, we work with agents too.
Over roughly 25 hours, from 11:00 on October 28 to 12:00 on October 29, counting only the workflows (scripts that launch several agents in stages) run for company work in a single Claude Code session, there were 15 workflows and 104 agents. At peak, that session had 6 workflows and 14 agents working at once. Group the stage names and a few patterns keep repeating: implement→review→fix, sweep→verify→critique, research→recommend→rebut, build→judge. We also settled on a handful of rules. The agent that looks for problems and the agent that tries to break things are different agents, fixes happen only once, and people decide merges, deploys and anything sent outside. Sub-agents can't see our conversations with people, so a secondhand “they said to do it” never counts as approval. As the number of agents grew, so did the number of things people had to decide. I'll cover that, along with the question of who owns a run, in the next posts.
For teams bringing agents into their work, it may be worth reversing the usual order. Before adding agents, separate the side that writes from the side that judges. Split parallel work along units with no shared state, such as channels or folders, and give judgment calls a narrow answer format and a cap on how many times they run. Cap reruns too, and when there's no progress, it's better to stop and leave a visible mark. Then, however many agents are running, you can still trace who changed what.
The next round of competition in multi-agent systems will probably turn less on how many agents you launch than on where you split the work and who makes the judgment calls.