Building a Production AI Agent in Spring Boot: The Supervisor Pattern with Specialist Agents (Part 5)

"Can I return the blue jacket and use the store credit to order the same one in a large?" The tester from Parts 2, 3, and 4 again. That one sentence touches four systems: the order lookup from Part 1, the return policy, the store credit balance, and the product catalog for the size swap. My agent had nine tools for exactly these jobs, and it still fumbled. It picked the wrong tool twice. It searched products when it should have read the return policy. Then it quoted a policy line that belonged to a different category. And nothing in the logs said why. The system prompt had grown into a wall of rules: "use this tool when the shopper gives concrete criteria", "use that tool only for returns", "never confuse store credit with refunds". Every rule I added fixed one incident and made the next question slightly harder. That is the wall every single-agent system hits. This part of the series is what I did about it. I stopped asking one agent to know everything, and turned it into a supervisor that hands work to specialist agents. I am a Senior Software Engineer II at BS23 in Dhaka, and I have been building production AI agents with Spring Boot and Spring AI for over a year. Everything below comes from the same e-commerce agent as Parts 1 through 4: same project, same tools, same memory, same observability. The only change is the shape of the system. When One Agent Stops Scaling A single agent works until it does not, and the failure is gradual. Three things degrade together, so you rarely notice any one of them. The system prompt becomes a rulebook. Every new domain adds a paragraph of instructions. The prompt grows past what a model can apply consistently, and the rules start fighting each other. The "use this tool when the shopper gives concrete criteria" fix from Part 4 was a symptom of this: I was patching tool-selection mistakes with more words, and the words themselves started confusing the model on edge cases. Tool selection gets worse, not better. The model sees all nine schemas on every call. More candidates mean more near-misses, and a near-miss on a checkout tool is expensive. Part 4 showed me this directly: the trace had semanticSearchProducts firing on keyword queries where searchProducts with a filter was right. The model was not broken. It was choosing from a menu that was too big for the question. Memory pollutes across domains. One chat memory holds order history, product queries, and policy questions in the same thread. The sliding window from Part 2 then drops useful context because a shopping session filled it with policy chat. The right context gets evicted by unrelated context. The fix is to shrink every decision the model makes. Give each model fewer tools, a narrower system prompt, and a memory that only holds its own topic. The Supervisor Pattern, in One Paragraph Instead of one agent with nine tools, you run three agents with three tools each, plus one agent on top that does no real work at all. The supervisor reads the question, picks the specialist that owns that topic, hands the task over, and relays the answer back. The specialists never see each other and never see the supervisor's reasoning. Each specialist has its own system prompt, its own tools, and its own memory namespace. If a question needs order lookup and product search, the supervisor calls the order specialist, calls the product specialist, and combines the results. Spring AI gives you the exact building block for this. From the tools reference, you can define a ToolCallback from a ChatClient, and the docs name the use case directly: "to build a modular agentic application". A specialist agent is just a ChatClient with a narrow system prompt, and a supervisor is a ChatClient that exposes those specialists as tools. Step 1: The Specialists Are Just ChatClients A specialist is the agent from Part 1, minus the ambition. Same ChatClient.Builder, same @Tool methods, same memory advisor. The difference is scope. @Bean ChatClient orderAgent(ChatClient.Builder builder, OrderTools orderTools, ChatMemory chatMemory) { return builder .defaultSystem(""" You handle order questions only: order status, delivery dates, and return eligibility for orders. Use the order tools. Never answer product or account questions yourself. If the question is off-topic, say the order specialist cannot help. """) .defaultTools(orderTools) .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build()) .build(); } @Bean ChatClient productAgent(ChatClient.Builder builder, ProductTools productTools, ChatMemory chatMemory) { return builder .defaultSystem(""" You handle product questions only: search, price, stock, and product details. Use the product tools. Never answer order or account questions yourself. """) .defaultTools(productTools) .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build()) .build(); } Three things are deliberate here. The system prompt says what the agent is not. "Never answer product questions" is as important as "use the order tools". A specialist that stays silent on off-topic questions forces the supervisor to route correctly, instead of the specialist guessing. Each specialist gets only its own tools. The order specialist's model never sees the product schemas. That is the whole point: the tool menu the model chooses from went from nine entries to three. Memory is shared but namespaced, which I get to in a moment. Build one of these per domain. My project ended up with order, product, and account specialists, but the pattern is the same for any split you choose: returns, pricing, support, whatever your domain says. Step 2: The Supervisor Delegates Through Tools The supervisor is a ChatClient whose tools are the specialists. I implement each delegation as a @Tool method that calls the specialist client and returns its answer as the tool result. @Service public class ShoppingSupervisor { private final ChatClient supervisorClient; private final ChatClient orderAgent; private final ChatClient productAgent; private final ChatClient accountAgent; public ShoppingSupervisor(ChatClient.Builder builder, ChatClient orderAgent, ChatClient productAgent, ChatClient accountAgent) { this.orderAgent = orderAgent; this.productAgent = productAgent; this.accountAgent = accountAgent; this.supervisorClient = builder .defaultSystem(""" You are the supervisor of a shopping support team. You have three specialists: orderAgent, productAgent, and accountAgent. Read the user's question, choose the one specialist that owns the topic, and call only that tool. Do not answer the question yourself. If the question needs two specialists, call both, then combine their answers for the user. """) .defaultTools(this) .build(); } @Tool(description = "Handle order questions: order status, delivery, and return eligibility for an order") public String delegateOrder( @ToolParam(description = "The user's full question") String question, ToolContext toolContext) { return orderAgent.prompt() .user(question) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, specialistConversationId(toolContext, "orders"))) .call() .content(); } @Tool(description = "Handle product questions: search, price, stock, and product details") public String delegateProduct( @ToolParam(description = "The user's full question") String question, ToolContext toolContext) { return productAgent.prompt() .user(question) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, specialistConversationId(toolContext, "products"))) .call() .content(); } public String chat(String userMessage, String conversationId) { return supervisorClient.prompt() .user(userMessage) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId)) .toolContext(Map.of("conversationId", conversationId)) .call() .content(); } } Two details in this code are worth more than the pattern itself. The conversation ID is namespaced per specialist. The supervisor stores its own turns under the raw conversationId. Each specialist stores its turns under orders: or products:. Without the prefix, the order agent's memory and the product agent's memory would write into the same window and evict each other, which is exactly the pollution problem we are trying to kill. With the prefix, each specialist has a private thread that only grows with its own topic. This is the memory-namespacing trick, and it is the difference between a supervisor that feels coherent and one that forgets mid-conversation. The conversation ID travels through ToolContext, not through the prompt. From the tools reference, ToolContext is data you attach to a chat call that the model never sees. It is the right channel for plumbing: conversation IDs, tenant IDs, user IDs. If you put the conversation ID in the prompt text, the model can quote it, drop it, or hallucinate a new one. Through ToolContext, it is passed to the tool method directly, untouched by the model. The supervisor's system prompt is where the routing rules live. Mine is deliberately short: pick the one specialist that owns the topic, call both only when the question needs two, never answer yourself. The routing decision is now a single decision the model makes with a three-item menu, instead of a nine-item menu plus a rulebook. What the Model Sees, and What It Does Not With this design, the model surfaces are clean at every level. The supervisor's model sees three tools: delegateOrder, delegateProduct, delegateAccount. It never sees product schemas or policy text. Each specialist's model sees its own tools only. The order agent does not know the product catalog exists. ToolContext carries conversationId without any of it entering a prompt. The specialist answer comes back as the tool result, and the supervisor formats the final reply to the user. That last hop matters. The supervisor's final answer is where the user's message gets its tone and structure. The specialist returns facts; the supervisor returns prose. Keep the specialists terse and let the supervisor do the talking. One option to know about: the returnDirect attribute on tools, also documented in the tools reference. By default a tool result goes back to the model for post-processing. With returnDirect, the result goes straight to the caller and ends the loop. For a RAG-style specialist that produces a complete answer, returnDirect saves a model round trip. I do not use it here, because I want the supervisor to combine and rephrase, but it is the knob to reach for when your delegation result is already final. Keeping the Chain Observable Everything from Part 4 still works, with one adjustment to how you read it. Each ChatClient call produces its own set of observations. The supervisor's call is one ChatClient observation. Each delegation tool call is an observation inside the supervisor's span. Each specialist call is its own nested set: model call, token usage, tool calls inside the specialist. The observability reference covers the metric names, and they do not change when you add a supervisor. What changes is that you now get a span tree instead of one flat span. Two things made that tree readable in practice. Name your chat clients. If you leave the defaults, every span in the tree says ChatClient, and you cannot tell the supervisor from the order agent. Set the client name in the builder so the trace reads supervisor, orderAgent, productAgent. The one-line change is worth more than any dashboard. Keep the conversation ID on every span. Because the ID rides through ToolContext and the advisor, the supervisor span and all nested specialist spans carry the same conversationId attribute. That is how the 40-second question from Part 4 becomes a 40-second question with a shape: you filter by conversation ID and read the tree top to bottom. The alert from Part 4 needs a small rethink. The AgentReplySlow alert measures the supervisor's ChatClient duration, and that number now includes specialist calls. For the user-facing question that is correct: the user waits for the whole tree. But when it fires, the first thing you do is look at which child span ate the time, not assume the supervisor model is slow. In my run, the answer to "which link is slow" was almost always a specialist call, because the specialist's model call is the same latency as before, just nested one level down. The Honest Cost Section The supervisor pattern costs real money and real latency, and you should price it before you build it. Every delegated question is at least two model calls. The supervisor reads the question and picks a tool, then the specialist answers. A two-specialist question is three calls. The supervisor's routing call is not free, and on a small model it is the one most likely to misroute. Latency roughly doubles for delegated questions. Part 3 measured 10 to 12 seconds for a full answer on the single agent. With a supervisor, plan for that plus one routing round trip. Streaming still helps perception, but the first token now waits for the routing call first. Tokens add up on the supervisor side. The supervisor's context is small, but it runs on every question, even the ones a specialist could have answered alone. For high-volume, single-domain traffic, a direct route to the specialist is cheaper than a supervisor round trip. When does the pattern pay for itself? When the domain genuinely has multiple topics with distinct tools and distinct memory needs, and when wrong-tool incidents are costing you real debugging time. For a single-purpose agent with four tools, do not build a supervisor. The rule I use: split when the system prompt needs a section per domain, and not before. Four Pitfalls I Hit The supervisor tried to answer instead of delegating. On the first version, the system prompt said "call the right specialist", and the model still answered easy questions itself. The fix was the explicit negative: "Do not answer the question yourself." The model treats that as a hard boundary, and it works. The supervisor called every specialist for one question. With three tools visible, the model occasionally called all three "to be safe". The fix was pricing it into the prompt: "call both only when the question needs two" was not enough, so I added "If one specialist is enough, call exactly one." Test your routing prompt with a handful of real questions before you trust it. Memory pollution between supervisor and specialists. Before namespacing, the supervisor's turns and the order agent's turns shared one window, and the agent forgot the jacket question while answering the policy question. The orders: prefix from Step 2 fixed it and is the single highest-value line in this part. Latency surprises in the traces. The first time the alert fired, I blamed the specialist model. The span tree showed the specialist call was fast and the supervisor's post-processing was slow, because I had asked the supervisor to summarize and rephrase long specialist output. Trimming the specialist replies to facts made the final hop cheap. Keep specialist output short; the supervisor is not a summarizer. The Supervisor Checklist If you take nothing else from this part, take this list. Split by domain, not by volume. One specialist per topic with its own tools and its own system prompt. Shrink the tool menu. Every model should choose from three tools, not nine. Say what the agent is not. "Never answer X questions" belongs in every specialist prompt. Namespace the conversation ID per specialist. orders:, products:. Without this, memory pollutes. Thread the ID through ToolContext, not the prompt. The model cannot corrupt what it never sees. Name every ChatClient. The span tree is useless when every span says ChatClient. Keep the supervisor's routing prompt short. One decision, a three-item menu, and a hard "do not answer yourself". Price two round trips per question. The supervisor call is real cost and real latency. Expect wrong-tool bugs to move. They do not disappear, they move to the routing decision. Test routing with real questions. What Comes Next The supervisor changed how the agent scales, but it also changed how it fails. Next part is about testing the whole thing: how I regression-test an agent loop with tool calls, memory, and delegation, so the tester's questions stop being the test suite. Have you split an agent into specialists? What broke first when you tried? I read every response. I write about Java, Spring Boot, and AI agents every week. Subscribe, it's free. Bookmark this one. The checklist is the part you will re-read the week the supervisor starts answering questions it should delegate.