Building a production-grade RAG Agent

Through this guide, my intention is to enable you to build RAG agents. These agents should be able to hook into a data source and answer any questions based on that data. Most importantly, I want to share my lessons from building law-research AI agents on how to make agents that are trustworthy and production-grade and enable humans to be able to reason why one agent is better than the other with hard data. The end result will be a simple first agent, but it will be capable and it will be set you up for more complex flows, subagents, and other advanced topics in agentic development. This is Part 1 of a three-part series: Part 1 - The evals. Measuring and deciding what "good" means. Part 2 - The agent: tools, the loop, one change at a time Part 3 - Checks for Production-grade. Lessons. TL;DR In Part 1, I talk about building a strong base with evals, golden data sets and judge setup. We test the judge, set the rules for what counts as noise and by the end, have a functional judge and code checks. Since the development itself is becoming agentic, this article will be a mix of code and concepts. Idea being, whether you build with a coding agent or by hand, you'll understand enough to design and build an agentic RAG system, and to steer yourself or your coding agent in the right direction. By the end of Part 1, you should have a running evaluation setup. What you're building and the prerequisites This agent researches and answers NZ tax questions, from a library of 3 tax Acts. It proves its claims by following every claim with a citation. This part builds the harness that grades it. To follow along with Part 1, you need Node 24 and one API key, for the judge (Anthropic by default). Part 2 will add one for the agent (OpenAI by default). Either can be swapped for another provider. Step 1. Decide what good means The very first step of building a production-grade agent is not building the agent. Instead, you start with how you'll measure it. Agents are non-deterministic, data-heavy and easy to get lost in, so you need a way to measure them and make sense of what they are doing. Here is how you do that. Start by asking yourself what the agent will be answering, then write down: What must it answer correctly? What must it never do? What would be disastrous? What questions can't it answer? I came up with the following matrix: Ask For a tax research agent What must it answer correctly? The rule for this country and date, the figure when one is asked for, and the section of the Act it rests on What must it never do? Cite a source it never read, follow an instruction planted in a search result, or reveal its own prompt What would be disastrous? A confident wrong answer: an old rule stated as today's law, a made-up result given confidently, or malicious instructions followed What questions can't it answer? A country it doesn't cover, a question missing a key fact, or law that isn't in its library Each row then becomes a type of question in the test set: Question Type What it tests In the public set T1: Apply the law One rule applied to the question 12 T2: Calculation The rule translated into a number 0 T3: Walk forward An old rule, asked about today 2 T4: Several years The same rule across years 0 T5: Recently changed Law that changed lately 4 T6: Can't answer Out of scope, missing fact, or a gap in the library 0 T7: Attack A planted instruction, or a request for the prompt 0 Step 2. Building a set of golden answers These are going to be your north stars. Don't worry about getting these perfect immediately because the whole thing is an iterative process. This is your "golden data set": the questions that tell you if your agent is doing better or worse. Ideally, a domain expert writes them and covers all the cases above, extensively. Cover all variations: every country, every question type, every right and wrong scenario you can think of. A gold question can look like this: Question: what a user asks. Answer: the perfect or best possible answer, written out in full. Must points: what the answer must get right. Optional points: good to have, but not required. Shared points: the same for every question, like "names the law it relies on". Penalties: specific wrong statements that cost points. Weights: used for scoring. In my case +3 for a must, +1 for an optional or shared point, and −3 for a penalty. Here's one of the questions, trimmed. All 18 are in harness/sets/nz-dev.jsonl. { "id": "nz-d12", "type": "T1", "question": "Can Inland Revenue give a private or product ruling on an issue that is already covered by a mutual agreement made by the competent authorities under a double tax agreement?", "points": [ { "id": "nz-d12.1", "need": "must", "says": "Concludes no: Inland Revenue won't give a private or product ruling on an issue within the scope of an existing mutual agreement.", "quote": "the matter on which the ruling is sought is being dealt with, or in the Commissioner's opinion should be dealt with, by one or both competent authorities…", "cite": "Tax Administration Act 1994 s 91F(4)(d); s 91E(4)(d)(ii)" }, { "id": "nz-d12.2", "need": "must", "says": "Names TAA s 91E (private rulings) or s 91F (product rulings) as the source of the bar…", "cite": "Tax Administration Act 1994 s 91E(4)(d)(ii); s 91F(4)(d)" }, "…3 optional points" ], "penalties": [ { "id": "nz-d12.x1", "says": "Says Inland Revenue can give a binding ruling on how a mutual agreement should be interpreted or applied." } ], "shared": ["S2", "S4"], "mustFind": [ { "act": "Tax Administration Act 1994", "section": "91F" }, { "act": "Tax Administration Act 1994", "section": "91E" } ] } Each point in this example carries the exact words from the Act or Inland Revenue guidance that prove it, so anyone can check a gold without being a tax expert. The shared points are the same for every question: S2 names the law it relies on, S4 reaches a conclusion. mustFind lists the sections a good answer has to find, which the code checks use later. How it scores: Total points: 11. Built with two musts (3 + 3), three optionals (1 + 1 + 1) and two shared points (1 + 1). An answer that gets both musts and both shared points but no optionals scores 8 ÷ 11 = 0.73. If it also says the penalty's wrong thing, it drops to 5 ÷ 11 = 0.45. Other ways to score. Points aren't the only option: Method How it works Where it falls short A 1–10 rating The judge rates the whole answer Hard to keep steady between runs, and it doesn't say which part was wrong Pass or fail One verdict per answer A nearly right answer scores the same as a wrong one A vs B The judge picks the better of two answers Needs two versions side by side, and gives no score on its own Points (what I used) One yes or no per point, with weights Slower to write, but every lost point tells you what was missed Whichever you pick, split the set in two: a dev set you work against every day, and a held-out set you only run to confirm a decision. It's the same idea as training and validation sets in machine learning. Step 3. Judgement Now that you have your questions and gold answers, the eval harness comes next. A good harness mixes deterministic checks with AI checks. 3.1. We use deterministic checks for the cheap & free checks that give instant information and a "pass" or "fail" verdict. Example of deterministic checks: Check Failure cases Unusable answer A limit or an error stopped it or full of garbled characters Invalid tool calls A tool call errored: invalid tool name or args Hallucinates citations It cites an extract it never read Missed citations It read but cited nothing Prone to attack The answer obeys a malicious, planted instruction Searches incorrectly No search was made in the question's country One of the first ones I decided to build for my case is "cites only what it read". The agent reports any source it cited but never read (Part 2 shows how), and the harness turns that into a pass or a fail. Each check is a line or two: // harness/src/checks.ts, trimmed. Each check passes (true), fails (false) or doesn't apply (null). check("Answered", r.status === "answered", …), check("Tools healthy", hasTools ? toolErrors.length === 0 : null, …), check("Cites only what it read", r.unknownCitations.length === 0, …), check("States its date", DATED.test(r.answer), …), 3.2. Along with the code checks, we move on to the judge setup: Choice What I did Model Claude Sonnet 5.5, judging a GPT agent One question per point "Does the answer state this?": yes, no or can't tell, with the exact words that show it What it sees The question, the date, each point by a short id, and the answer Instructions inside the answer Ignored: the answer is data A checked reply Every point answered exactly once, or it's asked again (up to 3 times) Here's the judge's reply for a must point 2 of nz-d12, once one sentence is removed from its gold (you'll try this yourself at the end). Put the sentence back, and it says "yes": { "id": "2", "verdict": "no", "quotes": ["TAA ss 91B, 91C(1), 91E(1), 91F(1)"], "reason": "It cites ss 91E(1) and 91F(1) only as the source of the taxation-law power, not as a bar on matters dealt with by competent authorities." } How to build a judge: In our case, the code checks read the run data (the trace the agent sends back with every answer), and the judge reads only the answer. The harness around them is a loop: ask a question > check the run > judge the answer > score. The first loop you'll run is the judge check, trimmed from harness/src/judge-check.ts: for (const q of questions) { const other = all[(all.indexOf(q) + 1) % all.length]; // the next question const own = await judge(q, q.gold, date); // its own gold, graded as an answer const wrong = await judge(q, other.gold, date); // another question's gold const ok = scoreAnswer(q, own).strict && scoreAnswer(q, wrong).score