My AP agent only trusts what Hindsight remembers

An invoice from our packaging supplier arrived 4.52% above the purchase-order price, with a note: "revised rates due to kraft paper increase." The right call was to hold it. Their contract allows a surcharge of up to 2.5%, and nothing more. My agent got that right for one reason only: it remembered a single decision from weeks earlier, when a 4.26% variance from the same vendor was sent back to procurement. When that one memory went missing, the agent stopped being sure and handed the invoice to a person. That behaviour is the whole design. I built Precedent, an accounts-payable exception agent, around one rule: the agent may only be as confident as its memory allows. The memory is Hindsight, an open-source agent memory system, and it turned out to be the most important part of the stack. The problem: the same exceptions, every month Every mid-size company pays vendors through a 3-way match: purchase order, goods receipt, invoice. The ones that don't match land on an AP executive's desk. What surprised me is how repetitive they are: The packaging supplier adds a diesel surcharge, allowed up to 2.5% by clause 7.2 of their contract. The grain trader's ERP bills in quintals against POs written in kilograms. The logistics vendor invoices the full PO on the first partial dispatch. The label printer re-sends unpaid invoices with /R on the end. Someone sends a polite letter: "our bank account has changed, please remit to the new one." The right call is obvious if you remember how it went last time. Usually exactly one senior person does. A stateless LLM doesn't help: give it the invoice, the PO and the policy, and it writes a thoughtful paragraph and escalates everything. Clause 7.2 isn't in the documents. It's in the history of decisions. How it hangs together ![Precedent architecture: the agent recalls precedents from Hindsight, decides, and every outcome is retained back into memory] A deterministic 3-way match flags the exception: price variance, quantity, unit of measure, GST type, unplanned charges, duplicates, bank-detail changes. The agent then does three things: Recall precedents from Hindsight, scoped by vendor and exception type. Decide: an LLM (Groq, gpt-oss-120b) reasons over those precedents and submits a decision through a function call. An auditable reasoner checks that decision against the evidence. Retain: whatever happens, the outcome goes back into Hindsight. That includes a person overriding the agent, with their reason. Above 75% confidence, the agent clears the invoice. Below that, a person confirms with one click. Bank-account changes always go to treasury, whatever the confidence. Tags are the schema Hindsight's retain, recall and reflect operations are simple, but the real design work was deciding what a memory is. Hindsight extracts facts from natural language, which is great for reflection. I also needed to recall precisely, so every resolved exception is written as prose and tagged like a database row: "content": f"AP exception resolved on {inv['received_at']}. Vendor {inv['vendor_name']} ... " f"Exception: {primary.label} — {primary.summary} " f"Decision: {ACTIONS[final_action]} by {decided_by}. Reason: {reason}", "tags": [f"vendor:{inv['vendor_id']}", f"exception:{primary.type}", f"action:{final_action}", f"mag:{mag}", f"outcome:{outcome}", "kind:case"], "document_id": f"case-{inv['invoice_id']}", The prose lets Hindsight build good facts and observations: "Shree Balaji's surcharges are approved under clause 7.2". The tags let the agent ask exact questions: this vendor, this exception, what was decided. The document_id matters more than I expected. Hindsight extracts several facts per document, so without de-duplicating by document, one case would vote three times. Confidence comes only from memory This is the part I'm most opinionated about. With an empty memory bank, every exception scores 0% and goes to a person. Autonomy is earned one resolved case at a time. For price variances and extra charges, the reasoner learns the vendor's tolerance as a threshold rather than a vote. Approvals set a ceiling, holds set a floor, and the gap between them is a grey zone that always goes to a human: if ok and m