Reading Hebrew public tenders with LLMs: scanned annexes, moving deadlines and a strict trust order

Israeli public bodies publish procurement tenders in a lot of different places: municipal sites, ministry portals, company-owned pages, and PDFs attached to all of them. The listing page is often just a title and a date. Everything a supplier actually needs (what is being bought, who is allowed to bid, what to submit and by when) lives in attached documents, written in legal Hebrew, sometimes scanned, sometimes amended after publication. I build Tendy, which crawls those publishers, reads the documents with LLMs and turns each tender into a structured decision: what is being bought, can this supplier bid, what happens next. Most of the decisions that mattered are about restraining the model rather than prompting it harder. The pipeline in one paragraph Each crawled item moves through a state machine: received -> normalized -> documents_processing -> docs_processed -> structured -> published, with side exits for quarantine, past-deadline and duplicates. Every handler guards on the state it expects, so a redelivered queue job is a harmless no-op. Documents fan out into one job per attachment and fan back in at a barrier. Then one LLM pass structures the evidence, a separate cheap pass categorizes, and publish makes the tender public. Normalize: dead tenders must cost nothing The first stage uses no LLM at all. It cleans Hebrew text and parses dates in Asia/Jerusalem, because a date-only deadline is meaningless without a timezone, and "today" is still open. It also runs a deterministic expiry check on crawler-owned fields before anything expensive happens: // simplified sketch, not production code function expiryPrecheck(item: CrawlItem, now: Instant): "settle" | "continue" { if (!isFiniteBiddingProcedure(item.kind)) return "continue"; // registries stay open for years const deadline = parseDeadline(item.deadline, "Asia/Jerusalem"); if (deadline === undefined) return "continue"; // fail open return endOfDay(deadline) < now ? "settle" : "continue"; } It only fires on a confidently parsed, confidently past date. A settled record becomes a minimal archived page rather than being dropped, so it keeps its identity and version chain while never costing a download, an OCR page or an LLM call. Killing a live tender costs a supplier a bid, while letting a dead one through costs cents, so every guard is biased toward "continue". The same logic runs later, once documents are read. Many dead records arrive with no deadline on the listing, but their PDFs state a submission deadline that passed years ago. A gate at the fan-in barrier takes the latest date across all submission-deadline findings and settles the tender only if that date is more than 7 days in the past. Taking the latest date means a postponement recorded in the documents keeps the tender alive, and the grace period absorbs day/month ambiguity in dotted Israeli dates. Zero parseable dates means "unknown", not "expired". Documents: where most of the truth lives Each attachment is its own job: download with a timeout and retries, hash it, store the original, then extract text. Native PDF extraction goes first. If a document yields almost no useful text (under 100 characters), we render the pages to images and send them to a hosted OCR model. That path sits behind its own concurrency gate so a pile of scanned annexes cannot starve ordinary enrichment. Extracted text is split into page-aware chunks of about 24,000 characters with one page of overlap. The model returns findings as JSON, and every finding must carry a page range and a short verbatim excerpt. We validate each finding against its page range and keep the excerpt a true substring of its source page, which makes citations checkable later. Findings are deduplicated across documents, keeping every supporting filename. Two details saved us money and failures: If a substantive chunk returns no findings, a focused "rescue" prompt retries it, and a chunk whose output was cut off is split recursively into smaller page-aware slices. The rescue pass uses the same strict validation, so an honestly empty result stays empty. Output tokens dominated this stage's cost, largely the model retyping Hebrew we already hold on disk. A character budget per excerpt, scaled by materiality (more for eligibility and money, less for scope and contacts), cut output without dropping findings. A tender also waits for its documents. A transient download failure re-enqueues that one attachment for later, and the barrier will not advance while any attachment is pending. Structuring once against everything beats structuring early and again after a retry sweep. The barrier must count every pending row, including ones a cheaper read path might skip. Work that is never attempted produces no errors, so we also alarm on a rate of zero next to a denominator that is still moving. One structuring pass into typed fields A single LLM call receives crawler fields, normalized data and the deduplicated evidence, and returns a schema-validated object. Output is JSON, thinking mode is off, and recoverable schema drift is repaired deterministically before any retry. The design principle is that decision logic reads typed fields, never free-form Hebrew strings. Take eligibility (תנאי סף). The model must keep three things apart: { "thresholds": { "conditions": ["..."], // canonical bidder checklist "groups": [ { "operator": "all", "conditions": ["..."] }, { "operator": "any", "conditions": ["..."] }, // a real alternative route { "operator": "all", "applies_when": "bids for lot B", "conditions": ["..."] } ] }, "key_personnel_requirements": ["..."], // qualifications of a named role, not the bidder "scope": "..." // technical specs are scope, not eligibility } Groups preserve AND/OR logic instead of flattening it into a list, because "experience with X, or alternatively Y" is a different requirement from "X and Y". Lot-specific conditions only apply to bidders for that lot. Money is typed by business meaning too: a bid guarantee, a performance guarantee, a participation fee paid by the bidder and compensation paid to ranked bidders are different fields, and VAT inclusion is tracked explicitly. Anything that lives only in prose is invisible to filtering and matching. The threshold conditions guide shows how often Israeli tenders require experience, turnover or classification. Those requirements decide whether a supplier can bid at all, so they have to be machine-readable. When high-risk evidence is present, a second independent review pass returns a complete corrected object, not a verdict or a patch. Both prompts share a byte-identical prefix (system message and evidence) with pass-specific instructions last, so prefix caching covers the second call. The trust order Documents and listings disagree constantly, so we settled on a strict order: The publisher's current listing wins on identity (title, tender number, link) and on the current deadline. Grounded document findings fill gaps and add requirements, terms and risks, but never silently replace identity. Only an explicit later amendment can replace a current deadline. The LLM may describe a conflict. It may never invent a winner. Concretely, if a PDF says the deadline is the 20th and the listing says the 27th, we show the 27th. Getting this backwards means a supplier misses a bid they were preparing. The easy part to miss is enforcement, because a prompt instruction is a request, not a guarantee. The authoritative fields go first in the prompt, but deterministic code also rewrites the candidate's dates to their canonical values before review and again after every reviewer response, so a model cannot undo precedence. Each deadline records its source (crawler_current, amendment, document_original, not_stated), and a postponed deadline must clear any stale stored date. A final scan checks public copy for superseded dates; one focused revision may run, and a stale date that survives blocks publication. Versioning: amendments are data Publishers amend tenders, and we keep the history. Only the latest version is shown, and old versions inflate every count if you forget the filter. "Unchanged" has one definition: a hash over the publisher-owned fields of the raw item, canonicalized (attachment URLs sorted and stripped of rotating signatures, whitespace collapsed, dates normalized). Nothing we generate is hashed, because our prose differs on every run and hashing it minted versions for tenders nobody amended. Not every row is a tender, either: award notices, cancellations and job ads share the feed, so counts need a record-kind filter. Deterministic signals beat another LLM call Some of the most useful fields involve no model. The supplier-size signal (micro, small, medium, large) is computed deterministically, and only strong evidence sets a band: required turnover, contractor classification, guarantee size, budget, or required annual work volume. No strong signal means no band. Showing nothing is better than confidently mis-sizing a tender, and a deterministic rule is easy to explain. Categorization is likewise a separate, cheap pass over title, gist and keywords only, into a two-level taxonomy, and a failure there does not block publishing. Categories describe domain, not buyer: a municipality buying forklifts is material handling, not municipal services. Cost discipline and the testing trap LLM cost is a product constraint, so the rules are boring. Dead tenders never reach a model, identical files are reused by content hash, unchanged listings are dropped at ingest, and a prompt change is validated on 3 tenders, then 10, then a decision is made. We never re-enrich the whole corpus to see what a prompt does. The testing lesson is harder: mocked LLM tests do not catch provider contract errors. We once shipped a prompt that returned HTTP 400 on every real call while every test was green. Keep a few smoke checks that hit the real provider, and judge replays against a recorded baseline rather than accepting output merely because the schema parses. Hebrew-specific notes Everything is RTL-first, and RTL extraction quirks mean text-matching heuristics have to be conservative. Budgets appear as free text in some places and typed numbers in others. NUL bytes from PDF extractors crash PostgreSQL text and JSONB columns, so sanitize before persisting. Lessons Enforce source precedence in code, not just in the prompt. Make "unknown" a first-class value. Missing information is not a failure, and no signal should mean no output. Put expensive work behind cheap deterministic gates, and bias those gates toward fail-open. Require page ranges and verbatim excerpts for every extracted fact so you can check citations. Type your fields by business meaning, and let decision logic read only those fields. Alarm on work that never happened, not just work that failed. Test against the real provider at least once, because mocks share your assumptions. Validate prompt changes on a handful of records before touching the corpus. If you want to query this data from an AI assistant, there is a public MCP server, documented at github.com/ofir09/tendy-mcp. I would like to hear how others handle conflicting sources in document pipelines.