Declared Degradation: A Fallback Contract for Agents That Lose Their Cloud Model

A scenario worth designing for (illustrative) The scenario below is illustrative and does not describe a real incident, customer, or deployment. A nightly job classifies a few thousand support tickets with a remote model, writes tags to a queue, and marks each row complete. When the transport starts timing out, a retry wrapper returns an empty tag list instead of raising an error. Dashboards keep rendering, the queue keeps draining, and nobody notices the missing tags until a weekly report looks wrong. That failure is not a model-quality problem. It is a missing contract between a capability and its degraded form, and a local-first design makes the contract explicit. The pattern below declares what each tool may do when the cloud path disappears, then tests that the declared behavior is what actually happens. Three tiers, three honest outcomes The important decision is not local versus cloud in general. It is which capabilities may leave the host at all, and what each one is allowed to return when the remote path is unavailable. A three-tier declaration keeps that decision reviewable in code review rather than buried in retry logic. Tier Leaves the host Allowed fallback Typical member local_only never local model or refuse PII scan, redaction, secret lookup cloud_preferred yes cache or local model ticket tagging, first-draft text cloud_required yes refuse cross-document synthesis, long-context comparison The value of the tier is that it converts an availability question into a residency question. A local_only capability cannot silently become a cloud capability because a retry looked convenient. A cloud_required capability cannot pretend to succeed with an empty payload, because the only legal outcome without transport is refusal. Step 1: Declare the contract next to the tool Keep the declaration in the same module as the tool so it moves with the code. The dataclass below is illustrative Python; it has not been executed for this article, and the resolver bodies are deliberately small. # illustrative only — not executed for this article from dataclasses import dataclass from enum import Enum class Tier(str, Enum): LOCAL_ONLY = "local_only" CLOUD_PREFERRED = "cloud_preferred" CLOUD_REQUIRED = "cloud_required" class Fallback(str, Enum): LOCAL_MODEL = "local_model" CACHED_RESULT = "cached_result" REFUSE = "refuse" @dataclass(frozen=True) class Capability: name: str tier: Tier fallback: Fallback max_staleness_s: int = 0 # 0 means cache is not allowed CAPABILITIES = { "pii_scan": Capability("pii_scan", Tier.LOCAL_ONLY, Fallback.REFUSE), "ticket_tag": Capability("ticket_tag", Tier.CLOUD_PREFERRED, Fallback.CACHED_RESULT, 900), "ticket_summary": Capability("ticket_summary", Tier.CLOUD_REQUIRED, Fallback.REFUSE), } Every capability now has exactly one legal degraded outcome, and that outcome is data rather than a comment. Reviewers can diff the table when a new tool is added, and a missing entry becomes a startup error instead of a runtime surprise. The max_staleness_s field forces the cache question to be answered before the cache exists. Step 2: Return outcomes the caller must handle A resolver that raises on some paths and returns None on others pushes the decision into every caller. Return a small outcome object instead, where status distinguishes success, degradation, and refusal, and source records where the payload came from. # illustrative only — not executed for this article from dataclasses import dataclass @dataclass class Outcome: tool: str status: str # "ok" | "degraded" | "refused" source: str # "local" | "cloud" | "cache" | "none" payload: dict | None = None def resolve(cap: Capability, transport_ok: bool, cache) -> Outcome: if cap.tier is Tier.LOCAL_ONLY: return Outcome(cap.name, "ok", "local", local_model(cap)) if transport_ok: return Outcome(cap.name, "ok", "cloud", cloud_model(cap)) if cap.fallback is Fallback.CACHED_RESULT and cap.max_staleness_s > 0: hit = cache.get(cap.name, max_age_s=cap.max_staleness_s) if hit is not None: return Outcome(cap.name, "degraded", "cache", hit) if cap.fallback is Fallback.LOCAL_MODEL: return Outcome(cap.name, "degraded", "local", local_model(cap)) return Outcome(cap.name, "refused", "none", None) Two details matter more than the code. First, side effects belong after the refusal check, so a refused job never half-writes a row. Second, refusal must be a distinct status, because a caller that treats it as an exception will eventually wrap it in a bare except. Idempotency keys on the write path keep retries safe when a job is resumed after a degraded run. Step 3: Flip the transport in a test, not in production The harness does not need a real network outage. It needs one switch that makes the cloud path unavailable, and a table of expected statuses per capability. Run the same job twice and compare outcomes against the declaration. # illustrative only — expected behavior, not observed output import pytest from agent_contract import CAPABILITIES, resolve EXPECTED_WHEN_DOWN = { "pii_scan": "ok", "ticket_tag": "degraded", "ticket_summary": "refused", } @pytest.mark.parametrize("name", list(CAPABILITIES)) def test_declared_degradation(name): cap = CAPABILITIES[name] out = resolve(cap, transport_ok=False, cache=EmptyCache()) assert out.status == EXPECTED_WHEN_DOWN[name] if cap.tier is not Tier.LOCAL_ONLY: assert out.source != "cloud" The check itself is ordinary: run pytest -q tests/test_degradation.py in your own repository. A pass should mean every capability was exercised at least once with the transport down, and no outcome claimed a cloud source while the transport was unavailable. Adding a new tool without a row in EXPECTED_WHEN_DOWN fails the suite by construction, which is the point. Where the remote path fits, and how to pick The degraded path only matters if the preferred path is worth keeping. Use a short decision table before adopting any remote endpoint, free or paid, and revisit it when terms change. Signal Favors local-first Favors remote Latency budget tight, interactive batch, tolerant Data residency secrets or regulated text public or already-redacted text Offline requirement host must finish alone network assumed available Model size needed small task, small model capability only a larger model provides When the remote path is worth keeping but billing commitment is the blocker, the free tier of the project described here is a reasonable place to measure the cloud_preferred tier before committing. The operator states that the project is open source and currently offers 10 million free tokens plus a free server option, and those terms can change, so confirm the current quota and server conditions on the project's official page before depending on them. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Limits, and who should skip this The harness proves that declared outcomes occur; it does not prove that fallback output is good. A cached tag from fifteen minutes ago may be acceptable for routing and unacceptable for billing, and no test will tell you which. Refusal also costs throughput, so a workflow that cannot tolerate dropped jobs should queue them for later rather than refuse in place. Skip this pattern if you have a single tool with no fallback and no residency constraint, because the declaration adds indirection without buying a decision. Skip it as well if nobody owns the tier table, since an unowned contract drifts faster than the code around it. Teams that already log per-call source and status in production have most of the value already and mainly need the expectations table. Verification checklist before you ship Every registered tool has a tier, a fallback, and a row in the expectations table. The transport-down run produces no ok outcome with a cloud source. Refusals are handled by the queue, not swallowed by a retry wrapper. Cache staleness limits are set by the consumer of the result, not the producer. The tier table and the free-tier terms both have a named owner and a review date. MonkeyCode offers free models; check the current product terms for availability and limits.