When to Trust the Recommendation and When to Probe

Quick answer: The recommendation attached to a suspicion is a starting point, and the decision to trust it turns on the evidence. When the trigger is clear, the code anchor is reproducible, and a runnable check backs it, you can act on it. When the path is uncertain, the reaching data is unknown, or the change sits in shared code, probe deeper and confirm with a real request. A confident recommendation with no evidence is still a hypothesis. Why the recommendation needs a decision The review returns a priority, evidence, and a recommendation, which is useful because it names a concrete next move rather than just a warning. It is also generated by a model that can miss defects or raise false suspicions, and it never runs the application, so accepting it without a check hands the merge decision to a process that cannot observe the result. This is not a reason to ignore it. It is a reason to know which conditions make a recommendation safe to act on and which demand a closer look. Trust versus probe Condition Safe to trust? Why Clear trigger on a named line As a lead The anchor is checkable Reproducible evidence attached More confidence A command and output back it Runnable check passes High The behavior is verified, not inferred Uncertain reaching data Probe The trigger may never fire Shared helper or table Probe The blast radius is broad AI-only claim Probe It stays a hypothesis until verified The first three rows build on each other, and the strongest case is a recommendation with a clear trigger and a passing runnable check. The last three rows are where the reviewer should slow down. How to decide on a real finding Read the recommendation and the evidence it cites, then confirm the anchor really exists. Check whether a reproducible command and raw output support the claim. If the path is clear and the check passes, act on the recommendation. If the reaching data is unknown, trace the inputs and callers before deciding. If the change is in shared code, follow every caller to size the blast radius. When the code cannot settle it, send a real request and assert the behavior. Take a recommendation to add a guard so archived notes survive a selection change. The anchor is clear, the evidence shows the filter runs on a path that receives archived rows, and a real request reproduces the deletion, so the recommendation is trusted and the guard is added. A second recommendation to refactor a shared helper has no runnable check and reaches six callers, so the reviewer probes, traces the callers, and finds one that depends on the current behavior, which changes the plan. The model-generated, model-trusted boundary The trust model keeps a hard line: the capability menu is model-generated but never model-trusted, and every action must sit in a fixed registry, with hallucinated references dropped. The same idea applies to a review recommendation, where the model can propose but the evidence has to confirm. An AI-only claim is always classified as a hypothesis, and the deterministic verdict passes only fully verified evidence, with no LLM involved in that judgment. Powerduck keeps the recommendation and the evidence side by side, so the reviewer can tell a verified lead from a confident guess. When the evidence and the recommendation disagree Sometimes the evidence points one way and the recommendation points another, and the evidence should win. A model can suggest a fix that the reproducible command does not support, perhaps because the guard the recommendation assumes is not present in the code. When that happens, do not average the two or trust the confident tone; re-read the raw output and resolve the symbols again to see which is grounded. If the evidence is inconclusive, the recommendation cannot promote it, and the finding stays open until a real request settles it. A recommendation is a proposed move, and the evidence is the permission to make it, and a disagreement is a signal to run another check rather than pick the answer you prefer. Keeping these two separate is what stops a polished suggestion from overriding a failing result. Honest limits Even a verified recommendation is bounded to the paths the check runs, and dynamic loading or external consumers can remain outside it. Static analysis can miss framework entry points, and no incoming reference does not make a file safe to delete. When evidence is partial, keep the verdict partial rather than promoting it, and confirm the rest with a real request. Frequently asked questions When can I act directly on a recommendation? When the trigger is clear, the evidence is reproducible, and a runnable check passes. What is the main reason to probe? Uncertain reaching data or a shared-code change, where the blast radius is not yet known. Is an AI-only recommendation worthless? It is a useful lead and stays a hypothesis until a dynamic or contract check backs it. The bottom line Trust the recommendation when the anchor is clear, the evidence is reproducible, and a runnable check passes, and probe whenever the path, the data, or the shared-code reach is uncertain. A model can propose confidently without being able to observe the result, so an AI-only claim stays a hypothesis. The evidence, not the tone of the recommendation, is what makes the next move safe. Read more in the AI code review guide.