Two Fraud Reports Shared a Phone Number. Why That Wasn't Enough.

Building safer identity resolution and crime pattern intelligence in Axon OSINT. Imagine two people reporting completely separate online scams. The first lost money after communicating with someone who promised unusually high investment returns. The second received messages from someone pretending to represent a legitimate company. Different victims. Different stories. Different circumstances. But one detail appears in both reports. The same phone number. At first glance, this looks like an obvious connection. Two reports. One shared identifier. Potentially the same operation. A developer could write a matching function, connect the records, and display a relationship in an investigation graph. Problem solved? Not quite. What if the number was spoofed? What if it had been reassigned? What if both reports copied information from the same unreliable source? Suddenly, a seemingly straightforward technical task becomes something much more serious. The system might be correct about the matching data and completely wrong about what that match means. This illustrative scenario captures one of the hardest engineering problems we encountered while building Axon OSINT. The Problem With Being Technically Correct Most developers are familiar with deduplication. You normalize incoming data, compare identifiers, and determine whether two records belong together. For ordinary business software, this might mean combining duplicate customer records. In fraud intelligence, the consequences can be very different. A shared email address might suggest a connection between incidents. A cryptocurrency wallet might appear in several reports. A domain might be associated with multiple suspicious websites. But none of these observations automatically establishes who controls the identifier, whether two activities share an operator, or whether a particular person committed a crime. Consider an IP address. It may be associated with a server used by several unrelated services. A phone number may be spoofed or reassigned. A username may be used by entirely different people on different platforms. And even when two reports contain exactly the same information, those reports may not represent independent confirmation. Matching data is not the same as matching identities. And matching identities is not the same as proving wrongdoing. That distinction fundamentally changed how we approached identity resolution. We Had to Separate Observations From Conclusions When building Axon OSINT's identity resolution and crime pattern capabilities, one principle became particularly important: An observation should remain an observation until there is a justified reason to treat it as something more. Suppose an investigator encounters a suspicious cryptocurrency address. The system can record that the address was observed, where it appeared, and which incident it was associated with. If another report contains the same address, the system can identify a potential relationship. But that relationship is not yet a conclusion about the people behind the incidents. There are several different questions: Was the same identifier observed? Is the observation reliable? Does the surrounding evidence support a meaningful relationship? Is there information that contradicts the proposed connection? What is still unknown? These questions sound simple. Yet answering them properly requires much more than comparing strings. We had to design around the difference between what the system detected, what the evidence suggests, and what an investigator has actually reviewed. That separation prevents an automated observation from quietly becoming an established fact. Why More Data Doesn't Always Mean Better Evidence One particularly interesting challenge was the relationship between data volume and evidence quality. Imagine five different sources reporting the same suspicious domain. Five sources sound more convincing than one. But what if four of them copied the information from the fifth? What looks like independent corroboration could actually be the same claim repeated several times. Or imagine an identifier appearing in many reports because it belongs to a widely used service rather than a single malicious actor. The number of matches increases. The strength of the conclusion does not necessarily increase with it. This is a general problem in intelligence analysis: repetition can create the appearance of certainty without adding independent evidence. For Axon OSINT, that meant treating source context and evidence provenance as essential parts of the analytical process. A relationship should not become more trustworthy merely because the same underlying information has been imported repeatedly. And a missing piece of evidence should not automatically be treated as evidence against a hypothesis. Sometimes the correct answer is simply: We don't know yet. For a system designed to help investigators, that is a perfectly valid answer. False Positives Are Not Just Technical Errors In many software products, a false positive is an inconvenience. An overly aggressive spam filter might place a legitimate email in the wrong folder. A recommendation engine might suggest something irrelevant. But in investigative software, a false positive can shape the direction of an investigation. It might cause an analyst to spend time investigating an unrelated person. It could create misleading relationships between separate incidents. And if an unsupported connection spreads into reports or downstream decisions, correcting the original mistake becomes increasingly difficult. That's why identity resolution cannot be reduced to a binary question: Match or no match? Some relationships are plausible but unverified. Others are supported by stronger evidence. Some contain contradictions. Others are eventually rejected as false positives. These distinctions matter because investigative knowledge evolves. A system must be able to change its assessment without pretending its earlier conclusions never existed. We approached this as a problem of preserving both evidence and the history of analytical decisions. An investigator may reconsider a potential relationship when new information becomes available. That should not require rewriting the original observation or erasing why the earlier assessment existed. A Graph Can Be Persuasive Even When It's Wrong Investigation graphs are powerful. They turn disconnected records into something people can understand visually. A phone number connects two incidents. A domain connects several websites. A wallet appears across multiple reports. Suddenly, the investigator can see possible relationships that were difficult to recognize in a table. But visualization introduces another risk. A line drawn between two nodes can look more authoritative than the evidence behind it. People naturally interpret connected objects as meaningfully related. The graph may be technically accurate — two records really do reference the same identifier — while the viewer draws a much stronger conclusion. This became an important design consideration for Axon OSINT. We wanted an investigation graph that helped analysts explore evidence without quietly making decisions for them. A shared indicator should remain distinguishable from a reviewed relationship. A potential crime pattern should remain distinguishable from a confirmed finding about the people involved. And different incidents should preserve their individual context, even when they share evidence. The purpose of the graph is not to manufacture certainty. It's to make relationships easier to examine, challenge, and understand. The Unexpected Challenge: Keeping Unverified Reports Out of Pattern Analysis Another problem emerged from the source of the information itself. Fraud intelligence often begins with reports submitted by ordinary people. These reports can be extremely valuable. A single suspicious message might contain a phone number, website, wallet address, or other useful indicator. But public reporting also introduces uncertainty. A submission may be incomplete. It may contain mistakes. The person submitting it may not have completed the required confirmation process. Or the report may not have the necessary consent for broader analytical processing. If every incoming submission immediately contributes to pattern detection, the system risks building relationships from information that has not satisfied its admission requirements. During the development of Axon OSINT, we strengthened this boundary to prevent unconfirmed reports from entering Crime Pattern processing. This was not simply a frontend validation issue. The important question was whether the analytical processing layer itself would respect the required conditions. A visual warning or disabled button is not a sufficient substitute for backend enforcement. Trustworthy analytics begins before the first matching operation. The quality of an analytical result depends partly on which data is permitted to enter the process. Rebuilding Intelligence Without Rewriting History There is another difficulty that becomes visible only after an analytical system starts evolving. What happens when an earlier relationship is corrected? Suppose two observations were believed to be connected. Later, an investigator discovers conflicting evidence. The relationship should no longer be presented in the same way. But perhaps that relationship already influenced a larger group of related incidents. Now the system must account for the changed interpretation. This is why corrections and invalidation are important concepts in analytical software. It isn't enough to change the label on one record. Dependent interpretations may need to be reconsidered. The original evidence should remain traceable. And the system should avoid creating duplicate or contradictory analytical objects whenever information is processed again. We addressed these concerns as part of Axon OSINT's identity resolution and crime pattern development. The broader engineering lesson is that correcting an analytical conclusion must be treated as a normal operation, not an exceptional failure. Real-world evidence changes. Software that works with evidence must be designed for that reality. What We Actually Built The work resulted in an identity resolution and crime pattern foundation that distinguishes observations, candidate relationships, review decisions, and investigation-oriented representations of related incidents. The system is designed to preserve the distinction between observed indicators and conclusions about identity or conduct. It also supports reviewable relationships, handling of conflicting information, and controlled movement of analytical findings into investigation workflows. But I want to emphasize what this does not mean. Axon OSINT does not automatically determine that someone is a criminal because multiple reports share an identifier. A potential pattern is not proof of a coordinated criminal operation. And an analytical risk signal is not a legal determination. The goal is to support better investigations by helping people identify, examine, and organize meaningful connections. Not to replace human judgment with an impressive-looking graph. The Most Important Lesson When developers build systems that connect information, it's tempting to optimize for finding more relationships. More matches. More connections. More clusters. More insights. Those metrics can make a system appear increasingly intelligent. But in sensitive domains, maximizing the number of detected relationships is not necessarily the right objective. Sometimes the most valuable thing a system can do is refuse to make a conclusion. Sometimes it should preserve uncertainty. Sometimes it should show conflicting evidence. And sometimes the correct behavior is to prevent information from entering an analytical workflow at all. A trustworthy intelligence system isn't one that connects everything. It's one that knows when a connection is not enough. That is one of the most important lessons we've learned while building Axon OSINT. About Building Axon OSINT I'm documenting the engineering challenges behind Axon OSINT, a platform focused on digital investigations, open-source intelligence, and fraud-related information analysis. This series explores the difficult decisions behind building software that works with sensitive information — where correctness isn't just about whether the code runs, but also whether the system's conclusions can be trusted. More about the project: Axon OSINT A question for other developers: If you've built identity resolution, fraud detection, or data correlation systems, how do you prevent technically correct matches from becoming misleading conclusions?