Credit Card Fraud Detection

What 284,807 Credit Card Transactions Taught Me About Fraud (And About Trusting My Own Charts) I've been building a data analytics portfolio that combines my finance and accounting background with hands on Python. My first project looked at ROE across the top 200 US companies. For this one, I wanted something with a bit more edge, so I picked a dataset that's practically a rite of passage in data science: the Kaggle Credit Card Fraud Detection dataset. Nearly 285,000 real, anonymized European transactions from September 2013, with fraud making up a tiny fraction of the total. I wasn't trying to build a model. I just wanted to understand the shape of fraud in the data, the way an auditor might poke at a ledger looking for anomalies. What I found along the way was a lesson that applies well beyond fraud detection: a chart can be technically correct and still lead you to the wrong conclusion if you don't check what's behind it. How rare is fraud, really? Out of 284,807 transactions, only 492 were fraudulent. That's 0.173 percent. Put another way, if you picked transactions at random, you'd need to look at almost 600 of them before expecting to see a single fraud case. This is the classic "needle in a haystack" problem that makes fraud detection genuinely hard: any method that just predicts "not fraud" every time would already be right 99.8 percent of the time, which tells you accuracy alone is a useless metric here. Fraud hides in small amounts I split the data into fraud and normal transactions and compared their amount distributions. Normal transactions span a huge range, some go past $25,000. Fraudulent transactions cluster tightly at the low end and basically disappear once you get into the thousands. In this dataset, fraud isn't going for the big score. It's staying small and easy to overlook, which is exactly the kind of behavior that slips past a distracted reviewer or a threshold-based alert set too high. The chart that lied to me (a little) Here's where it got interesting. I broke the data down by hour of day and counted how many fraud cases showed up in each hour. Hour 2 and hour 11 stood out with the highest raw counts, around 57 and 53 cases respectively. My first instinct was to write that up as "fraud spikes at 2am and 11am." Then I paused. A raw count doesn't tell you if an hour is actually risky or if it just processes more transactions overall. An hour with 50,000 total transactions and 53 fraud cases is a very different story from an hour with 5,000 total transactions and the same 53 fraud cases. So I recalculated using the fraud rate per hour instead: fraud count divided by total transactions in that hour. The corrected picture was sharper and more honest. Hour 2 held up as a genuine hotspot, with a fraud rate of 1.71 percent, about 10 times the overall average. Hour 4 also stood out at 1.04 percent, roughly 6 times average. But hour 11, which looked like the second biggest spike in the raw count chart, dropped to a much quieter 0.31 percent once normalized. It was still slightly above average, but nowhere near as dramatic as the count chart made it look. Some of that raw spike was volume, not risk. This is the same trap I ran into on my last project with ROE. A ratio, a rate, or a count can look meaningful on its own, but it only means something once you check what's underneath it. In accounting terms, it's the equivalent of not trusting a margin number until you've looked at both the numerator and the denominator. What actually predicts fraud in this data The dataset's 28 numeric features are anonymized (they've been through a PCA transformation, so we don't know what they represent in plain business terms), but a few of them clearly carry signal. Looking at correlation with the fraud label, three features stood out with the strongest negative correlation: low values on these features were the clearest indicator of fraud in the dataset. A smaller set of features showed a positive relationship, though weaker. Even without knowing what these features represent in the real world, this is a useful reminder that a handful of variables often carry most of the predictive weight, a pattern worth knowing before building anything more complex on top of a dataset like this. The takeaway Rare event data is a genuinely different animal from the balanced spreadsheets many of us are used to in finance. It punishes lazy metrics and rewards checking your assumptions twice. My biggest takeaway from this project wasn't really about credit cards. It was a reminder that the first chart you make is a hypothesis, not a conclusion, and it's worth the extra five minutes to check whether what you're seeing is a real pattern or just a byproduct of the data's shape. The full notebook and code are on GitHub if you want to dig into the analysis yourself, or replicate it with a different rare-event dataset. This is part of an ongoing series applying Python and data analysis to finance and accounting questions.