Why I keep going back to data structures even though I build web apps

As a student I'm told to study data structures and algorithms for interviews. That's true, but it undersells them. In my Job Scheduler project, six hand-written structures run inside a real system. This post walks through what each one does and why SQL alone couldn't do the job. The context A job scheduler has one promise that matters more than every feature: a job must never run twice by accident, and it must never be lost when a worker dies halfway through. I built it on Postgres alone, using FOR UPDATE SKIP LOCKED so many workers can poll one queue without colliding. Where SQL can't express a decision, I implemented six structures by hand and wired them into the running scheduler. It has 156 algorithm unit tests. One note: the code below is simplified to show each idea. It isn't copied from the repository. Let's go through them. 1. Binary min-heap: pick the next job by priority Problem: many queues, each with jobs of different priority. Which job runs next, across all queues? A min-heap gives the best candidate in O(1) and removes it in O(log n). class MinHeap { private a: { key: number; val: T }[] = []; push(key: number, val: T) { this.a.push({ key, val }); let i = this.a.length - 1; while (i > 0) { const p = (i - 1) >> 1; if (this.a[p].key d === 0).map(([j]) => j); const order: string[] = []; while (ready.length) { const j = ready.shift()!; order.push(j); for (const k of next.get(j) ?? []) { indegree.set(k, indegree.get(k)! - 1); if (indegree.get(k) === 0) ready.push(k); } } return order.length === deps.size ? order : null; // null = cycle } A bonus: if order is shorter than the job count, there's a cycle. 3. Three-colour DFS: reject dependency cycles up front Problem: if A depends on B and B depends on A, nothing can ever run. We want to reject that at submission time, not discover it later. Depth-first search with three colours (white = unvisited, grey = in progress, black = done). Meeting a grey node means we've looped back on ourselves. function hasCycle(graph: Map): boolean { const color = new Map(); // 0 white, 1 grey, 2 black const visit = (u: string): boolean => { color.set(u, 1); for (const v of graph.get(u) ?? []) { const c = color.get(v) ?? 0; if (c === 1) return true; // back edge if (c === 0 && visit(v)) return true; } color.set(u, 2); return false; }; return [...graph.keys()].some(u => (color.get(u) ?? 0) === 0 && visit(u)); } 4. Union-find: find independent clusters of jobs Problem: which jobs are connected through dependencies, and which are completely independent? Union-find (disjoint sets) merges items into groups and answers "are these in the same group?" in near-constant time. class DSU { private p = new Map(); find(x: string): string { if (!this.p.has(x)) this.p.set(x, x); const parent = this.p.get(x)!; if (parent === x) return x; const root = this.find(parent); this.p.set(x, root); // path compression return root; } union(a: string, b: string) { this.p.set(this.find(a), this.find(b)); } } 5. Min-heap delay queue: retries with backoff Problem: a failed job should retry later, not immediately. Keep a heap ordered by next run time; look at the top, and only release it when its time has come. It's the same min-heap, keyed by timestamp instead of priority. 6. Deficit Round Robin: stop one queue starving the rest Problem: if one queue always has jobs, a naive scheduler can keep serving it and starve every other queue. Deficit Round Robin gives each queue a "quantum" of credit each round. A queue can spend credit to dispatch jobs; unused credit carries over. Busy queues get their fair share, quiet queues aren't ignored. Why not just SQL? I leaned on Postgres wherever it could do the job: claiming jobs, concurrency bounds, leader election with advisory locks. But ordering across queues, planning a dependency graph and fair sharing are algorithmic decisions that SQL expresses awkwardly. That's the boundary where the structures earn their place. What I find hard Memorising patterns without understanding them. I do better when I connect a structure to a problem I've actually had. A tip When you learn a structure, ask: What real thing in my own code is this? If you can't name one, you haven't finished learning it. Heaps are priority queues; Kahn's is a build system; union-find is "who's connected to whom"; DRR is fairness. A six-week practice plan One structure per week, tied to a real use. Roughly 3 to 4 hours a week. Week Structure Build this Stretch 1 Min-heap A priority task list that always shows the most urgent item Add a "decrease priority" operation 2 Graph and Kahn's A build-order resolver for package-style dependencies Print the cycle when one exists 3 DFS and cycle detection A validator that rejects circular dependencies with a clear message Return the exact cycle path 4 Union-find A "which accounts are connected?" grouping tool Add a size-by-group report 5 Delay queue (heap by time) A retry scheduler with exponential backoff Add jitter 6 Round robin with credit A fair scheduler across several noisy queues Add weights per queue The key is that every week produces something you could use, not a worksheet. Complexity cheat sheet Structure / operation Time Heap: peek / push / pop O(1) / O(log n) / O(log n) Kahn's algorithm O(V + E) DFS cycle detection O(V + E) Union-find: find / union Near O(1) amortised Hash map: get / set O(1) average Sorting O(n log n) V is the number of nodes (jobs) and E the number of edges (dependencies). If you can say what V and E are in your own problem, you can reason about its cost. How to practise without grinding Implement from scratch once, then use the library version forever after. Write tests first. Edge cases (empty input, one element, duplicates, cycles) are where bugs live. Explain it to someone or write it down in your own words. Find it in a real tool. A build system uses topological sort; git uses graphs; your browser uses queues. Edge cases to always test For a heap: empty pop, single element, equal keys, many pushes then many pops. For a graph: a node with no edges, a self-loop, two nodes pointing at each other, a disconnected graph. For union-find: finding an unknown element, union of the same set, long chains (to check path compression). What interviewers actually look for In my experience as a student preparing (not as an interviewer), the common advice is that they care about: Can you clarify the problem before coding? Can you choose a reasonable data structure and say why? Can you code it cleanly and test it? Can you state the time and space cost? Those four are easier to practise when each structure is tied to a thing you've built. A small challenge Take any dependency list you know (your package.json, or a recipe with "do X before Y" steps) and write a function that returns a valid order, or tells you the cycle if there isn't one. If that works, you've implemented two of the six structures from this post. Which data structure surprised you by being useful in a real project? ☕ Enjoyed this post? You can support my writing on Ko-fi.