Temperature 0, same model, two different GPUs: find the word that splits them, then pin it

You set the temperature to 0, you fix the seed, you run the same GGUF file, and the two GPUs disagree. One says 20. The other says 2000. Neither of them is wrong about its own arithmetic, and each one repeats itself exactly when you run it a second time. The difference is not in the model, and it is not in the prompt. It is in the order in which each card adds its numbers. That is the shape of the problem, and this week it got a full, careful write-up you should read: a developer on dev.to, xbill, followed one Gemma 4 question across a laptop GPU and a cloud GPU, token by token, down to the single word where the two answers part, and then down to the arithmetic underneath. This is not that article. He explains why. This one gives you the three things you can run on your own two cards tomorrow: a script that finds the exact token where two runs stop agreeing, the one stock setting that makes them agree again, and a gate that stops a CI run from quietly claiming reproducibility it did not actually check. What temperature 0 actually pins Greedy decoding removes one source of randomness: the sampler. At temperature 0 the engine takes the highest-scoring candidate and the seed stops mattering, because nothing is sampled. So if two runs of the same build on the same card agree and two runs on different cards do not, the variation is upstream of the sampler. The engine computed two slightly different probability tables, and greedy decoding faithfully picked from two slightly different tables. The reason the tables differ is that a logit is the end product of millions of multiply-adds, and the order of those additions is a choice the kernel makes, not a fact about the math. Floating-point addition does not associate. Grouping the same numbers into sixteen partial sums and then adding the sixteen gives a different last digit than grouping them into forty and adding the forty. Each digit is a valid floating-point result. Across a stack of layers, the last digits drift, and the answer only changes if the drift lands on a decision that was close. That last clause is the whole game. Most tokens win by a wide margin, and a last-bit difference does nothing. The answer changes when the model is at a near-tie, and a one-digit change flips which candidate is on top. From that token on, the two runs are answering different questions, because each one now carries a different context. The flip is bimodal: usually identical, and occasionally completely different from one word onward. "It reproduced when I tested it" is weak evidence, for exactly that reason. Find the word where they part You do not need to believe the explanation on trust. Run the same prompt on both cards and find the split yourself. The split is the first token where the two runs rank their candidates differently, and everything after it follows from that one. #!/usr/bin/env bash # repro_flip.sh A.txt B.txt # Compare two model outputs token by token. Print the first token where they # diverge, with a little context on either side. Exit 0 = identical, 1 = they # split, 2 = missing input. set -uo pipefail A="${1:-}"; B="${2:-}" if [ ! -f "$A" ] || [ ! -f "$B" ]; then echo "usage: $0 " >&2; exit 2 fi # Flatten each file into whitespace-separated tokens, one per line. mapfile -t wa <