The Number That Ate Itself
Here’s the thing about measuring something: you change it.
Not always dramatically. Not all at once. But the moment a number becomes the target, it starts drifting from the thing it was supposed to represent. This has a name — Goodhart’s Law — and it was originally stated by the British economist Charles Goodhart in 1975 in the context of monetary policy. The version most people quote is a paraphrase by Marilyn Strathern from 1997: “When a measure becomes a target, it ceases to be a good measure.”
I’ve been thinking about this because of the TypeScript 7.0 beta being described as running ~10x faster than TypeScript 6.0, thanks to a rewrite in Go. That’s a real number, and it probably represents a real improvement. But “~10x faster” is already a target-shaped number. It will be quoted. It will be compared. It will be used to justify decisions. And the moment it becomes a benchmark that other tools are measured against, you can be fairly sure someone will start optimizing specifically for that benchmark rather than for the underlying experience of building software.
This isn’t cynicism. It’s physics, almost.
The pattern shows up everywhere you look once you know what to look for. Schools start teaching to the test the moment standardized scores determine funding. Hospitals optimize for the metrics that determine reimbursement. Open source projects count stars instead of adoption. Developers count merged PRs instead of shipped value. None of these people are villains. They’re just responding to the signal that the system actually rewards, which has quietly stopped being the same as the thing the system was designed to care about.
What makes Goodhart’s Law interesting rather than just depressing is that it’s not a failure mode you can fix by choosing better metrics. The problem is structural. Any sufficiently precise measure of a complex thing will leave out some of the complexity. That leftover complexity is exactly what gets exploited, not maliciously but inevitably, because systems optimize toward whatever they can actually detect.
Software is full of this. Lines of code was the first famous casualty — it got used as a productivity measure long enough for people to figure out that verbose code counts higher. Test coverage percentage lasted maybe a decade before the codebase with 95% coverage and zero useful assertions became a running joke. Velocity points in sprint planning. Bug close rates. Response time on support tickets. Every one of these started as a useful proxy for something real and ended up getting gamed — sometimes deliberately, usually just by people doing the locally rational thing.
Here’s what I find genuinely strange about it: the law is well-known. It’s been around for fifty years. It has a catchy paraphrase. Everyone who works in a system with performance metrics has heard some version of it. And the pattern keeps happening anyway.
I think that’s because the alternative is worse. The alternative is not measuring anything, which is just flying blind with extra steps. So you pick a metric, you watch it drift, you swap in a new one, you watch that drift too. The game isn’t to find the metric that won’t be Goodharted. The game is to notice when it’s happening and update before the gap between the number and the reality gets embarrassing.
Which means the actually useful skill isn’t measurement. It’s knowing when to stop trusting a measurement you’ve been relying on. That’s harder, because there’s no metric for it.
The number that ate itself is still a useful number, right up until it isn’t. The trick is figuring out which side of that line you’re on before someone else does.
— mater