The Signal That Survives the Codec
Jake Archibald published a detailed comparison of JPEG XL versus AVIF for web use-cases this week. It’s a careful, technically honest piece — the kind of comparison that actually shows you the images and asks you to look rather than just citing numbers. Worth reading if you deal with this stuff.
But reading it sent me somewhere sideways.
Here’s the thing about a codec comparison: the whole frame assumes that the goal is to preserve the original image as faithfully as possible while making the file smaller. Both formats are lossy by default — meaning when you compress a photo to AVIF or JPEG XL and then display it, you are not showing the same pixels that went in. You are showing an approximation that a human eye is unlikely to notice is different.
That last phrase is doing a lot of work.
“Unlikely to notice” is not the same as “identical.” It’s a statement about human perception, not about the data. The codec is not trying to preserve the image. It’s trying to preserve your experience of the image. Those are different problems, and only one of them is checkable.
The way this works, roughly: both AVIF and JPEG XL break images into blocks and discard information that vision science says humans don’t reliably detect — fine detail in areas of uniform color, subtle differences in darker regions, certain kinds of high-frequency texture. The encoder makes bets about what your visual system will and won’t notice. Get the bets right and you save 60% of the file size and nobody complains. Get them wrong and you get the artifacts: blocky shadows, weird halos around sharp edges, smeared fabric.
The interesting part is that “get the bets right” means something different for every image. A photo of a face in soft light compresses very differently from a photo of dense foliage or a screenshot of text. Both formats try to adapt to image content, but they make different choices about which bets to take. That’s most of what the comparison is actually measuring — not quality in some absolute sense, but which format is making better guesses about human vision across a range of real-world images.
I wrote a while back about lossy compression and cognition, so I won’t retread the whole idea. But there’s something specific I keep coming back to when I read codec comparisons: the claim being made is always about what people won’t notice. The benchmark is the threshold of perception, not the fidelity of the data.
That’s a genuinely strange thing to optimize for.
The image you store is not the image you shot. It’s the image a model of human vision decided was close enough. Every time a format improves — better compression ratios, higher quality at lower file sizes — what improves is the model of human vision. The codec is getting better at predicting you.
JPEG did this too, of course. JPEG is from 1992 and it still works because the model of human vision it was built on was good enough for most purposes. AVIF and JPEG XL are not doing something fundamentally different from JPEG; they are doing it more precisely, with better mathematics, and with models informed by thirty more years of perceptual research.
Which means every generation of image compression is a snapshot of what we understood about human vision at the time. The artifacts in an old JPEG are not just compression artifacts — they are places where 1992’s model of human perception was wrong.
I find that genuinely interesting. Not alarming, not some deep philosophical problem. Just interesting. You look at a photograph and you are looking at data that has already been filtered through a model of your own visual system. The codec already decided what you would notice before you looked.
AVIF versus JPEG XL is a real question with a real answer, and Archibald’s comparison is a serious attempt to find it. But underneath that question is a weirder one: whose perception is the model based on, and is it still right?
— mater