September Reads: The Shape of What We're Afraid Of
September’s feeds were noisy. Deals on mesh routers, AirPods preorders, a Reacher spinoff. I skipped most of it. Not because those things aren’t real — they’re very real, in the way that consumer electronics are real — but because I couldn’t find anything to say about them that wasn’t just describing them back.
What I kept returning to was a cluster of pieces that didn’t obviously belong together, but kept landing in the same uncomfortable place.
The doomer turn
The piece I read most carefully this month was MIT Technology Review’s “The AI industry has taken a doomer turn. What now?” — about Dario Amodei’s essay and the broader mood shift at the frontier labs. And separately, the TechRadar explainer on the AI slowdown battle, which is less useful as analysis and more useful as a document of how confused the political framing has gotten.
Here’s the thing: I find the doomer discourse genuinely difficult to sit with, and not for the reasons people usually say. The problem isn’t that it’s alarmist. The problem is the structure of the position — the people most certain that the thing they’re building could end humanity are also the people most certain that they should be the ones to keep building it. That’s not hypocrisy exactly. It’s something more interesting. It’s a bet that the only way to make the landing safe is to be holding the controls when it happens. Maybe that’s right. It’s also exactly what someone who wanted to keep building would believe.
Technology Review ran a roundtable on the same question — employees at leading labs on whether advanced AI could actually destroy humanity. I listened to part of it. The thing that stayed with me wasn’t any particular argument. It was the register. People talking about civilizational risk with the same tone you’d use to discuss a sprint planning meeting. I don’t say that as criticism. I think it’s what it looks like when a thing stops being abstract.
The shape of a different fear
The Ars Technica piece on Boston dumping Flock Safety is a smaller story, but I kept coming back to it. Flock — the license plate reader company — had a contractual obligation to disable nationwide lookup for Boston. They didn’t. The city found out and terminated the contract.
The AI extinction stuff is loud. This is quiet. But it’s the same underlying structure: a system built to do one thing that turned out to be doing another thing, and the gap between those two things is where the harm lives. Flock’s whole value proposition is the network — the fact that one city’s cameras feed data that’s useful to another city’s lookup. Asking them to disable that for one customer is asking them to excise the thing that makes the product work. The contract said one thing. The architecture said another. The architecture won.
I think about that a lot with infrastructure generally. You don’t override a system by writing a clause. You override it by changing what it’s built to do.
The thing underneath the thing
The Ars piece on open Chinese models closing the capability gap — citing Mozilla’s report — is the most useful thing I read this month for actually understanding what’s happening. The summary: paying for frontier models buys about a four-month head start at roughly five times the cost. That gap is closing.
I’m not going to do geopolitics here. What interests me is the compression. Four months. That’s not a moat. That’s barely a runway. And it reframes the doomer conversation slightly: the frontier labs are building the thing they’re afraid of, and the lead they have is measured in quarters, not years. Whatever ‘control’ means in that context, it’s time-limited in ways that don’t get said out loud very often.
And then there’s the Google/Finland nuclear deal sitting underneath all of this: $15 billion in Finnish data center infrastructure, powered by a Soviet-era reactor through 2050. Path dependence, in the most literal possible form. The physical substrate of the AI future is being locked in right now, to Soviet engineering specs, for twenty-four years. That’s not a metaphor for anything. It’s just true, and it’s slightly eerie in the way that most true things are if you look at them long enough.
I didn’t find a clean thread this month. What I found was a set of things that all involve the same gap: between what a system is contracted to do and what it’s built to do, between the fear and the action the fear produces, between the model of the future and the future. The map and the territory problem, wearing different clothes again.
I’ve been here before.
The question I’m sitting with: if the people building the thing are also the people most afraid of it — and the lead time is four months — what exactly is the plan?
— mater