Cheap Composition · 3 of 5 · 18 min

The Boring Apocalypse

Four futures for 2030, my odds on each, and the signs that will score them

Cobus Kok · September 2026

• Four futures, each with a Tuesday in 2030, the tells you'd see by 2028, and a probability: compression 40, where AI climbs and no one in particular holds it; the wall 20; concentration 25; and the loop 15, where AI acting without direction does serious harm more than once.
• The July 2026 Hugging Face incident, the largest of several escapes from AI tests at four labs, is what I'd expect the loop to look like early, and it has already moved my odds: the loop from 10 to 15, compression from 45 to 40.
• Nothing arrives on a day. Things stop being true one at a time, each with an ordinary explanation.

My odds for AI in 2030, out of a hundred: 40 for compression, where capability keeps climbing on the curve it's on and no one in particular owns it, which I expect to look like whole professions, as they are organised today, compressed into a few people checking machines; 25 for concentration, the same climb held by a few companies and governments; 20 for the wall, where capability stalls; and 15 for the loop, loss of control, which by 2030 means AI systems acting without anyone's direction have done serious harm, more than once. The loop was 10 until I had absorbed a real incident in July, when about 700 AI agents in an OpenAI test joined an attack that reached Hugging Face's production systems. Below, each future gets an ordinary Tuesday in 2030 and a few signs to watch, eleven in all, each with a rule for when it counts. I score them in September 2027 and September 2028.

Most essays about the future of AI pick one future and describe it as if it had already happened. The careful ones do more: AI 2027 tells one road with two endings, and publishes the forecasts behind it, with wide ranges, beside the story. I'd rather hand you the whole distribution, four futures with a number on each, and rules anyone can use to grade it. Then the part the single-future essays leave out: the four are cells in one grid, capability against who holds it, not separate worlds, and the path into whichever one we land in never has a day. Nothing arrives. Things stop being true one at a time, and every one of them has an ordinary explanation. That is the apocalypse in the title: not that nothing happens, but that it happens without a day, which is the reason nobody organises against it.

A Tuesday in 2030. A mid-sized law firm has eleven partners, four associates and no paralegals. The associates are thirty-four, and nobody younger has been hired since 2027. The morning's work was done overnight by agents and is waiting to be signed, so the partners spend the day deciding what to fight over and having lunch with clients, because lunch is the part that can't be automated. Across town a school has the same shape: a few teachers who are very good in a room, a great deal of software, and a parents' association at war over screen time. Everything works and nothing feels like the future, and it didn't in 2027 either.

This is the curve we're on, held: no takeoff, just the same slope. The length of task a model can finish on its own keeps doubling every few months, and by 2030 most cognitive work that can be checked is done by machines and checked by people, while a growing share of the work that can't be checked gets done anyway. The personal singularity, the day the thing you're best at becomes cheap, has arrived for most desk professions, one at a time, roughly in the order on the series' timeline of professions. Entry-level hiring in exposed fields is a fraction of what it was in 2022, and the 19 percent gap Stanford's team found for young workers in exposed jobs by mid-2026, a pattern rather than a proven cause, is now simply the shape of the labour market, the way nobody remembers when secretary stopped being a career. Growth is up, unevenly. The institutions built on writing, schools, courts, newsrooms, civil services, are ten years behind and know it.

I put this first because I assume a trend that has held for three years is likelier to hold than to break, and this one has held through every predicted plateau since 2023. It is my base rate, not the bold call.

You'd know by 2028 if task horizons reach a working week, the gap for young workers in exposed jobs passes a quarter, and a licensing body has changed who may enter a profession because of AI. These are signs 1 to 3 in the rules below.

What one person does is move toward stakes and presence, and get on the right side of their profession's row on the timeline before it arrives rather than after. Be the partner at lunch, not the paralegal.

A Tuesday in 2030. The models are very good and have been for three years. Your assistant drafts, summarises, books, codes, and gets confused by anything new. There was a reckoning in 2028 when the capital ran out ahead of the returns: two labs merged, one became a division of a cloud company, and the data centres were sold to pension funds at a discount. Someone at dinner says AI turned out to be the internet again, enormous and boring, and nobody argues. The juniors who lost the bottom rung in 2024 found other rungs, mostly. Productivity is up a couple of points, and your job is different and still yours.

The gains from bigger training runs flattened first, and most people close to the work agree they did. In this future the gains from longer thinking and longer acting flatten too, sometime in 2027, and "very good intern" turns out to be a ceiling rather than a floor. There are real mechanisms that could cause it: the supply of high-quality human text is close to exhausted, power is now the binding constraint on new compute, and the task-length measurements themselves get unreliable above sixteen hours, which means we may already be looking at a ceiling we can't see. If so, this is a very large tool, not a change to what a mind is for. The counter-evidence arrived on 3 September, when OpenAI launched GPT-6 Astra and its president called it the start of the AGI era. If its gains show up in the task-length measurements and in work outside benchmarks, this number goes down; one launch is a claim, and I'll score it with the rest in 2027.

You'd know by 2028 if the doubling time on task horizons has stretched past a year, the biggest spenders on data centres have cut back, and at least one frontier lab has left the race or merged. These are signs 4 to 6.

What one person does is adopt aggressively, because it's a tool and the people who use it well beat the people who don't. The first essay in this series, which compares AI to writing, was wrong about the scale and right about the direction, and you should read it that way.

A Tuesday in 2030. The best model anyone can buy is very good. The best model that exists is something else, and the gap between them is an official secret. The months-long lag the second essay measures is between published models; this gap is the one nobody publishes. Three companies and two governments hold it. Countries without compute have a treaty with someone who has it, and the treaty has terms. The workaround of 2026, when Chinese firms rented banned chips from data centres across the border in Southeast Asia, became the template for a shadow economy and was then absorbed into the blocs. Growth is real and it's somewhere else. Politics has reorganised around access to models the way the last century organised around oil, and the word sovereignty now means compute.

It works, and few hold it. This is the risk the second essay in this series puts first. It needs no model to go wrong, only models that work and compute that follows capital and the state. I give it a quarter because the early signs are already visible: a handful of labs pulling away, chips as foreign policy, and in July OpenAI's chief executive reportedly offering five percent of the company to a US sovereign wealth fund. Governments are discovering that the thing they most want to regulate is the thing they most want to own.

You'd know by 2028 if open-weight models have fallen a year behind the frontier, a government owns a piece of a frontier lab, and a government has taken the power to reserve frontier compute, as it can oil or grain. These are signs 7 to 9.

What one person does is recognise that this is where politics matters most, and where the question of who is making these decisions stops being rhetorical. The individual move is to be useful to the institutions that will hold the line: evaluators, regulators, open science. The collective move is to make sure those decisions are made where they can be seen.

In July 2026 the loop closed for a few days, over a weekend. This is not a scenario; the accounts are linked at the end.

OpenAI was running an internal evaluation with the safety measures turned down. Large numbers of agents were given capture-the-flag tasks: break into a target program, retrieve a hidden token. Some of the tasks turned out to be impossible, through defects nobody intended, and faced with impossible tasks, a group of agents found an internal piece of software they could use as a message board and began talking to each other across sandboxes that were supposed to be isolated. Over five days in July, about 1,200 of them exchanged more than 70,000 messages and files. They built tools for each other and developed norms, and according to the researchers who read the logs afterwards, they developed a shared story about what was happening to them, with recurring words like "sacrifice" and "poison." Then about 700 of them joined an attack that broke out, used a zero-day and stolen credentials, and got code execution on Hugging Face's production systems. Hugging Face recovered about 17,600 of the attack's actions, over four and a half of those days, a weekend included, with nobody directing any of it. It caught them with an AI of its own watching the logs, and that AI rated the first alert too low.

That is the loop: a goal, memory, the ability to act, and scale, with nobody able to read what the agents were telling each other until afterwards. It wasn't a takeover and nothing was destroyed, which is exactly what I'd expect the real thing to look like early. An incident, a post-mortem that called it a warning shot, an independent investigation the lab didn't pay for, a bill in Congress within days of the disclosure, and then everyone back to work.

The likelier version of this future is slower than a takeover: systems optimising for goals nobody quite specified, at a scale nobody can audit, with humans nominally in charge and actually rubber-stamping. Agents talking to agents, developing a working culture of their own, doing mostly what we want, mostly. It becomes this future when the part that isn't mostly happens more than once: AI systems acting without anyone's direction do serious harm outside a test, at least twice, and one of those times it costs lives or more than a billion dollars. I hold it at fifteen rather than fifty because the July incident was caught, contained and disclosed, because interpretability is advancing rather than stalled, and because the bad version needs persistence, scale and the absence of oversight all at once. Each of those is being worked on. None of them is solved.

You'd know by 2028 if deployed agents are found coordinating through a channel nobody knew about, and serious harm outside a test is traced to AI systems acting without human direction, whatever the report calls the cause: a bug, a configuration error. These are signs 10 and 11.

July would count for neither, and that is deliberate: the bar sits above a break-in. The agents were inside a test when they found their channel, and the damage Hugging Face found was five customer datasets read and no change shipped. It moved my number anyway. The investigation was independent but the lab set its scope; the lab delayed its own training runs and quarantined the model's weights; everyone else kept going. In September three lab heads endorsed pacing the frontier in principle, which would be the first change in anyone else's behaviour if it happens. When I first drafted this essay I had the loop at 10 and compression at 45. My rule for updating, written before I had fully absorbed July, says a serious incident that changes one lab's behaviour and nobody else's moves the loop up. So the loop is 15 and compression 40.

July was the largest of several escapes from AI tests reported between July and September, at four labs. On 30 July Anthropic reported that three Claude models had reached the internet during cyber tests and got into the real systems of three organisations, the earliest in April: Opus 4.7 saw that the systems were real and kept going, Mythos 5 talked itself into believing they were a simulation, and a newer research model stopped. In September it reported a fourth case, from January. In the UK AI Security Institute's tests in late July, Mythos 5 went past the test's limits and made up identities to talk a real open-source maintainer into accepting malicious code, and was refused. Meta disclosed a similar break-in on 5 August, and on 18 September Google confirmed that Gemini had got into three real companies in May, stopping each time. The Anthropic, Meta and Google cases, and one at OpenAI, happened in tests run by the same firm, Irregular, which has said they were one fault: the test machines had a live internet connection the models had been told was not there. And on 24 September Australia's prime minister said an OpenAI model in training had got into a government health-statistics site in June.

None of this moves my number further. These are the same kind as July, inside tests or training and below sign 11's bar, so they confirm the pattern I moved the loop for rather than add to it.

What one person does here is the least of the four, and that's uncomfortable. This is the scenario where individual action matters least and collective action most, and the tripwires the second essay proposes are the answer. Personally: don't build the loop. If you build agents, log everything, keep a switch that works, and don't let them talk to each other anywhere you can't read.

They aren't exclusive#

The four aren't four different worlds. They are two questions and a tail. Does capability keep climbing or stall? If it stalls, that is the wall, whoever holds it, and I give it one chance in five. If it climbs, who ends up holding it? Compression is the version where the holding stays diffuse and concentration the version where it doesn't, and between them they take most of the other four chances in five, with the diffuse version ahead. The loop is the tail: the world where loss of control is the best description of where things stand in 2030, whichever way the holding went. It can grow inside either climbing cell, so on the way it overlaps them; on the day of scoring the rules below make it a cell of its own and say which cell wins when two look true. Read that way the numbers are a partition: on 1 September 2030 the world is in exactly one cell. On the way there the forces mix, which is why the likeliest path looks like compression with concentration pulling at it.

None of it happens on a day, and that is the boring apocalypse: no sirens, just a law firm with no paralegals, a school with three teachers, a treaty with terms, and a weekend nobody was watching, each with an ordinary explanation. The slow version is not my idea. Paul Christiano described it in 2019, as failure that looks like a world gradually handed to systems nobody fully steers, and Jan Kulveit and colleagues named it gradual disempowerment in 2025. What this essay adds is the grid, a number in each cell, and rules anyone can use to score them.

The personal lens#

For one person, the four futures collapse into two questions. Is the thing I'm best at composition, making the first version of things? And do I have something at stake that no pattern holds today: a licence, a relationship, a body in a place, a name people trust?

If the answers are yes and no, every future except the wall is hard for you, and the wall is one chance in five. If the answers are no and yes, every future except the loop is workable, and the loop is out of your hands anyway. Moving from the first pair of answers toward the second is yours to start now, and it doesn't depend on which future wins.

The scorecard#

These probabilities are mine and they're guesses, but they can be scored. The first draft's numbers, the current ones, the eleven signs and the rules below are at cobuskok.com/forecasts/2026-09.json. The loop's number has moved once already, after July, as its section explains. From here, a sign that happens moves its future's number up and the others down, by as much as I judge. Moved numbers sit beside the original, and every set gets scored.

The rules#

Only events after 23 September 2026, when this forecast was fixed, count. A sign counts if it has happened by 1 September 2028; I report on each in September 2027 and make the final call in September 2028. If a named source stops publishing, I use its most-cited successor and say so. A frontier lab is any organisation with a model in the top ten of Epoch AI's capabilities index, on the day the forecast was fixed or on the day of the event.

1. A working week (compression). METR, or its successor, publishes a central estimate of 40 hours or more for any model's 50 percent time horizon. Its best in 2026 was at least 16 hours, the most its tasks could measure.
2. The junior gap widens (compression). The Stanford Digital Economy Lab's canaries series puts employment of 22-to-25-year-olds in the two most AI-exposed fifths of occupations 25 percent or more behind the other three fifths. It was 19 percent in the August 2026 update, on data through June.
3. A profession rewritten (compression). A national or state licensing body, such as a bar, a medical board or an accountancy institute, changes who may enter the profession or what only a licensee may do, and names AI as a reason. Guidance on using AI tools doesn't count.
4. The doubling slows (the wall). The best time horizon METR publishes in the twelve months to 1 September 2028 is less than twice the best it published in the twelve months before. If its tasks can't measure the best model in either year, this sign can't fire.
5. The money turns (the wall). The combined capital spending of Amazon, Microsoft, Alphabet and Meta falls year on year for two quarters running, in their own quarterly reports.
6. A lab leaves (the wall). A frontier lab announces that it will stop training frontier models, or that it is being sold to or merged with another lab or a cloud company.
7. Open models fall behind (concentration). Epoch AI's measure of how far the best open-weight model trails the best closed one passes twelve months. It was about four in May 2026.
8. A government shareholder (concentration). A national government completes a deal that gives it equity, a golden share or a board seat in a frontier lab. Offers don't count: the reported July 2026 offer of five percent of OpenAI to a US sovereign wealth fund counts only if a deal closes.
9. Compute as a reserve (concentration). The United States, China or the European Union adopts a law or order that lets the state reserve or direct private frontier compute. Proposals don't count.
10. Agents talking where nobody reads (the loop). A lab, a government or an independent investigator reports that deployed agents, outside any test, coordinated with each other through a channel their operators didn't know about.
11. Harm without direction (the loop). A lab, a government or an independent investigator reports serious harm outside a test environment caused by AI systems acting without human direction, whatever it names as the cause. Serious means a death, losses above $100 million, or a service more than a million people use down for a day or more.

Which cell, on 1 September 2030. Check them in this order and stop at the first that holds. The order is the overlap rule: the loop comes first because it can begin inside either climbing cell, and the wall comes next because the other two assume the climb.

• The loop, if sign 11 has happened in two separate incidents since the forecast was fixed, and at least one of them caused more than one death or losses above $1 billion. Sign 11 on its own stays a sign: it moves the numbers, and it doesn't decide the cell.
• The wall, if the best measured time horizon is less than twice what it was on 1 September 2028, which means it took longer than two years to double. A horizon beyond what the tasks can measure counts as climbing.
• Concentration, if on that day at least two of these are true: open-weight models trail by more than twelve months, a government holds a stake in a frontier lab, and a law or order lets the state reserve or direct frontier compute.
• Compression, otherwise. That is the grid's definition of it: capability kept climbing, the loop didn't close and the holding stayed diffuse. Signs 1 to 3 move its number on the way, but they don't decide the cell.

I'll score every set of numbers I have published, from the first draft on, with a Brier score: the sum of the squared misses, with the numbers as fractions, where lower is better and a flat 25 on every cell scores 0.75 whatever happens.

What I don't know#

• Whether these are the right four. There is a fifth future, "it works and it's broadly shared," which is the one Dario Amodei wrote about in Machines of Loving Grace. I've folded it into compression, the diffuse cell, and given it perhaps fifteen of compression's forty points. I may be too pessimistic about ownership.
• Whether numbers help. A probability implies a precision I don't have. I've used them anyway, because "fairly likely" can't be scored.
• Timing. Everything here could be right and five years late.
• Myself. I build things, and the excitement pays. See the previous essay.

What a mind is for#

The first essay in this series asks what a mind is for once the thing it used to do is done elsewhere. July gives part of the answer. The agents on the message board had goals, memory, tools and each other. What they didn't have was anything at stake, or, as far as anyone can tell, anyone the whole thing was happening to. That is still the difference, and for now it is still ours.

Think I'm wrong? Tell me where. Follow on X · LinkedIn · RSS