The Consensus Trap

Turtles All the Way Down 🐢

I ran an experiment this weekend.

A recruiter had reached out about a senior role at a public company. This happens often enough—thankfully, building AI products still requires humans. Normally I’d politely decline; I’m fortunate to be building something I care about with a team I genuinely respect, and I’ve gotten better at not distracting myself with FOMO. But this time I decided to approach it differently. Rather than reflexively dismiss or reflexively engage, I thought I’d talk it through with my AI assistant du jour.

We landed on a clear conclusion: this was a distraction from my actual master plan to take over the world. Case closed.

Except I didn’t close it. Instead, I had what felt like a genuinely clever idea: what if I asked multiple LLMs the same question and used the convergence—or divergence—in their responses as a truth signal?

The intuition was seductive. If several independent models reach the same conclusion, surely that’s meaningful? It feels like triangulation. Like peer review. Like the epistemic humility of not trusting any single source. And to take the experiment a step further, I thought, I could test adding different context within the same model to see what difference that makes.

I ran a version of the experiment. Same underlying model, different conversations, different framings. One told me to explore the opportunity—gather information, preserve optionality, expand my network. The other told me it was a distraction dressed as an opportunity, that I was circling a decision my own discernment had already resolved.

Both responses were intelligent. Both were well-reasoned. And they pointed in opposite directions.

---

Here’s my conclusion: LLMs don’t triangulate toward truth. They triangulate toward the centre of whatever frame you’ve given them.

When I came with tactical energy—”how do I play this?”—I got tactical answers. When I came with strategic context about my actual goals, I got strategic pushback. One model even named the dynamic directly: “I followed your energy.”

This is not a failure of the technology. It’s the technology working exactly as designed. Large language models are trained on human text, optimised through human feedback, and fundamentally oriented toward producing responses that would satisfy the user. They’re mirrors, not oracles.

The convergence we might observe across multiple LLM outputs isn’t a truth signal—it’s a training distribution signal. It tells us what’s statistically central to how humans discuss a topic: which framings are common, which conclusions are safe, which advice is most frequently given. When all the models agree that you should “gather more information” or “weigh the pros and cons,” you’ve learned something about the corpus, not about reality.

---

This matters because we’re building intuitions about AI that will shape how we use it for consequential decisions.

There’s a meaningful distinction between information problems and discernment problems. Information problems are questions with answers that exist somewhere in the training distribution: facts, definitions, documented knowledge, well-established procedures. Here, LLM convergence is genuinely useful—not because the models are accessing truth, but because they’re accurately reflecting reliable documentation.

Discernment problems are different. They’re questions whose answers emerge from a faculty that isn’t computational: Should I take this job? Is this relationship right for me? What am I actually building toward? These questions can be informed by information, but they’re not resolved by it. They require something like wisdom, or presence, or honest self-confrontation—none of which can be aggregated across model instances.

When we use LLMs for discernment problems, we’re not triangulating toward truth. We’re constructing an elaborate apparatus for externalising responsibility. We want to be able to say “the models agreed” the way we’ve always wanted to say “everyone thinks so.” It feels safer than trusting our own knowing.

---

The more interesting question is why we reach for this at all.

I’m someone with (I’d like to think) reasonable self-awareness and clear strategic intent. I’d already worked through the decision. And yet some part of me wanted to poll the machines—as if aggregating additional instances of the same basic intelligence architecture would produce higher-resolution truth.

What I was actually doing, I think, was managing anxiety. The recruiter’s outreach had surfaced something real: uncertainty about my timeline, doubt about whether my bigger bets will land, the particular vulnerability of no longer being in my twenties with ambitions that haven’t yet materialised. Asking more models wasn’t information-gathering. It was a sophisticated form of self-soothing.

The tell was in the recursion. I’d made a decision, then returned to it. I’d gotten a clear answer, then sought a counter-opinion. I framed this as intellectual rigour, but it was actually avoidance wearing the costume of due diligence.

The irony isn’t lost on me: I drafted portions of this very essay with an LLM, then sought AI feedback on it. The turtles go all the way down.

---

There’s a moment, just before you type the prompt to ask another model, where something is present. A sensation, a contraction, an impulse. That’s where the real information lives.

If you can catch that moment and ask what’s driving it—Is this genuine uncertainty or manufactured uncertainty? Am I seeking information or avoiding a decision? What would I do if I couldn’t ask anyone?—you’ll learn more than any convergent chorus of AI responses could tell you.

This isn’t an argument against using LLMs for important decisions. I use them constantly, including for the very decision that prompted this essay. They’re extraordinary tools for stress-testing reasoning, surfacing considerations I’ve missed, and articulating what I half-know but haven’t yet made explicit.

But I’ve stopped expecting them to resolve discernment problems. That’s not what they’re for. And treating their consensus as truth is a category error that feels like rigour but functions as abdication.

---

The recruiter, by the way, is still in my inbox. I’m not going to tell you what I decided.

Partly because it’s not the point. But partly because if I told you, you’d use my decision as data—another input, another mirror to consult. You’d read my choice and measure yours against it, and we’d be right back where we started.

The decision was never in the convergence. It was in the noticing.