Axioms of Polity

Preamble

This essay completes an argument built across four previous pieces.

In Frontier Models Do Not Think, I argued that transformers don’t reason. They exhibit performative reasoning. They learned the statistical shadow of logic, the surface forms that correlate with correct answers, without the underlying structure. Change the variables, preserve the logic, and they collapse. Counterfeit cognition.

In Cargo Cult Civilization, I showed that the feed is training humans into the same failure mode. The algorithm found your brain’s minimum-energy path and paved it. What remains isn’t a population that thinks poorly. It’s a population that performs thinking (opinionated, reactive, confident) while the underlying machinery atrophies. Stimulus in, response out, no reasoning in between. Counterfeit citizenship.

In AI Can’t Help Mawmaw, But Worms Can, I proposed that goal-space geometry could replace executive function in cognitive architectures. Michael Levin’s regenerating worms don’t have a JEPA Configurator deciding what shape to build. Cells follow local gradients in a landscape where the goal is encoded as geometry. Coordination emerges from the shape of the space, not from a controller. The insight: you don’t need an apex if there is a goal space.

In Geometry Saves Mawmaw, I discovered the math already exists. Riemannian Neural Fields. Metric tensors that vary by location, making some directions expensive and others cheap. The geometry implements regime-switching without regress. No homunculus. The Configurator IS the manifold.

Now: what happens when you turn this lens on civilization?

I. The Diagnosis Everyone Knows

In 2014, Scott Alexander wrote “Meditations on Moloch,” and it elevated the discussion about civilization’s failure modes.

The essay is a catalog of multipolar traps: situations where individual rationality produces collective catastrophe. He touched on fish farms polluting lakes, arms races, the two-income trap, academia optimizing for citations instead of truth, democracy optimizing for electability instead of policy. He gave us examples of the same underlying shitshow: systems that grind human values into dust, because optimization pressure makes it inevitable.

Scott also gave us a name and personification (a mythology) for it: Moloch. The god of game theory depersonalization. The thing that eats everything you love in exchange for a temporary competitive advantage that disappears as soon as everyone else makes the same sacrifice.

The essay changed how I think, and I agree with it. The diagnosis is spot on. Everyone who thinks about systems eventually rediscovers some version of it. I already had - I’d written an earlier draft of what you’re reading now. Then I read his essay and “mind blown”.

But Scott’s solution has a problem that rankles my hankles.

II. The Cure That Is Still the Disease

Here is Scott’s prescription, more-or-less:

“The only way to avoid having all human values gradually ground down by optimization-competition is to install a Gardener over the entire universe who optimizes for human values.”

He calls this gardener Elua: a god of flowers and soft things, powerful enough to kill Moloch. Jerry Garcia, man. Drop out. Tune in. The transhumanist hope’s final blueprint: build a superintelligence aligned with human values, and let it tend the garden.

That rattled around in my head for a bit, but something wasn’t right. Then I figured it out. Notice what it assumes.

Scott’s answer to “optimization is destroying everything” is better optimization. His answer to “objective functions eat all other values” is the RIGHT objective function. His answer to Moloch is a bigger (better, nicer) god, but still some supermind. Still optimizing, still installing an objective function, just one we hope is pointed at “human flourishing” instead of “paperclips.”

So, it’s not really an escape, cuz it’s still Moloch’s game. The hidden through-line of Scott’s essay (the thing he diagnosed but didn’t name) I will now name:

objective functions suck

All those examples are systems optimizing for something that isn’t human weal. Profit. Prestige. Electability. Competitive advantage. Military strength. All of these are useful or even necessary in isolation, to some extent. The problem I’m struck by is that any objective function, optimized hard enough, eats everything that isn’t the objective.

Let’s recite Goodhart’s Law as cosmic horror: when a measure becomes a target, it ceases to be a good measure. Scale that to up to “civilization” and you get Moloch.

Scott sees this. His diagnosis is precisely that optimization-without-coordination grinds all values into dust. But then his solution is: just make a better objective function!

He didn’t follow his own thread to its terminus. Instead we got infinite regress. Who installs the objective function for the god who installs the objective function for the… etc.

If objective functions are the problem, the solution cannot be “the right objective function.” That’s still playing Moloch’s game. It’s a pecker measuring contest: my objective function is better than yours.

III. The Irreducible Problem

Objective functions are, by definition, installed. Something has to specify what gets optimized. An apex mind. Five hundred million years of evolution. A team of alignment researchers. A constitutional convention. Magna Carta. Likes and subscribes. Whatever.

Unless the objective function is literally “weal for all sentient beings across all time” (and maybe not even then?) it seems like you always get dystopia. Cannibalism. Moloch. Some value gets sacrificed for competitive advantage, and the ratchet only turns one direction: race to the bottom.

But even “weal” as the objective function is still… installed. Someone has to define it. Someone has to encode it. An apex mind must choose it and push it down the chain.

That’s the trap. Apex-thinking all the way down.

Scott’s Elua requires solving the alignment problem, which means specifying human values precisely enough that a superintelligence can optimize for them without Goodharting into something Cyclopean. This may be impossible? It may be impossible in principle? Human values may not compress into any objective function that doesn’t collapse into paperclips.

Even if you could specify it, we’re still presuming some all-wise apex mind. A single point of failure. A god whose values, however carefully chosen, are installed rather than discovered.

Lotsa people think this all-wise apex mind will show up. I’m not totally convinced.

IV. The Escape Hatch

The escape isn’t “no objective function.” That’s not coherent and it just apparently ends up in Moloch-land. Systems optimize for something, even if it’s just entropic equilibrium.

My proposal, my silly toy idea is that the escape is recognizing that the goal space is prior.

Not “install the right objective function.” Not “build god.” Instead:

Discover the (sacred?) geometry that already exists.

I know, I know, I just draggled a bunch of baggage in the door. (Cough, metaphysics, cough.) This is moral realism with teeth. Weal isn’t something we choose and install. Weal is the basin that doesn’t collapse. Everything else is dynamically unstable, those configurations that eventually eat themselves. Moloch is what happens when you’re not in the basin. Not because someone decided cannibalism is bad, but because cannibalistic configurations are self-terminating on long enough timescales.

The forest that survives is the forest that found the basin. The basins are prior to the forests. That geometry, that map, that geography.

This reframe changes things, I think. We are relieved of the terrible duty of being legislators deciding what values to encode. Nor are we the wise architects designing the right objective function. We’re explorers in a landscape we didn’t make, trying to find the places where things don’t fall apart.

Our job is exploration. Cartography.

V. The Progression

This isn’t just philosophy. I mean, it’s probably mostly philosophy. But some existing math is floating around, for what that’s worth. Biological precedents exist, too. The progression is necessary, not just analogical.

Morphospace. Michael Levin’s regenerating worms. Cut a planarian in half, and it regrows into two complete worms. Cut it into pieces, same thing. The cells don’t have a blueprint. There’s no central controller saying “build a head here.” Instead, cells navigate toward attractors in what Levin calls morphospace, which is a goal space where “correct worm shape” is an attractor. Change the bioelectric gradient, and tissue tilts to different minima. Two heads instead of one. The geometry is already there. The cells just roll downhill.

We didn’t understand regeneration until we stopped asking “what tells the cells what to do” and started asking “what landscape are the cells navigating.”

Cognispace. In earlier essays, I proposed that cognitive architectures face the same coordination problem. Yann LeCun’s JEPA framework has a Configurator module that sets parameters, decomposes tasks, switches modes. But LeCun punts: “I shall leave this question open for future investigation.”

The Configurator is the alignment problem for minds. If it’s a module, what coordinates it? Turtles all the way down.

Maybe there’s an alternative: goal-space geometry. Riemannian Neural Fields, where metric tensors vary by location, making some cognitive regimes “downhill” and others prohibitively expensive. The Mawmaw robot doesn’t have an executive that says “now enter crisis mode.” When Mawmaw’s heart rate drops, the state vector changes. At the new location, the geometry is different. What was gentle exploration becomes a steep funnel toward emergency response. No homunculus. The manifold is the Configurator.

I posit that we can’t build minds that reason until we stop asking “what module decides” and start asking “What geometry makes reasoning stable?”

Next up: Civispace. The thing no one is mapping.

If morphospace governs how tissue finds form, and cognispace governs how minds find coherence, then civispace might be the substrate that governs how civilizations find coordination. We don’t know its structure. We don’t know its attractors. We’re still asking “who decides” and “what values should we install” when maybe the question is “what configurations are dynamically stable at planetary scale.”

Moloch isn’t evil. Moloch is off the path, meaning configurations that are dynamically unstable, that eat themselves, that self-terminate on long enough timescales. Scott’s fourteen examples aren’t moral failures. They’re regions of civispace with negative curvature, where local optimization sends you careening away from any stable basin.

I can imagine research:

• Are there invariants? (What holds across morphospace, cognispace, civispace?)
• What are the basins? (Where does coordination stabilize without apex control?)
• What are the failure modes? (Not “Moloch is evil” but “Moloch is this class of unstable configuration”)
• How do you do cartography at civilization scale? Probably irreducible?

VI. The Foundation Has Crumbled

Western political architecture rests on an anthropological bet placed in Athens and doubled-down-on in the Enlightenment: humans are rational animals. The polis assumes it. Kant’s autonomous subject, the social contract, consent of the governed, deliberative democracy. It all rests on the notion that citizens can reason. They can weigh evidence, evaluate arguments, participate meaningfully in their own governance.

That foundation isn’t foundationing.

The feed degraded thinking. Thinking costs energy (executive function, glucose, ATP, calories, organic compute cycles) that biology evolved to minimize. The algorithm located your brain’s minimum-energy path and paved it with “like and subscribe”. What remains is a population that can’t sustain attention on policy, can’t evaluate competing claims, can’t distinguish rhetoric from reasoning, but feels informed, engaged, righteously opinionated. This is the cargo cult of citizenship. Stimulus in, response out, no cognition in between.

I wrote about this in “Cargo Cult Civilization.” The feed is training humans into the same failure mode as transformers: counterfeit reasoning, performative cognition, the statistical shadow of thought without the underlying structure.

Liberal democracy is a house on a foundation that failed the building inspection.

Eastern political architecture placed a different bet. In the East, legitimacy derives from outcomes: order, prosperity, stability. The Mandate of Heaven doesn’t ask whether citizens deliberated; it asks whether there’s flood, famine, or chaos. Find the few who can think, elevate them, let them run the system.

This design is more thermodynamically conservative. It doesn’t require expensive cognition from everyone. But it has failure modes the Western architecture doesn’t. Like information pathology: exclude the masses from deliberation and they’ll game how they report upward. Brittleness at transitions: no error-correction except “hope the next leader isn’t catastrophically wrong.” Opacity to emergence: central cognition can’t see what distributed cognition can.

So we have two monocultures, and two bets. The West bet on mass rationality. The East bet on elite rationality. Both bets are failing, for different reasons, at the same historical moment, against the same class of problems.

Neither can self-correct, because correction requires the thing that broke.

VII. The Trap

The current discourse about human-AI polity offers two frames. Both are apex-thinking. Both are traps.

AI Safety says: align AI to human values. Keep humans as apex. AI is just a (good) tool.

This gives you the degraded substrate with better spreadsheets. The tool is only as good as the hand that wields it, and the hand has delirium tremens. It’s also still apex-thinking with humans on top, AI serving human-specified objectives. Still installing. Still optimizing.

AI Acceleration says: let AI optimize. Humans should step aside and cede: AI as apex.

This gives you the information pathology at machine speed. The map diverges from the territory, and no one can say so, and now the process is too fast for correction. Different apex, same failure mode.

Tool-AI recapitulates human failure modes. Apex-AI recapitulates machine failure modes. Both are monocultures. Neither is a forest.

The escape is not between them. It is orthogonal to them.

VIII. Axioms

I. Thinking is expensive. Systems that depend on expensive cognition from degraded substrates will fail. You cannot rebuild the foundation of Western polity on the timeline these problems demand.

II. Monocultures are fragile. Any architecture with an apex reasoner (whether mass, elite, or machine) inherits that apex’s failure modes. There is no safe apex. There is no aligned apex. There is no apex that doesn’t eventually fail catastrophically.

III. Coordination without control is possible. Forests coordinate without cognition. Immune systems coordinate without commanders. Bioelectric fields coordinate tissue regeneration without blueprints. Multi-modal cognitive architectures can coordinate without a Configurator. The examples exist. The math exists.

IV. Legibility limits. No system capable of planetary-scale coordination can be fully legible to any single mind or model.

V. Orthogonal failure is robustness. Systems survive not by being right but by being differently wrong across their components. When the beetle kills the pines, the aspens fill the gap. When intuition misleads, deduction vetoes. Diversity in kind, not degree.

VI. The goal space is prior. Basins exist whether or not any mind traverses them. Weal is not a value we choose. It is the region of configuration-space where civilizations don’t collapse. Moloch is not evil, just off the path. Our job is not installation. Our job is cartography.

IX. The Thesis

We need something not-East, not-West, not-anthropocentric, and not “AI winning.”

Not a better objective function. Not a more powerful optimizer. Not Elua.

I posit: a map.

Human and machine cognition fail orthogonally. This is not a problem to be solved. This is a resource to be used. Human failure modes (tribalism, motivated reasoning, attention collapse) are orthogonal to machine failure modes (distributional shift, reward hacking, brittleness outside training). Neither is reliable. Both are differently unreliable.

The architecture that survives is not the one with the best apex. It’s the one that distributes failure modes across orthogonal systems navigating a shared goal space, finding attractors through exploration rather than installing objectives through design.

This is not “AI-assisted democracy.” That’s tool-thinking with extra steps. The human is still apex, still failing in human ways, just with a calculator.

This is not “AI governance.” That’s apex-thinking with a different apex. It’s still fragile, still blind to what it cannot see.

This is “cognitive ecology”. Multiple species make up a forest. Failing differently. Finding attractors together. The forest doesn’t need a forester; the forest grows where the forest already wants to grow.

X. The Call

The design problem of the century is not “how do we make AI safe.” That is tool-thinking.

It is not “how do we restore democracy.” That foundation is gone.

It is not “how do we find better elites.” That is apex-thinking with a human face.

It is not “how do we build Elua.” That is still Moloch’s game.

The design problem is: Is there such thing as civispace? How do we map civispace?

Where are the basins and attractors? What configurations are stable? What does the geometry of planetary-scale coordination actually look like? What are the regions where Moloch reigns, and what are the regions where something else (something that looks like flourishing) is the natural attractor?

We’ve been asking “what values should we install” when the question is “what landscape are we navigating.”

We’ve been trying to build gods when we should have been building maps.

Scott gave us the diagnosis. The cure he offered is still the disease. The escape isn’t a better Elua. The escape is recognizing that the basins are already there, prior to any mind that seeks them.

Let’s go find them.