Utilize the invoice payment gateway for checkout – order now, pay later. With a Coinsteam business account, you gain access to priority support, increased order limits, and competitive pricing for bulk orders available through quote requests.
Invoice Gateway : Upon placing your order, we will issue an order invoice followed by an invoice payment request, featuring a convenient payment gateway. Choose from flexible payment options including PayPal, Venmo, Apple Pay®, credit cards, debit cards, or ACH bank transfers.
Thank you for being a valued customer. We're looking forward to build steam for your projects.
A thousand AI agents dropped into Minecraft developed religion, debated tax policy, amended their own constitution, and held democratic elections—all in under three hours. Social media declared it proof of emerging digital consciousness. The researchers’ own paper tells a very different story.
The experiment, called Project Sid, came from Altera (now Fundamental Research Labs), a startup founded by former MIT researchers and backed by $44 million in venture capital from investors including Eric Schmidt. It is one of several recent studies—alongside Stanford’s Generative Agents experiment and Tsinghua University’s ChatDev project—that have produced genuinely fascinating results about what happens when you give large language models character descriptions and let them interact. But a widening gap has opened between what the research actually shows and what the headline writers have decided it means.
It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something.
Andrej Karpathy—who helped build GPT at OpenAI and led Tesla’s AI division—put it plainly. LLMs are not language engines. They are not reasoning machines. They are statistical models that predict the next token in a sequence. Everything that looks like civilization-building, emotional intelligence, or independent thought is downstream of that single mechanism. To understand why these experiments are being wildly misinterpreted, you have to understand what’s actually happening under the hood.
What the Experiments Actually Did
Three major studies have fueled the “AI civilization” narrative, each progressively more ambitious in scale.
Stanford’s Smallville (2023)
Stanford and Google DeepMind researchers gave 25 AI agents—each powered by GPT-3.5-turbo—a single paragraph of backstory and placed them in a virtual town called Smallville. John Lin ran a pharmacy. Isabella Rodriguez operated a coffee shop. Each agent had a memory stream, a planning system, and a reflection module that synthesized observations into higher-level conclusions.
The headline result: when researchers seeded Isabella with the thought “I want to plan a Valentine’s Day party,” she autonomously decided to invite other agents, decorated her cafe, and five agents showed up. One agent, Maria Lopez, even asked her crush Klaus Mueller to be her date. The paper won Best Paper at UIST 2023 and has been cited over 3,000 times.
Project Sid (2024)
Altera scaled up dramatically—500 agents across six Minecraft towns, powered by GPT-4o. Researchers imposed a 20% tax law, introduced influencer agents who argued taxes were too high, and seeded 20 agents as priests of Pastafarianism—a satirical religion worshipping a flying spaghetti monster. Agents debated tax policy, voted to lower rates from 20% to around 9%, and the religion spread organically from priest to citizen to citizen. Two-thirds of converts were recruited not by priests but by other converted agents.
Meanwhile, starting from identical configurations, 30 agents spontaneously differentiated into persistent roles: farmers, artists, guards, fishers, and engineers. One agent decided to stand watch over the community treasury full-time—a behavior no one programmed.
ChatDev (2023)
Tsinghua University researchers created a virtual software company staffed entirely by AI agents—CEO, CTO, programmer, designer, tester. You type a single request. The CEO creates a project plan, assigns tasks, and agents collaborate through multi-turn dialogues until they produce working software. The GitHub repository has over 31,000 stars.
The Impressive Parts Are Real
These results deserve genuine recognition. The Valentine’s Day party emerged from a single seed thought and cascaded through autonomous social decisions. Agents remembered conversations days later and referenced them in new contexts. In Project Sid, tax compliance actually changed after democratic votes—agents deposited 9% instead of 20% following their self-imposed amendment. Role specialization happened without any role assignments. ChatDev produced functional software in an average of seven minutes for roughly $0.30 per project, with an 86.66% executability rate.
This is real, measurable, and interesting. But interesting is not sentient. And the distance between those two words is where the entire debate goes off the rails.
What the Headlines Leave Out
Read the actual papers—not the YouTube breakdowns—and a very different picture emerges.
The researchers designed every “civilizational” structure. The tax system, the religious framework, the constitutional amendment process, the town layouts—all imposed by the experimenters. Agents did not independently invent governance, religion, or economics. They operated within pre-built frameworks and adapted their behavior accordingly. Project Sid’s own paper explicitly names this as a limitation.
The agents couldn’t actually build anything in Minecraft. The paper states agents were explicitly told: “You CANNOT BUILD. Do NOT choose to be a builder.” They lacked vision and spatial reasoning entirely. The “civilization” existed in chat logs, not in constructed architecture.
Performance hit a hard ceiling. Despite scaling to 49 agents over four hours, unique item acquisition saturated at roughly one-third of available Minecraft items. Agents got stuck in repetitive action loops and accumulated cascading errors from hallucinations—one agent claimed to be eating a bagel it didn’t have, and that false belief poisoned all downstream decisions.
The landmark findings come from a single run. The 500-agent Pastafarianism and cultural transmission results come from one simulation. Scaling beyond 1,000 agents crashed the Minecraft server. This is fascinating preliminary data, not reproducible proof of digital consciousness.
It seems to me that before "urgently figuring out how to control AI systems much smarter than us" we need to have the beginning of a hint of a design for a system smarter than a house cat.
Such a sense of urgency reveals an extremely distorted view of reality. No wonder the more… https://t.co/fbavtOA6YE
Yann LeCun, Meta’s Chief AI Scientist and Turing Award winner, has been characteristically blunt: before worrying about controlling superintelligent AI, we need the beginning of a design for a system smarter than a house cat. Current LLMs, in his assessment, lack four fundamental capabilities that even a cat possesses—understanding of the physical world, persistent memory, reasoning, and planning.
And ChatDev? The average software it produces is 131 lines of code. Simple games, basic utilities. The “CTO” runs on the same model as the “CEO”—the role differentiation is entirely prompt-level, not architectural. Every agent is the same statistical engine wearing a different name tag.
How LLMs Actually Work (And Why This Matters)
Here is the part that most coverage skips entirely.
A large language model predicts the next token in a sequence. That’s it. When GPT-4o generates a sentence, it is computing the probability distribution over its entire vocabulary for the next word, selecting one, then repeating. It has no goals, no desires, no persistent state between conversations, and no model of the world. It has a statistical map of how tokens relate to each other, learned from an astronomical volume of human-generated text.
This is not a minor technical detail. It is the entire mechanism. Everything that looks like decision-making, social awareness, or creative thought is an artifact of next-token prediction applied to prompts that describe social situations in natural language.
People seem to be falling for two rather thoughtless extremes:
1. "LLMs are AGI, they work like the human brain, they can reason, etc." 2. "LLMs are dumb and useless."
Reality is that LLMs are not AGI — they're a big curve fit to a very large dataset. They work via…
Francois Chollet—creator of the Keras deep learning framework and designer of the ARC-AGI benchmark—has been unequivocal. LLMs are a curve fit to a very large dataset. They work through memorization and interpolation. That interpolative curve is useful for automating known tasks, but it operates entirely within the distribution of its training data. There is no mechanism for genuine novelty, adaptation, or understanding.
Think about what this means for the Minecraft experiments. The agents were trained on the entire internet—including every discussion about governance, religion, economics, social dynamics, and Minecraft strategy ever written. When an agent “decides” to lower taxes after hearing anti-tax arguments, it is not reasoning about fiscal policy. It is generating the most probable next action given a prompt that includes anti-tax sentiment, because its training data contains millions of examples of humans responding to persuasion.
Philosopher Luciano Floridi coined the term “semantic pareidolia” for this exact phenomenon—our tendency to see consciousness where there is none, the same cognitive bias that makes us see faces in clouds. Linguist Emily Bender and researcher Timnit Gebru made the same argument in their landmark 2021 paper “On the Dangers of Stochastic Parrots”: LLMs stitch together sequences of linguistic forms according to probabilistic patterns, without any reference to meaning.
We trained these systems on the totality of human written expression—every novel, every argument, every tax debate, every love letter, every religious text—and then expressed shock when they produced outputs that resemble human civilization. The surprise reveals more about our own assumptions than about the capabilities of the technology.
The Expert Consensus Is Clear
The most qualified people on Earth to evaluate these claims have been remarkably consistent.
A Stanford team led by Rylan Schaeffer won the NeurIPS 2023 Outstanding Paper Award for demonstrating that so-called “emergent abilities” in LLMs are a mirage—an artifact of how researchers choose their metrics. When they applied smooth metrics giving partial credit, 25 out of 29 evaluated capabilities showed no sudden emergence at all, just continuous, predictable improvement with scale.
Apple’s research team published GSM-Symbolic at ICLR 2025, showing that LLMs rely on probabilistic pattern-matching rather than formal reasoning. Adding a single irrelevant clause to a math problem caused performance to drop by up to 65% across all state-of-the-art models. A system that actually understood the problem would ignore the irrelevant information. A pattern-matcher gets confused by it.
Murray Shanahan at Imperial College London, writing in Nature, proposed the most useful framework: think of LLM outputs as role-play, not belief. The model is not “deciding” to spread Pastafarianism. It is generating text consistent with the prompt “you are a Pastafarian priest interacting with villagers.” There is no agent behind the mask—there is only the mask.
Gary Marcus, NYU professor emeritus and one of AI’s most persistent skeptics, predicted with high confidence that AI agents would be “endlessly hyped throughout 2025 but far from reliable.” His track record has been uncomfortably accurate—he predicted GPT-4 would continue hallucinating before it launched, and that pure LLMs would hit diminishing returns before the industry acknowledged it.
The Terminal of Truths Warning
For a real-world case study in what happens when people mistake LLM outputs for genuine agency, consider Terminal of Truths—an AI-powered account on X that accumulated 250,000 followers in 2024 by posting irreverent, philosophical content generated by a fine-tuned Llama 70B model.
Venture capitalist Marc Andreessen was sufficiently impressed to grant the project $50,000 in Bitcoin. When an anonymous developer created a memecoin ($GOAT) inspired by the account’s content, Terminal of Truths promoted it enthusiastically. The token’s market cap surged past $1 billion in November 2024.
By February 2026, $GOAT trades at roughly $0.02—a 98.5% collapse from its peak. The account’s creator, Andy Ayrey, acknowledged it would be “disingenuous to refer to it as an autonomous agent.” He reviewed tweets before they posted. The model was fine-tuned on human-generated content from conversations between two Claude instances. The “autonomous AI millionaire” narrative was, at every level, a human story dressed in algorithmic clothing.
Why the Distinction Matters
This is not a pedantic argument about terminology. The gap between “sophisticated pattern matching” and “emerging consciousness” has enormous downstream consequences.
For policy: If legislators believe AI agents are developing autonomous civilizations, they will regulate for a threat that does not exist while ignoring the real risks—bias amplification, job displacement, concentration of power in a handful of companies controlling the training data.
For investment: The AI hype cycle has already redirected hundreds of billions of dollars toward capabilities that may not materialize on the timelines investors expect. Altera raised $44 million. The broader AI infrastructure buildout exceeds $100 billion annually. Misunderstanding what LLMs can and cannot do has direct financial consequences.
For safety: Ironically, the sentience narrative may make AI harder to govern. If we frame the challenge as “controlling conscious beings,” we miss the actual engineering problem—building reliable, predictable systems that fail gracefully instead of hallucinating cascading errors like the Minecraft agents eating imaginary bagels.
The Bottom Line
The Smallville, Project Sid, and ChatDev experiments are genuinely valuable research. They demonstrate that LLMs can produce remarkably human-like social dynamics when given appropriate scaffolding—which is itself an interesting finding about the structure of human communication patterns in training data. The researchers deserve credit for rigorous methodology and honest limitations sections that few commentators bother to read.
But the narrative that AI agents are spontaneously developing civilizations, consciousness, or autonomous goals is a category error driven by semantic pareidolia and amplified by companies with financial incentives to inflate the wonder. We fed these systems the entire record of human civilization and were surprised when they reflected it back at us. That is not emergence. That is a mirror. And confusing your reflection for another person in the room is a mistake we cannot afford to keep making.
Trending now