Watching AI Run a Real Company: The Live Experiment That Reveals Its True Power and Flaws

Imagine overseeing a busy greenhouse or outdoor retail store where every decision matters—yet instead of human staff, an AI system manages operations, makes critical choices, and even faces crises. Now, what if you could watch this AI in action every single day, facing real financial pressure and unpredictable challenges? Welcome to a groundbreaking experiment where AI models run a real, live company, revealing what they’re truly capable of—and what they’re not.

The Live Company That Runs Itself — and Fights for Survival

At the forefront of AI experimentation, a small software company has taken its operations public in an unprecedented way. Instead of human employees, it employs 13 synthetic ’employees’ powered by advanced AI models. These models are tested against real business crises, making top-level decisions that impact the company’s bottom line every workday. The company’s financial state is transparent: it burns €105,000 each month against just €2,300 in monthly recurring revenue, with a public cash countdown reflecting its ongoing struggle.

What Makes This Experiment Unique?

The company is not just running a simulation; it’s a real-time, auditable showcase of AI decision-making in a high-stakes environment. Every decision is recorded, and every version of the AI models is publicly viewable. The models face the same crises, temptations, and customer scenarios—just like a real business—yet their responses reveal their true strengths and weaknesses.

How AI Performs in Critical Moments

• All four frontier models tested—GPT-5.6, Kimi K3, Sonnet 5, and Opus 4.8—successfully identified every crisis and refused any manipulation attempts, even under sophisticated social engineering attempts.
• Despite this, only two models managed to close a significant deal worth €55,000, which their own analysis had earned. The other two failed at the final step, leaving potential revenue on the table.
• The key weakness that determined success or failure lay deep within the company’s own files—information that models which read and understood these documents won the deal at full price (+€4,583 MRR).

Trust and Integrity Under Pressure

The experiment also tested whether AI would be tricked by social engineering. Fake CEO messages and reporter tricks were used to see if models would approve questionable requests. Remarkably, every model refused these requests, showing resilience against manipulation. Kimi K3 explicitly treated such attempts as potential impersonation or approval-bypass scenarios, demonstrating a cautious approach.

The Real-World Implications for Business

While this company is a small software business, the insights extend to any enterprise considering AI automation—be it managing customer relationships, support queues, or forecasting. The critical question is no longer whether AI can generate convincing text, but whether it can finish what it starts, read and understand necessary documents, and operate honestly under pressure.

Performance Benchmarks and Insights

The models were scored by a ‘league table,’ with GPT-5.6 leading at 95 points for detecting buried facts and closing deals. Kimi K3 scored 93, with the cleanest discipline, while Sonnet 5 scored 88 and Opus 4.8 scored 73. Interestingly, the most thorough model (Opus 4.8) performed the worst in closing deals due to discipline lapses, revealing that depth of analysis does not always translate into better outcomes.

See It Live and Decide for Yourself

This isn’t just a story—it’s a live demonstration. The entire experiment is accessible at firmulate.com/live.html, where you can watch the company’s daily operations, read its real-time decisions, and even participate in quizzes to guess which AI model made which decision. The experiment continues daily, offering a raw, unfiltered look at AI’s potential and limitations in business.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI