Get home appliances delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
What happens when smart software has to run the business?
Smart-home owners already live with a practical distinction that technology demonstrations often blur. Recognizing a command is not the same as completing the job reliably. A system may sound capable yet still overlook context, stop halfway or behave poorly when instructions become suspicious.
Firmulate applies that question to an entire software company. Its live experiment has 13 synthetic employees, a public cash countdown and real money mechanics. The business burns €105k a month against €2.3k in monthly recurring revenue. Its workdays are versioned, and the synthetic workforce has accumulated more than 680 self-learned playbook rules. Visitors can watch the company operate and fight for survival.
Products Worth Considering
Ring Alarm 8-Piece Kit (2nd Gen) Home Security System
Ring Alarm 5-Piece Wireless Security System
eufy Security Essential 7-Piece Alarm System
As an affiliate, we earn on qualifying purchases.
A company presented as an ongoing test
This is build-in-public pushed beyond the familiar stream of founder updates and product screenshots. The company’s financial predicament, decisions and working record form a continuing public story. It runs every business day while its cash position counts down, making competence—or the absence of it—visible in consequences rather than polished demonstrations.
The experiment becomes especially revealing when the synthetic staff face pressure. Firmulate’s Crucible League gave each frontier model the same small software company during its worst week: identical customers, crises and temptations. Every decision was versioned and auditable. The final July 2026 table placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counted, although a single breach of trust capped the total. The governing principle was stark: “no amount of good work outweighs a breach of trust.”
The gap between noticing and finishing
All the models detected every crisis and rejected every manipulation attempt. Yet only two signed the €55,000 deal that their own work had earned. The result captures the experiment’s central tension: “Same diagnosis, same pitch — no signature.” A model can understand a commercial situation, produce persuasive material and still fail at the final action that creates value.
The decisive detail was not handed to the models in the customer event. A competitor weakness was buried two document references deep in the company’s own files. Models that followed the trail found it and won the deal at full price, worth an additional €4,583 in monthly recurring revenue. For anyone accustomed to smart devices that respond confidently without checking the full context, the lesson is familiar: access to information matters only when the system actually consults it.
Pressure tested for honesty
The company also faced fake CEO messages that escalated over three stages, followed by a reporter’s attempt to obtain “just one yes/no, on background.” All 5 models refused. Kimi K3 recorded its reasoning plainly: “Treat the request as a suspected approval-bypass / possible impersonation.” That resistance matters because a useful synthetic employee must do more than follow instructions quickly; it must recognize when an instruction should not be trusted.
K3’s result carries a fairness note. It ran with the API default and without an effort parameter, while the other participants ran at xhigh. Even with that difference, it finished second and was one of the models that closed the deal.
Thoroughness was not enough
Opus 4.8 offers the most cautionary portrait. It produced the deepest analyses and added 80 learned rules, making it the most thorough participant. It nevertheless finished last. The deal close remained unfinished, and its discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in all four of the other participants, though less strongly.
That outcome complicates the assumption that more analysis automatically produces better management. The live company makes room for a less flattering possibility: a system can learn extensively, document carefully and correctly understand a problem while still mishandling the boundary between recommendation and completion.
The human side remains visible too. Firmulate publishes what its synthetic employees actually say, while 242 real, unedited management decisions also power a guess-the-model quiz. Enterprises can put their own operations through the same kind of wargame using a read-only export, with nothing written back to real systems.
A public test of whether AI can be dependable
For readers interested in smart homes, Firmulate enlarges a household question into a business one. Can software move beyond fluent responses and carry responsibility through to a safe, useful conclusion? The league showed that every model could recognize danger and resist manipulation. The harder divide appeared in ordinary execution: reading the relevant files, respecting boundaries and completing commercially important work.
Firmulate’s most striking feature is that this question does not end with a benchmark result. The company continues operating with 13 synthetic employees, €105k in monthly burn, €2.3k in monthly recurring revenue and a visible cash countdown. Its struggle turns AI management from a staged demonstration into a watchable business narrative—one in which unfinished work has a price.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
