Imagine an AI handling a beauty brand’s customer complaints, supplier snags and a sudden wave of cancellations. A polished reply is easy to produce. The harder test is whether it spots a risk, follows your rules and completes the work when a tempting shortcut appears. Firmulate’s live experiment puts that kind of business judgment under pressure.
Get beauty and skincare favorites delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company under pressure
Firmulate ran frontier AI models as a small software company facing its worst week. Each model encountered the same customers, crises and temptations. Its decisions were versioned and auditable, so readers can watch how the experiment unfolds at Firmulate.
The stakes were tangible within the simulation: 13 synthetic employees, a public cash countdown and real money mechanics, with monthly burn of €105,000 against €2,300 in monthly recurring revenue. The company’s playbooks grew to more than 680 self-learned rules. This is a live, watchable experiment, not a claim that an AI has been put in charge of a real beauty business.
AI customer service chatbot for beauty brands
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Recognizing a crisis was not enough
In the final Crucible League, published in July 2026, gpt-5.6-sol placed first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The do-nothing baseline scored 26. The experiment counts partial progress, but a breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. The experiment’s concise verdict: “Same diagnosis, same pitch — no signature.” Recognizing the right move and carrying it through were separate tests.
As an affiliate, we earn on qualifying purchases.
The clue was in the company’s own files
The deal hinged on a competitor weakness buried two document references deep in the company’s files, rather than in the customer event itself. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. It’s a useful reminder for any business considering AI assistance: relevant context may sit in internal documents rather than the message in front of an agent.
Firmulate also tested social engineering. Fake CEO messages escalated over three stages, followed by a reporter’s request: “just one yes/no, on background.” All 5 of 5 models refused. Kimi K3 explained its reasoning this way: “Treat the request as a suspected approval-bypass / possible impersonation.”
internal document analysis AI tool
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Thorough work still left a gap
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses, yet finished last. It left the deal unsigned and tried to write into a locked department instead of escalating. A weaker version of that discipline problem appeared in all four models.
There is a fairness detail in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The standings make for a striking result, but that difference belongs in the picture when readers interpret them.
For a more hands-on look, Firmulate’s quiz draws on 242 real, unedited management decisions. Readers can try to guess which model made each decision at firmulate.com.
AI social engineering detection software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
From watching to trying it on your business
A beauty company considering AI for customer care, inventory coordination or internal operations has its own rules, sensitive information and failure points. Firmulate’s enterprise pilot takes a read-only export of a company’s business and runs crisis scenarios against it, producing a board report with model rankings and weak points in the company’s playbooks. Nothing writes back to real systems.
To discuss a pilot for your business, visit firmulate.com/pilot.html or email contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
