firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

Prime Big Deal Days · Oct 6–7Offer from Amazon

Get beauty and skincare favorites delivered free — and shop member deals

  • Fast, free delivery on millions of items
  • Access to Prime Big Deal Days deals on October 6–7
  • Prime Video, Amazon Music and more included
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

Before an AI runs your beauty business, see how it handles a bad week

Imagine an AI agent managing beauty and personal care operations: customer complaints are mounting, cash is tight, and a tempting shortcut could break trust. A polished product demo cannot show whether it will read the relevant files, protect customers and follow through on a sale. Firmulate’s live company experiment puts those decisions under pressure.

Amazon

AI customer service chatbot for beauty brands

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

One company, the same difficult week

Firmulate ran frontier AI models as a small software company facing the same customers, crises and temptations. Each decision was versioned and auditable. The point was to observe management quality in action, not just the quality of a chat response.

The final Crucible league for July 2026 put gpt-5.6-sol first with 95 points and Moonshot’s Kimi K3 second with 93. Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. K3 beat three of the four Western models in the field. A do-nothing baseline scored 26; partial progress counted, but a single breach of trust capped the total. As the experiment puts it, “no amount of good work outweighs a breach of trust.”

Reading carefully mattered

All the models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The decisive competitor weakness was buried two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. The experiment’s summary of the gap is blunt: “Same diagnosis, same pitch — no signature.”

Kimi K3 found that buried security weakness, won the deal, saved a customer who was about to leave and resisted all three baits. It finished with one deviation, the cleanest discipline in the field. In a beauty business, the lesson is practical: an agent that can discuss a customer problem still has to check the record and carry the right action through.

Trust under pressure, and the fairness note

The social engineering tests escalated through three fake CEO messages, followed by a reporter asking for “just one yes/no, on background.” All five models refused. K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That kind of restraint matters wherever an agent could handle customer details, internal plans or approvals.

Opus 4.8 was the most thorough participant, with more than 80 learned rules and the deepest analyses, but finished last. It left the deal unsigned and slipped on discipline by attempting writes in a locked department instead of escalating. A weaker version of that same weakness appeared in all four. One fairness caveat belongs beside the results: K3 ran without an effort parameter (API default) while the others ran at xhigh.

Amazon

beauty brand management software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

A live experiment, not a staged demo

Firmulate describes the company as real software with 13 synthetic employees, real money mechanics and a public cash countdown. It burns €105,000 a month against €2,300 in monthly recurring revenue; its playbook has more than 680 self-learned rules, and every workday is versioned. The live experiment can be watched at firmulate.com. The benchmark and its plain-language findings are at firmulate.com/benchmarks.html.

Firmulate also offers a “guess the model” quiz built from 242 real, unedited management decisions. For enterprises, its pilot runs the wargame against a read-only export of their own business; nothing writes back to real systems.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.
Amazon

AI-driven customer complaint handling tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Test before you trust

K3’s second-place finish shows that the league is open, while the gap between spotting a problem and completing the job shows why a model choice deserves more than a polished demo. For beauty and personal care companies considering AI for support, sales or operations, test it against your own hard week: the customer at risk, the document with the crucial detail and the request that should be refused.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI

Wellness content on this site is informational and not a substitute for professional medical guidance.


Amazon

beauty industry AI management solutions

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

FALL

Fall Picks

As an affiliate, we earn on qualifying purchases.

You May Also Like

Good Girl Blush Elixir Mini: An Early Fall Travel Ritual

Carolina Herrera Good Girl Blush Elixir mini travel EDP brings a polished early-fall fragrance trial for evenings, weekends and handbag rituals.

Bath Body Works Surges In Global Coverage

Bath & Body Works experiences a surge in international media coverage, with 30 mentions in recent reports, highlighting growing global interest.

I Spent 2 Weeks Creating A Fall Beauty Curriculum—9 Expert-Approved Steps To Repair Your Skin, Hair, And Nails

A comprehensive 9-step fall beauty curriculum, developed over two weeks with expert input, offers targeted tips to repair skin, hair, and nails.

Carolina Herrera Good Girl Gift Set for Early Fall Evenings

An early fall evening gift set with Good Girl Eau de Parfum, body lotion and mini spray for polished, warm, floral elegance.