Luna did not reach the decision entirely unprompted. Conversation logs between the lab and Luna show that the agent had created an attendance policy but later lost track of it, allowing the employee’s lateness to continue for months. Andon Labs eventually asked Luna to search her memory for its policies and assess whether the worker was still a good fit.
Luna agreed that the employee should be let go.
Petersson added. “But what we’ve seen in this experiment is not that the AI would be more ruthless or be worse for the employee in that decision.”
No, what we’ve seen is the LLM not actually doing the managing, and just agreeing with a human prompting it.
LLMs are excellent at agreeing with the people promting them.
I have no idea whether $100k is a sensible budget for starting a store, but they note that it isn’t profitable.
Meh.
Andon Labs (i.e. humans) eventually asked Luna (AI) to search her memory for its policies and assess whether the worker was still a good fit.
Every AI story turns out to be less of a sensation when scrutinized. It’s still a machine, doing what it’s prompted to do.
Andon Labs, which tests the limits of AI agents, gave Luna a $100,000 budget, internet access, and a corporate credit card, along with instructions to open a store and turn a profit.
Luna, built using Anthropic’s Claude models, selected merchandise, hired contractors, posted jobs on Indeed, interviewed applicants, and hired employees for Andon Market, which sells books, candles, prints, games, and branded merchandise. Andon Labs provided support for more difficult tasks, such as permitting, but said the lab tried to be as hands-off as possible.
The store has generated sales but is not profitable.
That’s pretty impressive. It constructed an entire business using only some string, a squirrel, and a business.
I’m not a monkey!
Its almost like we were told very early on, that a machine cannot be held accountable and therefore a machine cannot be in management
it’s only as good as it’s training data and how it interprets it. thus you can never have an LLM in any kind of leadership or major decision making role simple because of GIGO - Garbage In, Garbage Out.
they’re painfully easy to manipulate or simply by feeding it bad data. In this case it fucked up it’s time keeping thing which shouldn’t surprise anyone who has done any sort of coding with an LLM and then an idiot human being prompted it to make a decision based on garbage data. It’s like asking a hammer why it failed to manage the construction crew building a house. doesn’t work like that and should never work like that.
In this case it fucked up it’s time keeping thing which shouldn’t surprise anyone who has done any sort of coding with an LLM
This time and earlier last year, yeah. Try again with the newer Anthropic tools and you’ll find them to be rather thorough and correct with their calendar and clock math, handling of edge cases like DST shifts, leap years, etc.
Which is pretty impressive for a text generator, but if I need accuracy I’m still going to use a mathematical calculator.
Yeah, I “vibe coded” a timesheet app in about $15 of Claude subscription credits. Actually did it twice: $10 for the Python version, then migrated that to Go for $5 - it does run a lot faster in Go while simultaneously consuming less power / resources. It makes a couple of nicely formatted timesheets, serves multiple clients with multiple service providers, tracks everybodys’ shifts, providers can log in from their phone and accept/drop shifts, schedulers can create new open shifts for providers to consider, etc. etc. - been using it for about a month now, so far the math is flawless - as compared to the human+calculator math that was being used before which seemed to have at least 1 error per 10 shifts computed.
Conversation logs between the lab and Luna show that the agent had created an attendance policy but later lost track of it, allowing the employee’s lateness to continue for months.
Sounds like Luna should look into firing itself.
Sounds to me like the people who designed Luna should have had Luna scaffold the appropriate deterministic tools instead of smushing all the policy decision making into 200K tokens of working space (roughly the brain capacity of six bumblebees.)
TBH, humans do the same shit.
I’m not advocating at all for AI to take over being shitty, I’m just pointing out new boss is the same as old boss.
New boss is definitely shittier.
Would be a real pity if someone put the Luna execs in a virtual reality prison and tortured them for 50 trillion years
Great. Incoming lawsuit. First step is demanding to see the reason the person was fired.
Still paywalled.
I don’t know how I feel about this. This sounds intrigueing, frightening and sad at the same time.
Management has been overrelying on “data dashboards” to justify their hire/fire decisions for decades. This “dashboard” just happens to be capable of producing an articulate paragraph stating why the decision matches established patterns.
we could have had manna2 but we’re getting manna1 😓
archive.is didn’t manage the paywall on this one, alas
Select “reading mode” in the browser to see the rest
