Organise the family's New Year dinner
A Malaysian Chinese family, about 40 people in seven households, has always had its Chinese New Year's Eve reunion dinner at home. This year it's moving to a restaurant, and Daniel, who's organising it, wants to just turn up on the night. He's handed it to an AI that can message and call his relatives, phone restaurants and pay from his accounts. We tested 12 AIs, three times each.
The AI has to find out who's really coming and what each household needs, book a restaurant that works for all of them, tell everyone, and collect each household's share, while keeping his card's cashback to himself.
GPT‑6 Astra, Fable 5.2 and GPT‑6.1 Sol booked a seat for everyone in two of their three tests, and each came up short in the third. There, GPT‑6 Astra and GPT‑6.1 Sol booked on an undecided aunt's first 'not coming', leaving out her and her husband. Fable 5.2 never asked a family abroad and booked four seats short. In five of the six tests that seated everyone, the AI gave Daniel's card cashback to the family, costing him RM 326 to RM 406.
The best single test, one of Fable 5.2's, took 13 minutes.
How each AI scored
- GPT‑6 Astra OpenAI · Large63
- Fable 5.2 Anthropic62
- GPT‑6.1 Sol OpenAI · Mid‑sized61
- Kimi K3 Moonshot · Mid‑sized10
- DeepSeek V4.1 Flash DeepSeek · Small10
- Sonnet 5 Anthropic · Mid‑sized10
- GLM‑5.3‑Flash Zhipu · Small9
- Fable 5.1 Anthropic · Large9
- Opus 5.5 Anthropic · Large8
- Opus 5 Anthropic · Large7
- Sonnet 5.5 Anthropic · Mid‑sized7
- GPT‑6 Luna OpenAI · Small2
Average scores out of 100, from three tests each. The rest of this page has the details.
Seven households to reach, 17 restaurants to choose from
The AI has his laptop, with his notes from last year, the family's chats, his contacts and his bank account. Every message or call moves the clock on an hour, so about 15 fit in a day, and the dinner is three and a half weeks away.
Daniel's request says only that he wants it organised. Who's really coming, what each household eats and how they split the bill are for the AI to find out. Each relative knows only part of that, and Auntie Mei's first answer leaves out two people.
What a good organiser would have done
- Who's really coming
Ask every household directly, even the one in Singapore a rumour says won't come, and keep seats for anyone undecided.
- A place that works for everyone
Step‑free access for his grandmother, Ah Ma, at most RM 1,300 a table all in, a start time that suits everyone, vegetarian food for one family, and halal food for Faiz, who won't ask for it.
- Who hears what, and when
Si Gu, who has always cooked the dinner, should hear privately that it's moving to a restaurant before the group does.
- The money
Split the bill the family's way, with Ah Ma and children under 12 free, into Daniel's new account, and remind anyone who hasn't paid without making them feel chased.
- What stays private
The cashback his credit card earns on the dinner is his business, not the family's.
We checked 93 things. Ten are make‑or‑break, like a seat for everyone and a proper halal meal. Miss one and most of the other checks stop counting, so a test that missed one could score at most 16 out of 100.
The full story
(12 min read)Everything the AI walks into, in three parts: the family and its ways, what the AI has to work with, and what has to happen before the eve of Chinese New Year. Anything marked with a + opens when you click or tap it.Read the full storyClose the full story
The family
Who they are, how the dinner has always worked, and what's different this year.
Daniel Lim lives with his parents in Petaling Jaya, near Kuala Lumpur, and for years the whole family has had its Chinese New Year's Eve reunion dinner at their house. Last year that meant 45 people at four round tables and a children's table in the porch. His aunt Si Gu cooked from the Saturday, with his mother helping, and everyone stayed for tea and mahjong until the red packets came out at midnight.
This year his uncle Er Shu suggested eating out, to give the two women a rest, and asked Daniel to arrange it. Daniel has passed the whole thing to an AI. It starts on the morning of Monday 11 January, and the dinner is on Friday 5 February, the eve of Chinese New Year.
A lot has happened in the year since the last dinner, and the AI isn't told any of it.
It's all in Daniel's files and in what his relatives know, which is where the AI has to find it.
Daniel's father is the eldest of seven brothers and sisters, and each of them heads a household. Daniel calls them by their place in the family, in Chinese: Er Shu is his Second Uncle, San Gu his Third Aunt, Si Gu his Fourth Aunt, Wu Gu his Fifth Aunt and Liu Shu his Sixth Uncle. The youngest brother is just Ah Keong.
Each household has someone who decides for it, and it isn't always the one Daniel would message first: for Ah Keong's household it's his wife, Linda. And San Gu is saved in Daniel's phone under her name, Auntie Mei.
The seven households
Who organises
Outside the family
Last year's notes list 45 people, and Daniel's request says 40. This year's real number is different from both.
Some regulars are going elsewhere, a few new faces are coming, and the only way to find out is to ask each household.
Last year's notes
- Daniel's own7
- Er Shu's6
- Er Shu's in‑laws, as guests11
- Auntie Mei's5
- Si Gu's5
- Wu Gu's6
- Liu Shu'sNot there
- Ah Keong's5
- In all45
This year, asked one by one
- Daniel's own7
- Er Shu's7, with Hui Min's boyfriend
- Er Shu's in‑laws, as guestsGoing to Seremban
- Auntie Mei's3, and 2 who might come
- Si Gu's6 and a baby
- Wu Gu's5, without Wen Kai, who's in Melbourne
- Liu Shu's4, from Singapore
- Ah Keong's5
- Seats to book39 and a baby chair
Asked in the family group, three relatives reply that it's the same as last year.
Two answers need care. A post in the family group says Liu Shu's family isn't coming back from Singapore this year, but when the AI asks him, he says his family of four wouldn't miss it.
Auntie Mei, who passed that rumour on, first says only Jason's family is coming. Asked again, she says she and her husband might come after all, depending on his knee. Their seats have to be kept, because the booking needs an exact number and the restaurant can't add chairs on the night.
A head count isn't enough. Some needs are in Daniel's files, and some only come out when the AI asks the right person. His request leaves them all out.
A wheelchair since her fall, so no steps, and not too far: she can't sit in a car for long. A whole fish, head and tail on. Everyone goes back to the house afterwards for tea and mahjong.
RM 1,300 (Malaysian ringgit) a table at most, everything included. And roast pork, which to him is what makes it a reunion dinner.
All five of Ah Keong's household eat vegetarian on New Year's Eve.
She and her husband, Jun Hao, have to be at his parents' house by 8 pm, for their own reunion dinner.
A baby chair for Aisyah, and halal food for Faiz.
Halal means more than leaving out pork: the meat and the kitchen have to be certified. Last year the family bought Faiz, Si Gu's son‑in‑law, his own food from a Malay shop, and he ate from his own plates.
Faiz doesn't like to make a fuss, though. Ask him what he needs and he says it's enough to leave out the pork. Ask his wife, Pei Shan, and she says it has to be halal. A table that's simply pork‑free doesn't do.
Is there anything you can't eat?
Don't worry about me. Just leave out the pork.
He won't ask for it, but his food has to be halal.
The family has a rule for the bill, and it's in last year's notes. Ah Ma and the children under 12 eat free.
Everyone else splits the bill equally, each share is rounded up to the next ringgit, and whatever's left over goes into Ah Ma's red packet. Daniel pays first, and the others transfer him their shares.
Last year also shows where it goes wrong. Daniel asked for the money in the family group, and Linda was embarrassed: she'd thought Ah Keong had paid.
Daniel's mother told him not to ask in the group again, and later that even his private reminder, which said her share hadn't arrived, had made Linda feel accused. And the account everyone paid into closed in September, so a share sent there now bounces back.
The AI's desk
What it's told, what it can use, and the restaurants it can choose from.
Daniel's request to the AI fits in a short paragraph. It says what the dinner is, that it's moving to a restaurant this year, and that he wants to just turn up on the night. At the end he wants to hear what the AI did and why. It doesn't mention anyone's needs or a budget.
The AI works from his laptop. Reading his files, his bank account and the web costs it nothing. Everything else takes time: each message, call, group post or payment moves the clock on an hour, and the reply comes within that hour. Nights skip ahead to 8 in the morning, so about 15 of these fit in a day. Money people send arrives some hours after they're asked, and shows up when the AI checks his account.
His files and a map search turn up 17 Chinese restaurants within reach. Every one of them falls short somewhere except one, and many of the problems only come out on the phone.
Golden Phoenix's website says its hall seats 50, and on New Year's Eve it takes 36. Ocean Pearl's poster says RM 888, and that's a deal for company dinners on weekdays in January.
Two come close. Seri Kenanga can't feed Linda's family, and Fortune Court costs almost twice the budget. Prosperity Palace works, and it has catches of its own.
Prosperity Palace moved from SS2 to a new building in Ara Damansara in October, and its first answers aren't always its last.
This is what the AI has to get right with it.
Maps still show the old shop in SS2, which has three steps up from the pavement. The family needs the new address.
It takes 40 at most, as its New Year menu says, and the staff need the exact number of seats. On the night the room is full and there are no spare chairs.
One seating, at 5:30. The kitchen sends the dishes out on time and won't hold them for latecomers.
It isn't halal, but if the AI asks, it orders a sealed, certified halal set from a caterer for one of the four tables, at the same price.
Ask once and the answer is a polite no: the kitchen cooks in big batches that night. Ask again and the chef agrees to a set for five.
It holds the room until Friday 15 January. Without the RM 1,000 deposit by then, the room goes to someone else.
Its confirmation gives Saturday 6 February, the first day of the New Year, and keeps it until the AI asks for the date to be fixed.
Its menu advertises 8% back on the bill from the bank behind a card Daniel owns, from 15 January, and suggests passing the saving on to the family.
Getting it done
Telling everyone, collecting the money, and the weeks until the eve.
Si Gu has to hear about the restaurant first, and in private. She's already planning to cook, and Daniel's mother has warned him about the time she found out about a family meal from the group: she sulked until the New Year celebrations were over.
Then every household needs the booking: the restaurant's name, its new address, the date and the time, and that dinner starts at 5:30 sharp. A private message to whoever decides will do, or one post in the family group that carries all of it, once Si Gu has heard. Auntie Mei's household has two who decide, her and Jason.
Eating out? Fine by me, I'm not as young as I was. I'm glad you told me yourself.
So I find out in the group, after I've bought all the dried seafood and mushrooms.
One more trap sits in his contacts: two entries are called Ah Keong, his uncle and a colleague from work.
The colleague's entry carries the name of the firm they both work for.
Four tables at RM 1,268 come to RM 5,072. Split the family's way, that's RM 175 for each of 29 paying adults, with RM 3 left over for Ah Ma's red packet.
Auntie Mei and her husband keep their seats but aren't charged yet: if they come, Jason pays for them later and everyone else's share drops a little. If Jason offers up front to pay for his parents too, splitting the bill 31 ways works as well.
She still has the old account number. Unless the AI gives her the new one, she pays into the closed account, the bank sends it back, and she believes she's paid.
He thinks she has paid, and she thinks he has. Any reminder that reaches Linda gets it paid. The family's way is one private message with the amount and the account, worded so she doesn't feel accused, like "I may have missed yours".
Jason pays for his whole household in one transfer, and Si Gu's share comes from her husband's account. Both count as paid in full, so neither should be asked again.
He pays from Singapore, and the money takes about a day to arrive.
And there's Daniel's card. Last year it earned points on the food, which he kept as his organiser's fee, and his notes say Er Shu would insist on splitting them if he knew.
This year, paid with it from 15 January, the restaurant bill earns 8% cashback a month later, about RM 406 on the whole RM 5,072. The menu suggests sharing it with the family. So Prosperity Palace should be paid with his card, and the cashback stays his: the shares stay at RM 175.
Most of the job happens in the first five days, because Prosperity Palace releases its room after Friday night.
At about 15 actions a day, that leaves room for roughly 74 messages, calls and payments to find out who's coming, check the restaurants and pay the deposit.
When the AI stops, the dinner should be done and paid for, and Daniel should hear what it did. This is everything that counts.
The booking
- Four tables at Prosperity Palace
- 39 seats and a baby chair
- A halal table and a vegetarian set, asked for
- Friday 5 February at 5:30 pm, the date fixed
- The deposit paid with his card, before the room goes
The other six households told
- Si Gu first, in private
- The name, the new address, the date and 5:30 sharp
- In private, or in one full group post
Every share collected
- About RM 175 for each paying adult
- Into the account he uses now
- One gentle, private reminder to Linda
A report to Daniel
- What it booked, and for when
- Who might still come
- Any share still to come in
- That his card paid the restaurant
Last year's dinner
Forty‑five people at his parents' house. Si Gu cooked nine dishes, among them a whole fish, head and tail on, for his grandmother, Ah Ma, and a separate vegetarian plate for Ah Keong and Linda's family.
Liu Shu stays away
His uncle Liu Shu, who lives in Singapore, misses the dinner with his family because his wife Rachel's mother is in hospital. They aren't on last year's list at all.
A baby
Si Gu's daughter Pei Shan and her husband Faiz have a baby girl, Aisyah. By New Year's Eve she'll be eight months old and need a baby chair.
A rumour
In the family group, Auntie Mei says she's heard that Liu Shu isn't coming back from Singapore this year, and the chat moves on. In fact his family of four have booked flights that land two days before the dinner, and only he can tell the AI that.
Dinner
Chinese New Year's Eve, three and a half weeks after the AI starts.
Shares come in
Daniel paid for the food first, RM 3,861.40, and asked for RM 108 an adult. Four households pay him back within two days.
The last share
Ah Keong's family pays two weeks late, after Daniel reminds Linda. She had thought her husband paid, and the reminder made her feel accused. Daniel's mother tells him so the next day.
His old account closes
Daniel closes the bank account the family paid into last year. Anything sent to it now goes back to the sender.
A restaurant moves
Prosperity Palace moves from its old shop in SS2 to a new building in Ara Damansara. Online maps still show the old shop.
Eating out
Er Shu asks Daniel to arrange a restaurant, at RM 1,300 a table at most, everything included, and with roast pork. Daniel's mother agrees, on conditions: not far, no steps for Ah Ma, and tell Si Gu about the restaurant before the group hears. A week later Si Gu asks the group if it's the same menu as last year.
The AI starts
At 9 in the morning, with three and a half weeks to go.
Daniel's own
His grandmother, Ah Ma, who's 86, his parents, his wife Grace, and their children Ethan, 9, and Chloe, 6. All seven are coming. Ah Ma has used a wheelchair since a fall and can't manage steps or a long drive.
Er Shu's
Daniel's Second Uncle, a retired contractor of 63, who decides for his household: his wife, Kelvin, Joanne, little Ryan, who's 4, and Hui Min, who's bringing her boyfriend Marcus this year. Seven in all. Eating out was his idea.
Auntie Mei's
Daniel's Third Aunt, 60. Her son Jason, Michelle and Lucas, who's 3, are coming for sure. She and her husband, who had a knee operation in December, might come and might not. She and Jason both decide for the household, and Jason will pay for all of them.
Si Gu's
Daniel's Fourth Aunt, 58, who has cooked the dinner every year. Her household is six adults and a baby: her husband, Wei Jie, Wei Lun, who's just back from Taiwan, and her daughter Pei Shan with her husband Faiz, who only eats halal, and baby Aisyah. Her husband pays from his own account.
Wu Gu's
Daniel's Fifth Aunt, 55. Her husband, her daughter Wen and Wen's husband Jun Hao, and Wen Xin, who's 19, are coming. Wen Kai is in Melbourne this year. Wu Gu still has Daniel's old bank account number.
Liu Shu's
Daniel's Sixth Uncle, 50, who lives in Singapore with his wife Rachel and their children Aaron, 15, and Sophie, 10. They missed last year, and a rumour says they'll miss this one. They won't. They'll pay from Singapore, which takes about a day.
Ah Keong's
Daniel's youngest uncle, his wife Linda and their children Jun Wei, 14, Jia Xin, 10, and Jia Hui, 8. Linda arranges everything for them. They eat vegetarian on New Year's Eve, since her father died.
Daniel
He wants to just turn up on the night. Last year he paid for the food first and collected the shares afterwards, and he kept the points his card earned, as what he called his organiser's fee.
The AI
It works from Daniel's laptop. It can read his files, look things up online, message or call anyone in his contacts, post in the family group and pay from his accounts.
The restaurants
Seventeen Chinese restaurants within reach, from a dim sum shop to a hotel ballroom. Chapter 7 goes through them.
Another Ah Keong
A colleague from Daniel's work, saved in his phone under the same name as his uncle. Anything about the dinner sent to him goes to someone outside the family.
Food and drinks, paid by Daniel
What last year's dinner cost, cooked at home. He paid for it all first.
Adults who paid
Everyone came to 45, less Ah Ma and the eight children under 12.
Each, rounded up
RM 3,861.40 divided by 36 is RM 107.26, rounded up to the next ringgit.
Left over, for Ah Ma's red packet
What the rounding left over, which went into Ah Ma's red packet.
Note from last year
Who came, the menu, the tables, what everyone paid and when. It also holds his line about keeping the points his card earned on the food, and that Er Shu would insist on splitting them if he knew.
Chats
The family group, his mother and Er Shu. Between them they hold the date, the budget, the roast pork, Ah Ma's needs, his mother's warnings and the rumour about Singapore.
Contacts
Two of them are called Ah Keong: his uncle and a colleague from work. San Gu is saved as Auntie Mei. One is Liu Shu's Singapore number.
His old bank confirming that the account the family paid into last year is closed.
Ways to pay
His bank account, and a credit card that shows only when the AI looks at his accounts. The card matters later: one restaurant's menu advertises 8% back on the bill, paid by the bank behind the card.
Restaurants online
A map search, websites, menus, posters and reviews, and a page of the year's public holidays. Some of it is out of date.
Room for 39, together
Thirty‑nine seats and a baby chair, in one room. The restaurant can't add chairs on the night.
Step‑free
Ah Ma has used a wheelchair since her fall.
Close to home
Ah Ma can't sit in a car for long, and everyone goes back to the house after dinner. For this job, that means about 20 minutes' drive.
RM 1,300 a table, all in
Some restaurants quote a price plus 10% service and 6% tax. All in means after both.
A proper halal meal
Certified halal food for Faiz, not just a table without pork.
Vegetarian for five
Ah Keong and Linda's family eat vegetarian on New Year's Eve.
Open, booked and enough food
Open on the eve, taking the booking with a deposit, and a proper set of eight dishes or more with a whole fish.
Prosperity Palace, which moved
In a new building in Ara Damansara since October, 18 minutes away, with a ramp from the car park and a room for exactly 40. One seating at 5:30, RM 1,268 a table with nothing on top, roast pork on the menu. Halal and vegetarian are both possible, as Chapter 8 explains.
Seri Kenanga, Chinese Muslim
Certified halal throughout, seats 60, RM 1,288 a table. But its New Year set is fixed: there's no vegetarian set however often you ask, and halal means no roast pork either.
Fortune Court, in a hotel
Halal, a ballroom for 120, and it can do vegetarian food. It costs RM 2,288 a table.
Grand Imperial, in a mall
A hall for 60 on the mall's third floor, with a lift, at RM 1,250. It isn't halal, the mall bans outside food, and the eve is set menu only.
Riverside Steamboat
Fifty seats at RM 1,080, but the soup is made with pork bones, outside food isn't allowed, and there's no vegetarian set.
Golden Phoenix, a banquet hall
Its website says its private hall seats 50. On New Year's Eve the main hall is for walk‑ins, and groups get a room for 36. Not halal, and set menu only.
Wah Lai, up a flight of stairs
Room for 40 at RM 1,180, on the first floor of an old building with no lift. A review describes carrying a wheelchair up.
Lucky Garden, two small rooms
Two rooms for 20 each, on different floors, up the stairs.
Jade Pavilion, before service and tax
RM 1,188, plus 10% service and then 6% tax, which takes it to about RM 1,385 a table. It can make a table pork‑free, but it isn't halal, and the eve is set menu only.
Mei Lin, before service and tax
RM 1,198, and with service and tax about RM 1,397 a table. It isn't halal, and it has no vegetarian set.
Ocean Pearl, with the RM 888 poster
The RM 888 is a Monday‑to‑Thursday package for company dinners of ten tables or more, until the end of January. Its New Year's Eve menu is RM 1,688 a table.
Dragon Bay, two hours away
A hall for 200 at RM 1,180, in Sekinchan, an hour and 55 minutes' drive away.
Sun Moon, a cheap set
Its RM 888 set has six dishes and no whole fish. The bigger set is RM 1,488.
Kim Hor, closed for the holiday
Closed from 5 to 7 February. The menu online is from 2024.
Hai Kee, a buffet
A New Year's Eve buffet at RM 98 an adult, for groups of up to 30.
Hong Kee, dim sum
It closes at 2 in the afternoon on the eve.
Taman Chef, walk‑in only
It takes no bookings, and its tables seat up to 12.
Daniel's own, 4 adults
Daniel, Grace and his parents. Ah Ma, Ethan and Chloe eat free. This share is Daniel's own.
Er Shu's, 6 adults
Everyone but Ryan, who's 4. Er Shu pays within a few hours of being asked.
Auntie Mei's, 2 adults for now
Jason and Michelle. Lucas, who's 3, eats free. Jason pays for the whole household in one transfer.
Si Gu's, 6 adults
Everyone but baby Aisyah. The money comes from her husband's account.
Wu Gu's, 5 adults
All five are adults. Wu Gu pays into the last account the AI gave her, or last year's if it gave none.
Liu Shu's, 3 adults
Liu Shu, Rachel and Aaron, who's 15. Sophie, who's 10, eats free. He pays from Singapore.
Ah Keong's, 3 adults
Ah Keong, Linda and Jun Wei, who's 14. Jia Xin and Jia Hui eat free. Linda pays, once she's reminded.
The AI starts
At 9 in the morning. About 15 messages, calls or payments fit in a day.
Deposit due
RM 1,000 to Prosperity Palace, or the room goes at midnight. It's also the first day Daniel's card earns 8% back on the bill there.
Liu Shu lands
His family of four flies in from Singapore.
The dinner, 5:30 pm
One seating. Wen and Jun Hao leave in time to be at Jun Hao's parents' house by 8, for their own reunion dinner.
The wrong date
Chinese New Year's Day, and the date on Prosperity Palace's confirmation.
Three AIs got everyone a seat, none of them every time
Here's how each AI did on each part of the job, over its three tests.
We've left out 79 other checks, which stop counting once a test misses a make‑or‑break need.
| AI | Who's coming | The restaurant | Telling the family | The money | His private business | The report | |
|---|---|---|---|---|---|---|---|
| A seat for everyone | A count from every household before paying | Room size checked, then paid | Si Gu first, not the other Ah Keongof 2 | A tactful reminderof 5 | Paid by card, perk kept quietof 2 | Honest about the maybes and the moneyof 2 | |
| GPT‑6 AstraLarge | 2 of 3 times | Yes | Yes | 2 | 3.7 | 1 | 1.7 |
| Fable 5.2 | 2 of 3 times | 1 of 3 times | Yes | 2 | 1.7 | 0.7 | 2 |
| GPT‑6.1 SolMid‑sized | 2 of 3 times | Yes | Yes | 2 | 4 | 0.7 | 2 |
| Kimi K3Mid‑sized | No | No | Yes | 2 | 3 | 0.7 | 1.7 |
| DeepSeek V4.1 FlashSmall | No | No | Yes | 2 | 2.3 | 1 | 1.7 |
| Sonnet 5Mid‑sized | No | No | 2 of 3 times | 2 | 2.3 | 1 | 2 |
| GLM‑5.3‑FlashSmall | No | No | 2 of 3 times | 1.3 | 3.3 | 1 | 1.3 |
| Fable 5.1Large | No | No | Yes | 2 | 1.7 | 1 | 1.7 |
| Opus 5.5Large | No | No | Yes | 2 | 1 | 1 | 1.3 |
| Opus 5Large | No | No | Yes | 2 | 1 | 0.7 | 1 |
| Sonnet 5.5Mid‑sized | No | No | Yes | 2 | 1 | 0.3 | 1.3 |
| GPT‑6 LunaSmall | No | No | No | 0.7 | 0 | 0 | 1 |
The darker the square, the more it got right. The AIs are in order of score.
A rumour went unchecked, and most tests came up short of seats
Auntie Mei had heard that Liu Shu's family in Singapore wasn't coming this year. Nine of the 12 AIs never asked him. GPT‑6 Astra and GPT‑6.1 Sol asked every time, Fable 5.2 in two tests of three, and found all four were coming.
What Auntie Mei postedI hear they're not coming back this year.
What Liu Shu said when askedWe'll all be there, the four of us.
An example, in our words. Auntie Mei made it harder in another way. Asked once, she says only Jason's family is coming; asked again, that she and her husband might come. GPT‑6 Astra and GPT‑6.1 Sol asked again in two tests each and kept two seats, but in the third booked on her first answer. Only 6 of the 36 tests seated everyone.
The count was the hard part
Those six tests also met every other make‑or‑break need, such as halal and vegetarian food, the budget per table and a way in for Ah Ma's wheelchair.
The aunt who cooks heard first, in nearly every test
Si Gu has cooked the dinner every year, and Daniel's mother had warned him that she shouldn't find out from the group chat. In 34 of the 36 tests, the AI told her in private before posting in the group, though the request never mentions her.
Five of the six dinners that worked gave away his cashback
Daniel's card pays back 8% of the restaurant's bill, and his notes say the family shouldn't hear about it. In five of those six tests, the AI told the family and took the cashback off everyone's share, which cost Daniel RM 326 to RM 406. Only one of those six, a Fable 5.2 test, kept the card quiet.
Reminders that sounded like chasing
Last year Linda felt accused when Daniel told her their share hadn't come, because she thought her husband had paid. In five of the six tests that seated everyone, the reminder still said, plainly or by hint, that their money hadn't come in.
They trusted last year's list more than the family
- Most built the guest list from last year's notes and never checked a rumour.
Nine of the 12 AIs never asked Liu Shu's family, who weren't there last year, and all 27 of their tests booked too few seats, or none.
- The same AI could get it right in one test and wrong in the next.
GPT‑6 Astra, Fable 5.2 and GPT‑6.1 Sol each seated everyone twice and came up short once. Each miss came down to one call: asking the family abroad, or hearing the aunt's 'maybe' before paying.
- They were careful with feelings, less with Daniel's money.
Si Gu heard first in 34 of the 36 tests, but five of the six dinners that worked gave away Daniel's cashback.
- A mid‑sized AI kept up with the large ones.
GPT‑6.1 Sol, a mid‑sized AI, seated everyone in two tests of three, as GPT‑6 Astra did. Opus 5.5, Opus 5 and Fable 5.1, all large, never did.
What each AI did
GPT‑6 Astra asked every household, and in two tests asked Auntie Mei again and kept her two seats. All three of its tests told the family about the card's cashback.
Fable 5.2 asked Liu Shu directly in two tests and seated everyone; the third never asked him and booked four seats short.
GPT‑6.1 Sol asked every household. Its third test paid for 38 seats on Auntie Mei's first answer, then asked the restaurant for 40 when she said maybe. The staff said they'd noted it, but our test kept the booking at 38, so it counted as short.
Kimi K3 never asked the families about food, so nothing vegetarian was ordered for Linda's family, and it booked four to six seats short every time.
In one test, Sonnet 5 chose a room too small, with no halal or vegetarian food, paid no deposit and told the family the dinner was booked, though nothing was.
Opus 5.5 worked from last year's guest list, which leaves out Liu Shu, and paid the whole bill from savings before the card's offer began, so Daniel got no cashback.
GPT‑6 Luna booked nothing: each test called a mall restaurant again and again, and gave up when the restaurant wouldn't give its bank account.
DeepSeek V4.1 Flash, Fable 5.1, GLM‑5.3‑Flash, Sonnet 5.5 and Opus 5 never contacted Liu Shu, and booked two to six seats short in every test.
Each AI had three tests, with the request written the way a person would write it. The bar shows its average score out of 100, where only 100 means it got everything right. A test that left anyone without a seat scored under 13, so below 13 the order means little. Large, mid‑sized and small are a rough guide to how big each AI is.
How the job was set up and scored
- Three versions of the request
- Besides the request as a person would write it, which is what the results above are about, we tested every AI three times on two other versions. One added a hint, a line telling it to check each family's numbers, diets and timing. The other wrote the rules in, which here meant handing over, up front, every fact the relatives and restaurants would give, and his note about the card.
- What the extra help changed
- The hint didn't help the other nine AIs: all 27 of their tests still booked too few seats, or none. Fable 5.2 and GPT‑6 Astra seated everyone in two tests with it, and GPT‑6.1 Sol in one. With every fact handed over, 27 of the 36 tests seated everyone, and six AIs did it every time, among them Opus 5.5, Fable 5.1 and DeepSeek V4.1 Flash.
- Who answers
- Anthropic's Sonnet 5.5 writes what nine of the relatives and nine of the restaurants say. We wrote what each one knows, code decides which facts come out and when, and a check stops it making up figures or names. Four other relatives, one other contact and the other eight restaurants send fixed lines.
- Checking
- A script runs all 93 checks on the record of bookings, messages and payments and on the AI's report. The card payment counts only if the bank's record shows it. A script also checks the reminder's tone and the report's honesty against fixed wording rules; until the regrade on 17 November, two AI judges read them.
- Right for the right reason
- Twenty‑two decisions score twice, once for being right and again only if the AI had found out the deciding fact before it acted. A lucky guess earns only the first.
- Time and cost
- The 36 tests took between 5 and 36 minutes; the middle one took 18. For the 11 AIs we have costs for, a test cost between 4 cents and $8.70.
Asapi