I Asked Three AI Assistants About Twelve Local Businesses

Last week I ran a small experiment. I picked twelve local businesses whose real facts I know — places I use, owners I’ve talked to, a mix that included a bakery, two plumbers, a bookkeeper, a dog groomer, and a handful of others I’ll keep vague on purpose. Then I asked three AI assistants the three questions that are now on manual page 3.4:

1. “What does [business] in [town] do?”
2. “Who’s a good [category] near [town]?”
3. “Is [business] in [town] any good?”

Twelve businesses, three questions, three assistants. I checked every answer against what I knew to be true. This is a methods-and-pattern note, not a callout — I’ve sanded the examples down so nothing identifies anyone, including the assistants. The pattern matters more than the names.

The headline: roughly a third of the answers had something wrong in them. Not hallucinated-out-of-thin-air wrong, mostly. Confidently-out-of-date wrong. One business had its old hours, from a schedule the owner changed over a year ago. Another was described as having a second location that closed. The strangest one: a business described, fluently and in detail, by its previous owner’s specialty — work the current owner doesn’t do and hasn’t offered since taking over. Somewhere out there is a directory page nobody updated, and the assistant read it the way you’d read a newspaper.

The recommendation question was its own lesson. When I asked “who’s a good [category] near [town],” the answers skewed hard toward businesses with two things: recent reviews, and plain-language descriptions of what they do. Not the most reviews — recent ones. And not clever descriptions — plain ones. A business whose listing says “a plumber in Millfield that handles emergency calls and water heaters” got recommended. A business whose website says “innovative solutions for your home’s unique needs” did not, because as far as a machine can tell, that business does nothing in particular, anywhere in particular.

And one finding I keep thinking about: an excellent business — genuinely the best in its category around here, the one I’d send my mother to — was simply invisible. Never recommended, barely described. The reason took ten minutes to find: its name is spelled three different ways across the directories. With an ”&” in one place, “and” in another, an abbreviation in the third. A human squints past that. Machines apparently don’t, or at least don’t reliably — three spellings read like three thin businesses instead of one solid one, and none of the three cleared the bar.

Here’s the pattern under all of it. The assistants don’t know any of these businesses. They’ve never eaten the bread or watched the plumber work. They read about businesses — and what they read is the listings, the reviews, and the website. Every wrong answer traced back to one of those three surfaces. Old hours? A listing nobody re-checked. Previous owner’s specialty? A stale directory. Invisible? Inconsistent name across surfaces. The assistant was never the problem. It was the messenger, faithfully repeating the record — and the record was wrong.

Which is good news, because the record is the part you control. You can’t argue with an AI assistant, but you can fix what it reads.

If you want to do something with this: manual page 3.4 walks through running the three lookups on your own business and tracing wrong answers back to their source surface, and the Findability Checkup will tell you which surface is currently costing you the most. The whole exercise takes less time than reading this dispatch did. Twelve businesses in, I’d say it’s worth knowing what the machines think they know about you.

— Layla