How we measured what AI recommends

What we were trying to find out

One question: when someone asks an AI assistant to recommend a security company in Oklahoma City, how much of the real market is even eligible to come up?

Not which company is best. Not which is most popular. Just how many of the businesses actually operating here the assistant is capable of naming at all.

Building the list of companies

We started from business listing data for security companies operating in the Oklahoma City metro, not from anyone’s recommendations. That matters, because a list built from recommendations would already be biased toward the companies that get recommended.

A sweep like that always drags in businesses that are not really in the trade — a locksmith, a guard service, an electronics store that happens to sell a camera. We reviewed those by hand and removed verified out-of-trade entries before anything was ranked. That exclusion list is fixed in advance and applied to every company equally; no individual result is ever hand-edited after the fact.

How we asked

We asked ChatGPT for a recommendation 4 different ways. This is the part most casual tests skip, and it turns out to matter more than anything else: the answer moves when the wording moves. A single question asked once tells you almost nothing.

All 4 phrasings were the kind of thing a real person types — asking for a good company, asking who to call, asking about home security specifically. Every answer was captured in full, including the ones that named nobody.

Matching names back to real companies

An assistant writes company names loosely. It drops a suffix, adds “Inc”, abbreviates, or uses a name close to but not quite the registered one. Matching that text back to a real business is where a measurement like this goes wrong.

Our matcher strips the generic words that nearly every company in this trade shares — “security”, “alarm”, “systems”, “services” — before comparing, and it refuses to declare a match on a single remaining shared word. That guard exists because an earlier version of it did exactly that and credited answers to the wrong companies. Every number on this site was produced after that fix.

The rule that changed the result

A company only counts as real if an independent business listing backs it up. If the sole evidence that a business exists is that the assistant said its name, we do not count it as a company — we count it as an unverified name.

This is not a technicality. In Oklahoma City, 5 of the 15 names produced had nothing behind them. Counting those as real companies would have made the assistant look considerably better informed than it is, and would have told you a business exists when we cannot show that it does.

What the numbers do and do not support

• They support a statement about coverage — how much of the market the assistant can name.
• They do not support any statement about quality. Nothing here measures how well a company installs or services anything.
• They describe one assistant, in one metro, in one month. Assistants change. The date on the page is not decoration.
• They are not a ranking. No company was named in all 4 phrasings, which is itself the finding.

A number we removed

An earlier draft of this project carried the claim that 92% of 1,882 Oklahoma security companies had never been named by ChatGPT. We checked it before publishing and it did not survive: the 1,882 figure was the size of a national company list from a different, earlier measurement; that list was later found to have been cut short by a software bug and was rebuilt at a different size; and no Oklahoma-only measurement of that scale was ever run.

We mention it because the same thing is probably true of other numbers you will read about this industry, and because a site that asks you to check its sources should show you the one it threw away.

When it gets re-run

We re-run the measurement and update the page rather than leaving an old number in place. Each figure carries the date of the run it came from. If you find a figure on this site without a date attached to it, that is a mistake and we would like to know.