We gave an AI system one pricing page. It answered cleanly. Then we added one legacy pricing page. The answer changed. That's the AI search problem in its simplest form.
the experiment
We wanted to test a simple question:
What happens when an AI system has access to both current and legacy business data?
So we used the Kodec Sandbox.
First, we ingested only HubSpot's current Marketing Hub pricing page.
Then we asked pricing questions.
The Sandbox answered cleanly.
Then we added HubSpot's legacy Marketing Hub contact pricing page.
That page lists legacy Starter pricing at $50/month with 1,000 contacts included, plus volume-based contact pricing. It also lists Professional at $800/month and Enterprise at $3,600/month, with onboarding fees for Professional and Enterprise.
Then we asked again.
The answer blended current, promotional, and legacy pricing.
why this matters
This wasn't a giant web crawl.
This wasn't "the model read some random Reddit thread."
This wasn't a third-party reseller.
This was two public pages from the same company.
That's what makes the test important.
If two pages can create ambiguity, imagine what happens across:
- A full enterprise website
- Help docs
- Partner pages
- PDFs
- Archived pages
- Blog posts
- App marketplace listings
- Review sites
- Sales collateral
- Press releases
- Legal terms
- Product catalogs
Now the AI isn't choosing between two facts.
It's reconciling thousands.
That's where the wrong business profile comes from.
the lesson
Legacy pages often need to exist.
The lesson: legacy data needs context.
AI systems need to know:
This page is legacy. This plan applies only to existing customers. This page is not canonical for new customer pricing. This pricing model has been replaced. This current page should win for new buyer questions. This fact is valid only under these conditions.
That context can't live only in a designer's layout or a human-readable footnote.
It needs to be part of the machine-readable architecture.
the real failure mode: context collapse
The AI didn't make up the legacy Starter price.
It saw the fact.
The failure was context collapse.
The system had the number, but not the rule.
That's how AI search breaks.
Not always through hallucination.
Sometimes through evidence-backed confusion.
That's worse, because the answer can look sourced while still being commercially wrong.
what this proves
The experiment proves five things:
1. Internal contradictions matter. AI search can break even before third-party sources enter the picture.
2. A source can be true and still harmful. Legacy pricing may be accurate for legacy customers but wrong for new buyers.
3. Retrieval needs rules, not just documents. The AI needs to know which fact applies in which context.
4. Schema must model relationships. Isolated markup doesn't solve current-versus-legacy ambiguity.
5. AI search needs regression testing. Every pricing update, product change, or page launch can change what AI believes.
what companies should do
Run a source-of-truth audit.
Start with pricing because pricing exposes contradictions fast.
Ask:
Which pricing page is canonical? Are legacy pages clearly marked? Are old PDFs indexed? Do marketplace listings match the current offer? Does schema define the current product tiers? Are retired products marked as retired? Can AI distinguish new-customer pricing from existing-customer pricing?
Then test.
Don't assume the model understands your website.
Ask it.
bottom line
AI search misrepresents businesses when data conflicts go unresolved.
One legacy page can distort the answer.
One contradiction can change the buyer's expectation.
One missing relationship can turn true facts into a wrong recommendation.
AI search is an architecture problem.