We had an AI run an SEO audit on our own website. The report looked professional: prioritised, specific, with an implementation timeline. It appeared entirely actionable.
Then we verified it item by item. The result is worth writing down, because it demonstrates both the real value and the real risk of AI in this kind of work.
Its top recommendation was fixing four pages that never existed
The report opened with “the most urgent item”: old URLs without redirects. It listed four specific addresses, said Google could still find them but they now returned 404, and that authority was being lost. A detailed mapping table followed, specifying which new page each should redirect to.
If true, that would deserve first place. So we checked.
Those four addresses had never existed on this website. We queried the content system across every state — published, draft, in the bin — and found the site has only ever held around thirty items. We compared the slugs one by one. Not one matched.
The cause is straightforward: AI cannot actually query Google’s index. It has neither the access nor the interface. But asked for an SEO audit, it produces something shaped like an SEO audit, and “old URLs need 301 redirects” is an extremely common item in that genre. So it included it, complete with specific addresses — because specific addresses look more professional than a general observation.
Following that advice would have spent a week configuring redirects pointing at pages that do not exist.
But it caught a real problem we had missed
Second item in the same report: search engines read “0 websites delivered” on our home page.
That one was correct, and it mattered.
The home page has a counter animating from 0 to 500+. The initial value written in the HTML was 0, and only when the element scrolled into view did a script change it to 500+.
People saw 500+. Crawlers, AI summarisers, screen readers and anything that does not execute scripts read “0 websites delivered” — the single number on the page intended to establish scale, stating the opposite.
This had been live for a long time. We had looked at that page countless times without noticing, because we always saw the result after the animation.
It caught several other genuine issues too: over-definite timelines on the SEO/GEO page that contradicted the disclaimer about not promising rankings a few lines above, and missing local credibility signals such as the registered entity and its registration number. We changed all of them.
Why both outcomes appeared in one report
The difference is not difficulty. It is whether the information is on the page.
“The HTML says 0” is a fact readable directly from the source. The AI had the page, saw the code, and judged correctly. It does this class of work quickly and thoroughly, without fatigue, and more reliably than a person reading line by line.
“Which old pages are still in Google’s index” is not on the page and requires querying an external system. The AI could not, and having been asked for a complete audit, it filled the gap with the conventional shape of such reports.
This is not lying. It is completing a section it cannot verify, and completing it convincingly. The more convincing, the more dangerous — that mapping table was well formatted and internally coherent, and nothing revealed the problem short of actually requesting those four addresses.
What to act on and what to verify
One dividing line emerges:
Conclusions readable from the page itself are usually reliable. Heading structure, missing tags, script-rendered content, whether structured data is correct, whether pages link to each other, whether copy is vague — it sees all of this and assesses it faster and more completely than a person.
Anything requiring an external system must be verified. Indexing status, ranking positions, competitor data, search volumes, backlinks, historical URLs — no reliable source exists. Treat every specific figure or list of this kind as unconfirmed.
The line is usable without knowing anything about SEO. Just ask: could this conclusion be reached by looking at my web page?
If yes, it is broadly trustworthy. If no, check it yourself — and checking is usually easy. Those four URLs took one browser request each.
Verification costs less than expected
Worth emphasising, because “if I have to check everything I may as well do it myself” is a natural reaction.
It is not accurate. The report contained twenty or thirty recommendations and took a minute to generate. Verifying all of it took about twenty minutes — requesting those addresses, querying the content system, viewing page source and searching for terms, reading a few pages.
Twenty minutes for a filtered list is far faster than auditing from scratch. The problem is not that verification is needed; it is that people skip it.
And the more professional a report looks, the more readily it gets skipped. That is probably the most practical risk in AI-generated work — not that it errs, but that its errors look so credible.
If you want to try it
Having AI audit your own site is worth doing, in this way:
Give it the actual page content. Pasting the source is far more reliable than supplying a URL, since an assistant given a URL may not have fetched anything.
Ask it to separate its own claims. Add a line to your prompt: list separately which conclusions come from the content I supplied and which are general inference. This markedly reduces the second kind blending into the first.
Verify every specific number and list. Especially the parts that look most professional.
Treat it as a first pass, not a conclusion. It is good at covering many checks quickly and pointing at what deserves attention. Which items actually matter, and in what order, remains a human judgement.
We use AI for the first pass of our own SEO and GEO audits and verify every conclusion against the site. Those four nonexistent URLs are what that step is for.