writing/ai search · · by Alex Bouchard
Three checks the AI SEO playbook skips
This week the writer behind Algo Insights had Claude read 66 SEO and GEO articles, 48 agent skills and 14 GitHub repos, and published what survived as one playbook. It is mostly right. It gets the bot question right, which most guides don't. It skips three checks that decide whether the rest of it works.
The playbook is “The only SEO/GEO article you need” by Algo Insights. Vendor quotes below were read on the vendors' own pages on 11 November 2026. Sources are at the end.
01 / what it gets right
Start with the playbook. Most of it holds up.
Its own summary is fair: most of the advice in circulation is the same dozen ideas, few of the big numbers come with data, and popular repos are not proof the advice is current. These four points are the ones I would keep.
The bot split
OpenAI's crawler page says OAI-SearchBot “is used to surface websites in search results in ChatGPT’s search features” and that disallowing GPTBot only signals that content “should not be used in training.” Most guides still blur the two. This one does not.
Read the live robots.txt
The file your CDN serves can differ from the one in your repo. Check what the crawler gets, not what you deployed.
Pages for decisions
Comparisons, alternatives and pricing pages are what people read right before choosing, and they are the pages answer engines quote.
Ignore the score out of 100
None of the scoring weights in the repos it reviewed were checked against real rankings or citations. Read the findings, not the number.
02 / three checks it skips
Can the real bot reach you, can the fetcher, and can you see it change?
A spoofed user-agent is a lead, not proof
The playbook's test is a curl request that claims to be OAI-SearchBot. It comes from your laptop's IP. OpenAI publishes the IP ranges its crawlers use (openai.com/searchbot.json), and bot managers can check them, so a firewall can treat the real crawler differently from your request in either direction. In my own 2026-08-07 probe of 18 major sites, reddit.com's robots.txt disallowed every AI crawler I tested, yet its firewall returned 200 to my request calling itself OAI-SearchBot and 403 to the same request calling itself GPTBot. robots.txt and the firewall disagreed, and neither told me what the real bot gets. Confirm in your server logs or your firewall's security events.
The fetchers that come when someone asks about you
Crawlers index ahead of time. Fetchers visit a page live, when a user's question needs it. OpenAI documents ChatGPT-User and says that, because these requests are started by a user, “robots.txt rules may not apply.” Anthropic documents Claude-User and says disabling it “prevents our system from retrieving your content in response to a user query.” If your firewall blocks these, you can be indexed and still fail at the moment someone asks about you by name.
Measurement that can see change
The playbook says answers move from week to week, then suggests running each question twice per engine. Two samples can't tell a real change from normal churn. Fix the question set, run it on a schedule, and compare like with like. And note what the method can't see: the API models with search on are not the consumer apps, and Google's AI Overviews and AI Mode are not covered by any of the four engines it lists.
03 / measuring it every week
What broke when I automated section 5
Since mid-October a fixed set of buyer questions has run weekly against four API models with web search on: GPT-5.5, Claude, Gemini and Perplexity's Sonar. Each miss gets one cause: not fetchable, not indexed, no page answers the question, or a competitor was chosen. Doing it by hand, as the playbook suggests, hides these four problems. Automating it exposed them.
Gemini's citations hide the domain
Through the API, Gemini returned each cited source as a Google redirect link on vertexaisearch.cloud.google.com, not the site's own URL. Compare hosts without resolving those links and every Gemini row reads as “not cited.” I found this on 2026-10-13, when resolving the links turned three recorded misses into citations.
A check that couldn't run is not a “no”
A missing key, an HTTP error or an empty response has to be recorded as unknown. My first index check against Bing's Webmaster API read a status field that is 0 even for pages Bing holds, and it was about to report seven pages as not indexed. A positive control, one page known to be indexed, caught it.
Not every engine accepts the same request
The Perplexity model I call through OpenRouter rejected the web-search tool parameter outright. A harness that sends one request shape to every engine will quietly record that engine as citing nobody.
It costs less than the trackers, not nothing
Asking one question of GPT-5.5 with search cost me about $0.14, Claude about $0.06 and Gemini about $0.05. My weekly run across four engines comes to roughly $7.
What this does not show: whether fixing a cause wins a citation. A fix raises the odds; answers change weekly, and I have no published before-and-after yet. When I do, it goes here with the data.
- The playbook — Algo Insights, “The only SEO/GEO article you need” (Medium, 11 Nov 2026) ↗
- OpenAI — Overview of OpenAI crawlers (read 2026-11-11) ↗
- Anthropic — Does Anthropic crawl data from the web? (read 2026-11-11) ↗
- major-sites-2026-08.md — the 18-site probe behind the reddit.com example ↗
- geo-crawl-audit — the probe (MIT) ↗
To the playbook's author: you said you would post SpaceCut's numbers, good or bad. If you want a baseline before you start, send me ten real buyer questions and I'll run them across the four engines, free.
Next step
Want to talk it through?
Book a 30-minute call. I will tell you plainly whether I can help and what I would test first.