I Logged Every URL Five LLMs Cited for 200 Buyer Prompts. The Pattern Was Infrastructure, Not Content.

Everyone wants the content trick that gets them cited by ChatGPT. After logging citations across 200 buyer prompts and five models, I'm fairly convinced the deciding factor is duller than that: whether a crawler can fetch your page and whether a chunk of it survives extraction. Here is what I actually did, what I found, and the checklist I now run before writing a single word of new content. The setup I took 200 prompts that real buyers type — comparison questions, "best X for Y" questions, troubleshooting questions, and pricing questions — across three categories I know well: developer tools, B2B SaaS, and e-commerce operations. Each prompt was run three times, on different days, against five systems: ChatGPT, Gemini, Claude, Perplexity, and DeepSeek. Whenever the answer included a citation or a linked source, I logged the URL. When there was no citation, I logged the brand names mentioned and searched for the page that most likely seeded them. That produced roughly 1,400 logged citation events. Not a peer-reviewed study. But large enough that the same patterns kept repeating. Finding 1: citations cluster into a small number of page types Sorting the cited URLs by structure rather than topic, they collapsed into six buckets: Comparison and alternative pages ("X vs Y", "alternatives to X") Documentation and reference pages with stable URLs Listicles and roundups that name many products Forum and community threads with first-hand answers Pricing and feature pages with explicit, structured facts Third-party directories and review aggregators In my sample, those six buckets accounted for the overwhelming majority of cited URLs. Original thought-leadership essays were almost never cited, even when they ranked well in Google. That stung, because essays are what everyone wants to write. The practical translation: if you want to be quoted, publish pages whose structure matches what the model is trying to answer. The model is assembling a comparison; give it a comparison page. It is verifying a price; give it a page where the price is a fact, not a paragraph of positioning. Finding 2: crawlability is a hard gate, and most teams fail it silently A model cannot cite a page it never fetched. This sounds obvious, and yet the most common failure I found was not "bad content" but "content that was never retrieved." Three failure modes, in order of how often I hit them: The robots.txt block. A surprising number of sites still disallow GPTBot, ClaudeBot, PerplexityBot, or Google-Extended — often inherited from a template, or added during a "block AI scrapers" panic in 2023 and forgotten. Blocking AI crawlers is a legitimate business decision. Doing it by accident is not. The CDN or WAF block. This one is invisible in your analytics. Cloudflare, Akamai, and bot-management rules frequently block unfamiliar user agents at the edge, before the request ever reaches your origin. Your server logs show nothing because nothing arrived. Your 404 logs show nothing because it was never a valid request. Everything looks fine until you notice you are absent from every AI answer in your category. The JavaScript-only render. If your content requires client-side rendering and executes slowly, some retrieval pipelines will time out or capture an empty shell. The diagnostic that actually works: stop reading dashboards, read raw logs. Filter for the specific user agents over a week, excluding your own IPs. If GPTBot appears zero times across 10,000 requests, you have your answer, and it is not a content problem. Finding 3: extraction happens in chunks, not pages This is the part that changed how I write. Retrieval systems split pages into passages and match those passages against the query. A page can rank well, be crawled successfully, and still never be cited — because no single chunk contains a clean, self-contained answer. What survives chunking looks like this: One question per page, or one question per H2 section. Two topics on one page means both lose clarity at extraction time. The answer in the first 40 words of the section. If your H2 is "How much does it cost?" and the next 200 words are context before the number appears, the chunk boundary will often land before the number. Tables over prose for anything with more than two variables. A pricing table compresses into a citable fact; a pricing paragraph compresses into mush. Full entity names instead of pronouns. A chunk that says "it integrates with the other tool" is worthless once separated from its page. A chunk that says "Acme Analytics integrates with Snowflake" survives. I rewrote one product page along those four rules and nothing else — same URL, same design, same keyword targeting. Within a month it started appearing as a cited source on prompts where it had previously been invisible. That is a sample size of one, so treat it as a signal, not a law. But the mechanism is at least coherent: I made the page easier to lift. Finding 4: entity consistency is the cheapest fix nobody does Ask five models what your company does, and you will often get five different descriptions. Mine differed on category, founding year, and even the primary use case. That happens because models assemble an entity from whatever sources they retrieve: your site, your LinkedIn, a directory listing from 2021, a podcast description written by a host who half-remembered your pitch. Inconsistent sources produce an inconsistent entity, and an inconsistent entity is a weak one — harder to match, harder to trust, less likely to be surfaced. The fix costs almost nothing: write down the canonical facts about your company in one place — one-sentence description, category, founding year, headquarters, pricing model, primary competitor set — and then push those exact strings everywhere you control. Your homepage, your about page, your schema.org Organization markup, your LinkedIn, your Crunchbase, your G2 profile, your directory listings. Then verify by asking each model the same question once a week and diffing the answers. When the descriptions converge, you know the entity is stabilizing. Finding 5: recency matters less than people think, but staleness is fatal I expected a strong recency bias. What I found was closer to a staleness penalty: recent pages were not automatically favored, but pages with visibly outdated facts (a 2022 price, a discontinued feature, a dead competitor still listed) were consistently skipped. The rule I use: any page that makes a factual claim about price, availability, or product capability needs a visible last-updated date and an actual update every quarter. Not a rewrite — an update. What I would do in week one If I were starting from zero tomorrow, in priority order: Run your ten buyer prompts across all five models. Record whether you are mentioned and which URLs are cited instead. Pull a week of raw server logs and grep for GPTBot, ClaudeBot, PerplexityBot, and Google-Extended. Zero hits means a block, not a content problem. Check robots.txt and your CDN rules for the same four user agents. Write the canonical entity paragraph and push it to every profile you control. Pick your three most important questions and rewrite those pages so each section answers one question in its first sentence. Re-run the ten prompts in 30 days and compare. Ignore week-to-week noise; a 10-15% swing is not a trend. The uncomfortable conclusion The work that moves AI citations is unglamorous: unblocking crawlers, structuring facts, making pages quotable, and keeping the story about your company identical everywhere. None of it produces a satisfying before-and-after screenshot. All of it compounds, because once a page becomes a reliable source for one query, it tends to get reused for adjacent ones. Content is still the input. But in my logs, the pages that won were the ones that were reachable and liftable — not the ones that were best written. I build a tool called Bidvix that runs the prompt-and-citation audit described above across ChatGPT, Gemini, Claude, Perplexity, and DeepSeek. The logging method in this post is the same one it automates. If you'd rather do it by hand, the six steps above are enough to start this week.