Why Your AI Citations Keep Changing: Inside ChatGPT's Four Search Pipelines

If you have been tracking how often ChatGPT mentions your brand, you have probably had this experience: a prompt that cited you last Tuesday does not cite you this Tuesday, and nothing on your side changed. No page was edited. No competitor published anything obvious. The number simply moved.

Most teams explain that away. The model was updated. The index refreshed. Someone else got a backlink. Those explanations are comfortable because they imply the movement means something.

Two pieces of independent research published in the last two weeks suggest a less comfortable explanation: a meaningful share of that movement is not signal at all. It is the retrieval layer underneath ChatGPT shuffling, invisibly, between four different ways of fetching the web.

This matters more than a typical platform curiosity, because it attacks the foundation of how the entire AI-visibility discipline currently measures itself.

---

What the researchers actually found

Around June 29 and 30, the SEO researcher Suganthan Mohanadasan did something that sounds obvious in hindsight and that nobody had published before: instead of reading the citation cards ChatGPT shows users, he read the raw network traffic ChatGPT generates while it searches.

Inside that JSON he found a field named result_source. It carried one of four values: serp, labrador, bright, and oxylabs.

These are not user-facing. They appear nowhere in the interface. They are internal labels for what appear to be four distinct retrieval pipelines — four different mechanisms by which ChatGPT goes and gets web content before writing an answer that cites it.

A second, much larger dataset assembled by Chris Green reproduced the finding. Green ran roughly 1,000 prompts up to ten times each, for close to 10,000 completed search runs. The same four labels appeared. And because the same prompts were run repeatedly, the dataset could measure something the first sample could not: how much the answer changes when nothing about the question changes.

Sources: Suganthan Mohanadasan, How ChatGPT picks sources · Search Engine Land, ChatGPT citations change because of hidden search pipelines (July 8, 2026) · Search Engine Journal, How ChatGPT actually picks sources

---

The four pipelines, and what they appear to be

The researchers' reading of the four labels — and this is interpretation, not documentation — is that each corresponds to a different way of sourcing the web:

• labrador appears associated with licensed publisher content, the material OpenAI has struck commercial agreements to use.
• bright appears to correspond to Bright Data, a commercial web-scraping service.
• oxylabs appears to correspond to Oxylabs, a competing scraping provider.
• serp appears to be an open-web search baseline.

In the samples examined, the distribution was heavily skewed. One pipeline — labrador — accounted for the large majority of runs, with the other three splitting a thin remainder.

Be careful with that distribution figure. We will come back to why in the caveats section, because it is the weakest part of the finding and it is the part most likely to end up in a slide deck.

The structural fact is the durable one, and it does not depend on the percentages being right: which pipeline serves a given answer is not fixed, and different pipelines surface different sources.

That is the whole story in one sentence. A licensed-publisher corpus and an open-web scrape do not contain the same pages. If the system draws from one on Tuesday and another on Thursday, the candidate pool from which your brand might be cited is not the same pool on those two days.

---

The instability, quantified

Green's repeat-run dataset put numbers on it. Across identical prompts run multiple times:

• About 11.6% of prompts changed their primary retrieval source between runs.
• Cited-URL overlap fell from 0.273 to 0.149 run over run — a decline of roughly 45%.
• Domain overlap fell from 0.265 to 0.155 — a decline of roughly 42%.

Sit with the second and third numbers, because they are more alarming than the first and they get quoted less.

An overlap coefficient of 0.149 means that when you ask ChatGPT the same question twice, the set of URLs it cites the second time shares only about 15% of its members with the first. Roughly six in seven cited URLs are different.

Domain overlap at 0.155 tells you this is not merely a matter of citing a different page on the same site. The publishers change too.

This is not a model that has a settled opinion about your category and expresses it consistently. On these measurements, it is a system that assembles a fresh, substantially different evidence set each time it is asked, and writes an answer from whatever it happened to pull.

Source: Search Engine Land, ChatGPT citations change because of hidden search pipelines (July 8, 2026)

---

Why this breaks single-shot AI visibility measurement

Nearly every AI-visibility product on the market — including, historically, a good portion of how this discipline has taught itself to work — does some version of the following:

1. Define a set of prompts a buyer might ask.
2. Run each prompt once against each engine.
3. Count how often the brand appears.
4. Divide, render a percentage, put it on a dashboard.

If retrieval is stable, that method is sound. If retrieval is not stable, step 2 is the problem, and everything downstream inherits it.

Consider a prompt where your brand genuinely appears in half of all possible responses. Run it once. You get a coin flip. The dashboard renders "0%" or "100%" for that prompt with exactly the same confidence it renders everything else. Run the whole prompt set once a week and the week-over-week delta you are reading — the one that generates the meeting, the one someone attributes to last month's content push — is substantially resampling noise.

The uncomfortable implication is that a portion of the AI-visibility movement being reported, explained and acted upon across the industry right now is measurement error being narrated as insight.

That is not an argument against measuring. It is an argument against measuring once.

---

The arithmetic nobody runs

Here is a calculation worth doing before your next reporting cycle. The following is our own illustrative arithmetic, not a finding from the research above, and it makes a simplifying assumption we will flag immediately after.

Treat each run of a prompt as an independent trial with some true probability p that your brand is cited. The standard error on your estimate after n runs is √(p(1−p)/n). At p = 0.5, which is the hardest case and therefore the honest one to plan against, the approximate 95% confidence interval around your measured citation rate looks like this:

Runs per prompt	Approx. 95% confidence interval
1	±98 points (meaningless)
5	±44 points
10	±31 points
30	±18 points
100	±10 points
400	±5 points

Read the row for 10 runs. Even after running a prompt ten times, a measured citation rate of 50% is consistent with a true rate anywhere from roughly 19% to 81%.

Now consider what that means for the claim teams most want to make: "our AI visibility improved 10 points this quarter." To resolve a 10-point change on a single prompt with any confidence, you need runs in the high hundreds. Almost nobody is doing that, and the ones who are usually cannot afford to do it across a large prompt set.

The simplifying assumption: real runs are not fully independent. Caching, session effects, time-of-day routing and correlated pipeline selection all mean consecutive runs are more alike than coin flips. That correlation does not rescue the picture — in practice it usually makes the effective sample size smaller than the raw run count, not larger. The table above should be read as an optimistic floor on how much sampling you need, not a pessimistic ceiling.

---

SEO solved this problem once already

There is a useful precedent here, and it should be reassuring rather than alarming.

Early rank tracking had the same disease. A tool checked a keyword once a day from one location and reported "position 4." Then personalization arrived, then localization, then continuous testing, and the single number stopped describing anything real. Two people searching the same term in the same city got different results, and a daily one-shot check reported one of them as the truth.

The discipline did not respond by abandoning rank tracking. It responded by changing what it measured and how it reported:

• Checks moved from one location to many, and results were reported by location rather than averaged into a single misleading figure.
• Single-keyword positions were demoted in favor of aggregate share-of-visibility metrics across a keyword set.
• Volatility itself became a product — the industry now watches algorithm-flux indices precisely because movement is expected and the question is whether today's movement exceeds the background rate.
• Practitioners learned to ask whether a change exceeded normal fluctuation before attributing it to their own work.

Every one of those adaptations maps directly onto the problem in front of us. AI-visibility measurement in 2026 is roughly where rank tracking was before it learned that a single observation of a personalized system is an anecdote. The difference is that this time the discipline has the benefit of knowing how the story ends, and can skip the years spent arguing about it.

---

A worked example of how this goes wrong

Here is the failure mode, concretely, because it is easy to nod along with the statistics and still make the error.

A B2B software team tracks 40 buyer prompts across ChatGPT. In April they run each prompt once and their brand appears in 12 of them: a 30% visibility score. They spend the quarter doing real work — a comparison page, three integration guides, a review-platform push. In July they run the same 40 prompts once each and appear in 16: a 40% score.

Ten points of improvement. The deck writes itself.

Now apply the sampling arithmetic. With one run per prompt across 40 prompts, the aggregate has roughly a ±15 point confidence interval at these rates. The April and July intervals overlap substantially. The measured 10-point gain is entirely consistent with no underlying change at all — and equally consistent with a 20-point gain that got understated.

The work may well have moved the number. The problem is that this measurement cannot tell you, and the team is about to spend next quarter's budget as though it did. If the content push is genuinely working, the same flawed measurement will eventually show a decline somewhere and someone will reasonably propose cutting the thing that was working.

That is the real cost of noisy measurement: not that it is imprecise, but that it produces confident conclusions in both directions, and the organization cannot tell which kind it is looking at.

The fix costs a fraction of the content budget. Cut the prompt set from 40 to 20, run each 10 times instead of once, and the same spend buys an interval tight enough to distinguish a real 10-point move from nothing.

---

What this does not mean

The finding is important. It is also being over-read already, and a few corrections are worth making before they harden into folklore.

The distribution percentages are weak. The figures on how often each pipeline fires come from a small sample — effectively one Pro account, roughly 1,240 records, skewed toward SaaS-category queries. Those specific percentages should not be quoted as ChatGPT's true global mix. The existence of the four labels is well-supported by two independent researchers reading actual network data. The frequencies are not.

The labels are not proven to be the cause. result_source is a fetch and vendor tag. It correlates with which sources appear. Describing it as the thing that "determines" your citation overstates a mechanism no outsider can fully observe. There are plenty of other moving parts — query rewriting, ranking within the retrieved set, the generation step itself — that the network traffic does not expose.

OpenAI has confirmed none of this. Not the pipelines, not their purpose, not how one is selected for a given query. This is careful outside observation of a closed system, which is the best anyone can do here, and it is not the same as disclosed architecture.

This is not a reason to stop doing the work. Retrieval instability does not mean content quality is irrelevant. It means content quality sets the probability that you get cited, and any single observation is a draw from that probability rather than a readout of it. Raising the underlying probability is still the entire job.

And it is not a ChatGPT-specific indictment. There is no equivalent public dataset for Gemini, Perplexity, Claude or AI Overviews. It would be a mistake to conclude those engines are more stable. The honest position is that nobody has looked in the same way yet.

---

Five changes to how you measure

If you accept the finding, the practical consequences are not subtle. These are the changes worth making to a measurement program.

1. Sample repeatedly, and decide your cadence deliberately

Replace "run each prompt once per cycle" with an explicit sampling design. Even five to ten runs per prompt is a dramatic improvement over one, and it is usually affordable if you trim the prompt set to the questions that actually map to buying decisions. A smaller prompt set sampled properly beats a large prompt set sampled once, every time.

2. Report a range, not a decimal point

A dashboard that says "Visibility: 43.2%" is claiming a precision the underlying data cannot support. "Visibility: 38–48%, 30 runs per prompt" is honest, and it has a useful side effect: it stops teams from reacting to movement that sits inside the interval. If this quarter's band overlaps last quarter's band, you do not have a result yet.

3. Separate prompt-level noise from brand-level signal

Individual prompts are noisy. Pooled across a well-constructed prompt set, the noise partially cancels and the aggregate becomes far more stable than any of its parts. Report the aggregate as your headline and treat individual prompt results as diagnostic leads, not as metrics. The common mistake is the reverse: leading with a dramatic single-prompt win.

4. Track stability as a metric in its own right

If the source set behind your category shuffles by 45% between runs, that volatility is itself information. A category where citations are unstable is a category where the engines have not settled on authoritative sources — which is an opportunity. A category with stable citation sets is one where incumbents are entrenched and displacement is harder. Most teams do not measure this at all, and it is arguably more strategically useful than the headline score.

5. Re-measure changes with the same design that produced the baseline

This is the one that quietly invalidates the most work. If your baseline was one run per prompt and your post-publication measurement is ten runs per prompt, the two numbers are not comparable and any "lift" you compute from them is partly an artifact of the method change. Freeze the design before you start, or accept that you have no before-and-after.

---

What this changes about content strategy

Less than you might think, and that is the good news.

If citation is partly a draw from a probability distribution, the work is to raise the probability. That is the same work it was before: be genuinely present in the sources these pipelines draw on, be described accurately and consistently across them, and be structured so that a machine can extract a clean answer without inferring it.

What the four-pipeline finding adds is a specific argument for breadth of presence over depth in any single property.

If one pipeline draws on licensed publisher content and another scrapes the open web, then a brand that exists thoroughly on its own website and nowhere else is well-served by one pipeline and poorly served by the others. A brand that is present across licensed publications, the open web, and the structured sources both draw on has a better chance of surviving whichever pipeline happens to fire.

That is an argument for off-site presence — industry publications, review platforms, reference sources, the places your category gets discussed — that does not depend on any particular theory of how the ranking works. It follows directly from the observation that there is more than one corpus.

---

What to watch next

Whether OpenAI comments. Confirmation or denial changes how much weight this deserves. Silence is the most likely outcome and is itself informative about how much of this layer is intended to be observable.

Whether the labels reproduce elsewhere. The frequencies come from one account and a narrow query mix. Replication on non-Pro accounts and across query categories would move them from directional to reliable.

Whether anyone runs this experiment on other engines. The absence of comparable data for Gemini, Perplexity, Claude and AI Overviews is a gap, not a clean bill of health.

Whether AI-visibility tooling adapts. If retrieval instability is real at this magnitude, single-shot measurement is measuring noise, and products that keep reporting one-run scores to two decimal places are selling false precision. The ones that move to repeated sampling, confidence intervals and stability metrics will be measuring something. This is the change worth demanding from any vendor you are evaluating, including us.

---

The takeaway

Four hidden retrieval pipelines sit between a buyer's question and the answer that does or does not mention you. Which one fires is not something you can see, set, or influence. About one prompt in nine changes its primary source between identical runs, and the set of cited URLs overlaps by roughly 15%.

The correct response is not despair, and it is not to keep reporting a single confident number while knowing better. It is to measure the way you would measure anything else noisy: sample repeatedly, report ranges, pool across prompts, track the volatility itself, and hold the design constant when you want to prove something changed.

The discipline that comes out the other side of this will be less dramatic on a dashboard and considerably more defensible in a room where someone asks how you know.

Sources: Suganthan Mohanadasan, How ChatGPT picks sources · Search Engine Land, ChatGPT citations change because of hidden search pipelines (July 8, 2026) · Search Engine Journal, How ChatGPT actually picks sources