Your colleague's ChatGPT answer doesn't match your AI visibility report. Which one is right?

Why does your colleague see a different answer from your report?

Because they asked a different engine, in effect. Your sales director types a question into ChatGPT on a paid plan, signed in, with months of chat history. Your AI visibility report ran the same question many times on clean sessions. The two outputs share a prompt and very little else. Since 7 October 2026, paid ChatGPT plans have run GPT-6 Sol, while Free and Go users moved to GPT-6 Luna from 8 October.¹ Two people on two plans can be asking two models.

The disagreement usually arrives as a screenshot in a team chat: "We're not in ChatGPT at all." Or the opposite: "It recommended us first, so why does the report say we're losing?" Neither reaction is foolish. The screenshot is real, and so is the report. They measure different things, and this post sets out how to tell which one should drive a decision.

Prior work and a disclosure

Sigzen AI sells AI visibility audits and programmes at published prices, so we have an interest in you trusting measured rates over screenshots.¹² The protocol below works with any tool or team, and most of it fits in a spreadsheet.

Others have covered parts of this. Ahrefs, which sells AI visibility tools, set out why prompt-level rank tracking breaks down when answers change between runs and between users.¹⁰ Seer Interactive wrote about how to brief executives when a visibility number is easy to win and hard to tie to revenue.¹¹ What this post adds is the step in between: what to do with the screenshot itself, and how to fold it into the report without letting it replace the report.

Seven conditions that change one answer

An AI answer depends on at least seven conditions, and a screenshot records none of them unless someone writes them down. Each one can change which brands appear, in which order, and with which sources. Two colleagues who differ on any one of them are not running the same test.

Engine, plan and model

The engine comes first. ChatGPT, Gemini, Microsoft Copilot, Google AI Overviews and Google AI Mode each choose sources on their own rules, so a brand can lead on one and be missing on another. Within one engine, the plan sets the model. OpenAI's release notes put GPT-6 Sol on Plus, Pro, Business and Enterprise and GPT-6 Luna on Free and Go.¹ A board member on an Enterprise seat and a buyer on the free tier read answers written by different models.

Logged-in state and memory

A signed-in account brings chat history and saved memories, and the engine may use both when it writes an answer. Someone who has spent a week asking ChatGPT about your company has taught it that the company matters to them. Their answer is the most contaminated sample in the building. Published studies tend to measure the logged-out case. Ahrefs' study of 66.6 million responses covered free, logged-out ChatGPT on GPT-5.6 Luna in August and September 2026, across all countries.²

Search, location, language, wording and chance

Whether the engine searches the web changes the answer most of all. In that Ahrefs sample, 14.4% of responses triggered a web search; the rest came from what the model already held.² A signed-in user can also switch search on by hand. Semrush, which also sells AI visibility tools, reported from its US clickstream panel that ChatGPT had web search enabled on 34.5% of queries as of February 2026, down from 46% in late 2024.³

Location and language pick which sources a search returns. Wording matters as well: "best freight forwarder for Canada" and "which freight forwarders handle Canadian customs well" are different prompts. Then there is chance. Ask the same engine the same question twice and the list can change between runs, which is why a single run is an anecdote.

Which differences are noise, and which are real?

Chance is noise; the other six conditions are real differences that a report either controls or splits out. That distinction decides what you do next. If the screenshot differs from the report only by chance, one more answer adds little. If it differs because the colleague used a paid model with search switched on, it may be showing a real segment of buyers your report does not sample.

Repeat runs tell the two apart, up to a point. MaxAEO, which sells AI visibility measurement, published guidance in July 2026 treating repeat runs of one prompt as clustered: they resemble each other, so each extra repeat adds less than a new prompt would.⁴ Dmitrij Żatuchin's July 2026 preprint breaks answer variation into parts, including the model and the language of the query.⁵ The practical reading is that variation has structure. Some of it belongs to the run, and some belongs to who is asking.

Our view: treat a screenshot as a hypothesis about a segment, never as a correction to the rate. "Paid-plan users with memory see us less often" is worth testing. "The report is wrong because I saw something else" is not, until the screenshot has been re-run under conditions anyone can repeat.

Why the screenshot still matters

It matters because buyers act on single answers. In G2's survey of 1,076 B2B decision makers in North America, EMEA and APAC (March 2026), 51% said they start software research in an AI chatbot more often than in Google.⁷ Nobody in that buying group sees your share of answer. Each of them sees one answer, on their own account, on the day they ask.

Buyers also know those answers can be wrong. In the same G2 survey, 64% said they encounter inaccurate AI recommendations often or very often.⁸ In TrustRadius's survey of 1,862 technology buyers in January 2026, 63% used AI somewhere in the purchase journey, and 94% of those fact-checked what it told them.⁹ A single answer starts the shortlist; checking decides it.

The group doing the asking is large. Forrester's 2025 fieldwork, exact dates not stated, found buying decisions involve on average 13 internal stakeholders and 9 external participants.⁶ Each of them can type a different question on a different plan. The rate is how you see the whole group; the screenshot is how you see one member of it.

How do you log a colleague's screenshot so it becomes evidence?

Capture the seven conditions at the moment it is shared, then re-run it. A screenshot without conditions cannot be compared with anything, and the conditions are gone within a day: the colleague forgets their exact wording, the model updates, the chat history grows. We use one row per screenshot in a shared log.

Illustrative. Field list used by the Sigzen AI editorial team; adapt the plan names to the engines your buyers use.

Ask for the full answer, not the cropped part with your name in it. The rivals named alongside you, and the sources linked, say more than your own position. A shared screenshot of a rival leading is useful too: it tells you which sources that rival is being credited from.

Then re-run it under clean conditions

Re-run the exact prompt on a fresh, signed-out session, on the same engine, from the same country, on three separate days. Then run it once more on the colleague's plan with memory switched off, if they can. Those runs split the difference into its parts: chance, account effect, or a real gap in conditions the baseline never samples. Our one-hour manual check covers how to set up a clean session, and our guide to re-run cadence covers why three days is the floor.

Reconciling one screenshot with a measured rate

A rate already predicts that some single answers will leave you out. If your report says you are named in about half of answers to a prompt, then about half of your colleagues' screenshots for that prompt should miss you, even when nothing is wrong. The screenshot that alarmed the sales director is often the report working as described.

The case that needs action is the other one. If the clean re-runs keep showing you while every paid-plan run with search switched on leaves you out, the screenshot has found a segment. That pattern points at the sources the searching runs read, which is a page or listing problem you can work on. Our note on memory and retrieval prompts explains how to split those two kinds of answer in the report.

Our view: never average a screenshot into the rate. One extra answer is too small to move a measured share, and folding it in hides the conditions that made it different. Keep it as a logged observation with a status: chance, account effect, or segment.

Should your report measure signed-in answers too?

Only as a separate line, and only once the clean baseline is stable. A baseline needs conditions anyone can repeat next month, which rules out real accounts with real histories. Signed-in measurement answers a different question: what does a buyer on a given plan, with a given kind of history, tend to see? That question is worth asking once screenshots keep pointing at the same segment.

Persona runs, not personal accounts

The repeatable way to measure signed-in answers is a persona account: a test account on a named plan, with memory either off or seeded with a fixed, written set of facts, used for nothing else. A procurement persona and a technical persona can be run side by side. Their scores go on their own lines, labelled with plan, model and memory state, and never mixed into the baseline.

Prompt shape changes with the user, too. Semrush compared January–February 2025 with January–February 2026 in its US clickstream data: search-enabled ChatGPT prompts grew from an average of 4.7 to 8.7 words, while non-search prompts fell from 24.9 to 13.5 words.³ A persona prompt set should be written the way that persona types, which is the work our measurement spec asks any vendor to document.

Our view: one persona line per buying role that keeps showing up in the log is enough. More than that turns the report into a set of anecdotes with error bars, and the board will read the most dramatic line.

What should you tell the board when screenshots disagree?

Tell them the rate, the range and the log. The rate is the share of clean answers that name you. The range says how far it could move by chance. The log lists the screenshots people raised that month, the conditions each one carried and what the re-runs found. A board that sees all three stops asking "which answer is true?" and starts asking which buyers see which answer.

Keep the language plain. "We are named in about four of ten clean answers to our core prompts, give or take a few points" is a sentence a board can repeat. "The chair saw us missing on 14 October; clean re-runs found the chair's answer inside the normal range" closes the loop on a screenshot without dismissing it.

Where the measured numbers come from

The audit and programme reports from Sigzen AI run 150 buyer prompts across ChatGPT, Gemini, Microsoft Copilot, Google AI Overviews and Google AI Mode, each prompt on three separate days, on clean sessions through our AI answer monitor.¹² Programme dashboards update weekly. A screenshot log sits next to that dashboard, and the AI Visibility Audit reads out both in ten working days.

A protocol for the next disagreement

Agree the protocol before the next screenshot lands, so the response is routine rather than a debate. The steps below take a day of elapsed time and under an hour of work, and they turn every alarmed message into a logged observation with a verdict.

Write down who owns the log. In most teams it belongs with whoever owns the visibility report, because they hold the clean baseline and can run the re-checks. Sales and leadership supply the screenshots; marketing supplies the verdict. That split keeps the people closest to buyers involved without letting each new answer reset the plan.

Our view: the screenshot war ends when everyone agrees what each artefact is for. The rate tracks progress, the log explains what people see, and a persona line covers a buyer segment the baseline cannot reach. Each answers a different question, and none of them should be asked to answer all three.