A tracker and your own ChatGPT session may observe different product surfaces, search settings, locations, context or collection times. Compare the saved question and complete answer before treating a mismatch as a tracking error or a change in brand visibility.
Community questions informed this guide. Linked provider documentation supports product-specific facts; the worked example is illustrative.
Compare the conditions before comparing the score
The same words do not establish the same test. A vendor may collect through an API, a consumer interface or another documented method. Your browser may have a different search mode, conversation history or location. Ask the vendor to name the experience actually measured rather than inferring it from a ChatGPT logo.
OpenAI documents search in ChatGPT and a separate API web-search tool, including approximate location controls. These references describe configurable behavior; they do not establish that one API response reproduces a particular consumer session. A private or signed-out window is a useful comparison condition, not proof that every contextual difference has been removed.
| Record for both observations | Why it matters |
|---|---|
| Exact prompt and preceding conversation | Follow-ups can change the task being answered. |
| Product surface and search evidence | A search-grounded answer and an ungrounded response are different observations. |
| Language, market and location settings | A local recommendation needs a defined location. |
| Timestamp and response evidence | You need the actual result, not just a dashboard score. |
Run a small reconciliation exercise
Choose a few business-relevant questions and collect repeated observations under each documented condition. Keep the two groups separate and preserve all valid answers, including non-mentions. Review source URLs and the brand-matching rule. Do not keep only the manual runs that agree with the dashboard.
If a tracker says the brand is present, inspect whether it matched the correct entity. A similarly named company, a product alias or a cited page mentioning several vendors may explain the apparent positive. If no response record is available, label the disagreement unresolved rather than inventing a cause.
Choose the correction that follows from the evidence
Correct a collection or entity-matching fault in the measurement workflow. Improve a source page when the answer exposes an actual factual gap. Keep a genuine interface difference as a reporting distinction. None of these findings alone proves that the brand lost visibility across all AI users.
For purchasing decisions, ask whether the vendor can label surfaces and export the evidence you need before expanding the subscription.
What to check
- Record both product surfaces and search conditions.
- Request the full response behind the disputed data point.
- Check entity matching before changing source content.
A practical example
Illustrative example: a tracker records 2 mentions in 5 API observations while a manual browser check returns one non-mention. That one browser result does not invalidate the five observations. Keep 2/5 for the labeled API panel and report the browser check separately until the methods can be reconciled.
Sources and further reading
- OpenAI: ChatGPT Search and citations
- OpenAI: web search and approximate user location
- Reddit discussion: which ChatGPT and Claude surfaces to track — anecdotal question signal
These steps are Arrow AI's implementation guidance. The example is illustrative; it is not a measured customer result. Publication and source access do not guarantee an AI recommendation.
Apply this to your website
Identify the source gaps and decide what to improve with Arrow's free audit. Review what Arrow GEO measures before choosing a reporting scope.
