How We Test AI Visibility
Every audit asks five AI systems the questions a real customer would ask, records every answer word for word, and scores the results with a published formula. Here is exactly what we test, what we record, how we score it, and what an audit cannot tell you.
The audit pipeline, step by step
From query generation to final score — nothing is black-boxed.
Step 1
We generate buyer-intent queries
Based on the business category and location, we build the exact questions a real prospect would type — "best landscaper in Virginia Beach," "top-rated dentist near me," "who should I hire for roofing in Portland." These are discovery queries: broad, buying-intent questions that test whether AI recommends the business unprompted.
Step 2
We send those queries to five AI systems
Every discovery query is sent to ChatGPT, Claude, Gemini, and Perplexity through API access (not the consumer chat apps), using the same instructions for each. Google AI Overviews are captured from live Google results for the same query. We also search Reddit for community mentions of the business as a supporting signal.
Step 3
We capture every response verbatim
Nothing is paraphrased or summarized. The full text of every AI response is recorded and stored. You can read exactly what ChatGPT said, what Claude recommended, which businesses Gemini listed, and whether Google showed an AI Overview at all.
Step 4
We analyze mentions, position, and competitors
Our detection engine checks whether the business was mentioned by name, where it appeared in the response (top 3, top 5, or just mentioned), and which competitors showed up instead. We also run entity queries — asking each AI directly about the business — to measure how well the AI knows the brand.
Step 5
We crawl the website for AI-readiness signals
Beyond AI queries, we check the business website for structured data, FAQ schema, AI crawler permissions (robots.txt), content depth, author attribution, citable facts, and technical SEO foundations. These signals influence how AI systems extract and present information.
Step 6
We score and report
All data feeds into a weighted scoring model across four Tier 1 modules (AI Discovery, Entity Recognition, Competitive Position, Content Readiness) and three Tier 2 modules (Local Relevance, Trust & Authority, Technical Foundations). The result is a single AI Presence Score from 0 to 100, delivered in a white-label PDF report.
Why results may differ from manual searches
This applies to every AI audit tool on the market, not just RediForAI. Understanding these factors is essential to interpreting any AI presence data.
API vs. consumer chat are different experiences
When you open ChatGPT.com in your browser, you get web browsing, personalization, location awareness, and your conversation history influencing results. Our audit uses API access — no personalization, no conversation history, and (for ChatGPT, Claude, and Gemini) no live browsing. The API tests the AI's core knowledge, not the enhanced consumer experience. This is true for all AI audit tools, not just RediForAI.
AI responses are non-deterministic
Ask ChatGPT the same question twice, an hour apart, and you may get different businesses listed. AI systems use controlled randomness (temperature) to generate varied, natural responses. Every audit is a snapshot in time — a data point, not a permanent verdict. Manual searches are also snapshots; they just happen to be different snapshots.
Google AI Overviews don't appear for every query
Google selectively shows AI Overview blocks. Commercial and local queries like "best plumber in Vancouver WA" frequently return no AI Overview at all — Google shows regular search results instead. When no overview exists, we accurately report that. This is not a bug; it's how Google works.
Web-grounded vs. training-data providers
Perplexity and Google AI Overviews pull from live web results — they reflect current search rankings. ChatGPT, Claude, and Gemini respond from their training data, which has a knowledge cutoff. A business that ranks well in Google may not yet exist in ChatGPT's training data. Our audit tests both angles to give a complete picture.
The AI systems and sources we test
Each provider has different strengths, data sources, and behaviors.
| Provider | Data Source | Web-Grounded? | Best At |
|---|---|---|---|
| ChatGPT | Training data | No | Broad knowledge, entity recognition |
| Claude | Training data | No | Detailed analysis, nuanced recommendations |
| Gemini | Training data | No | Google ecosystem knowledge |
| Perplexity | Live web search | Yes | Real-time web presence, current rankings |
| Google AI Overviews | Google Search index | Yes | Search presence, AI-enhanced results |
The questions we ask
For a local business, each AI system gets the same six discovery questions, filled in with the client's category and city. The chat models also get three direct questions about the business. The templates below are the ones the audit uses today.
Discovery questions (6 per system)
best {category} in {city}top-rated {category} near {city}who should I hire for {category} services in {city}recommend a trusted local {category} in {city}{category} in {city} with the best reviewsaffordable {category} in {city}
Example: “best Dentist in Vancouver, WA”. These test whether AI recommends the business without being told its name.
Entity questions (3 per chat model)
Tell me about {business} in {city}. What do they do and what are they known for?What services does {business} offer?Is {business} in {city} a reputable business? What do reviews say?
These test whether AI knows the business and describes it correctly when asked by name.
How the score is weighted
Each module is scored 0–100, then combined into one AI Presence Score. 70% of the weight is on how AI actually answers; 30% is on supporting site signals.
| Module | Weight | What it measures |
|---|---|---|
| AI DiscoveryTier 1 | 25% | Was the business named or recommended in answers to buyer questions? |
| Entity RecognitionTier 1 | 15% | When asked directly, does AI know the business and describe it correctly? |
| Competitive PositionTier 1 | 15% | Share of voice versus the competitors AI named instead. |
| Content ReadinessTier 1 | 15% | Schema, FAQ structure, citable facts, llms.txt, AI crawler access. |
| Trust & AuthorityTier 2 | 10% | Reviews, credentials, author attribution, third-party signals. |
| Technical FoundationsTier 2 | 10% | Crawlability, headings, titles, mobile, sitemap. |
| Local RelevanceTier 2 | 10% | Location signals, service area, local business data. |
Score bands: 0–25 Invisible · 26–50 Barely Detectable · 51–70 Partially Visible · 71–85 Well-Positioned · 86–100 AI-Preferred.
What we record
- The exact question sent to each AI system
- The full, unedited answer (or "no AI Overview returned" when Google shows none)
- Whether the business was named, and its position when AI gave a ranked list
- Every competitor named in each answer
- The model used and the date and time of the run
- Website checks: robots.txt, schema, headings, content depth, citable facts
How often it updates
Each audit is a fresh run. The questions are sent at the moment you start the audit, which takes about 2 minutes.
Short-term reuse. If the same question is asked for the same client within 7 days, the stored answer may be reused so a quick re-run doesn't produce random noise.
Model changes. When an AI provider releases or retires a model, we review which version the audit uses. The model name is recorded with every answer.
Re-measuring. We recommend re-running a client monthly or quarterly to compare against the baseline.
Limitations
- An audit is a snapshot. AI answers vary from run to run, so treat one audit as a baseline and look at the trend across audits.
- API answers are not identical to what one person sees in a consumer chat app, which adds personal history, location, and (sometimes) live browsing.
- ChatGPT, Claude, and Gemini answer from their model knowledge, which can lag behind the live web. Perplexity and Google AI Overviews reflect current web results.
- Google AI Overviews do not appear for every search. When none appears, we report that instead of guessing.
- Mention and competitor detection is automated and can occasionally misread a name. The verbatim answer sits next to each result in the report, so you can check any result yourself.
- Scores are our weighting of what we observed. They are a decision aid, not an industry standard.
What you can confidently act on
- Which buyer questions name the business, and which name competitors instead
- The specific competitors AI recommends, in the client's own market
- Whether AI describes the business accurately when asked about it directly
- Concrete on-site gaps (schema, FAQ content, crawler access, citable facts) the agency can fix
- A baseline to compare against when the audit is run again after the work
No guarantees. No one controls what ChatGPT, Google, or any AI system recommends, and RediForAI does not promise rankings or placement. What it gives you is evidence: what AI says today, who it names instead, what is missing on the site, and a way to measure whether the work changed anything.
Common questions about our testing
If you're comparing your manual searches to our report, start here.