Skip to content

Trialling an AI visibility tool? A two-week test plan before you pay for a year

A two-week test plan for an AI visibility tool trial: what to freeze on day one, ten answers to check by hand, stability, citations, export and real price.

By Published 14 min read
A reviewer with a clipboard works through a two-row calendar of trial days beside a panel of plain bar charts.

The short answer

A two-week trial of an AI visibility tool should prove five things before you pay for a year: it asks the engines the way your buyers do, its counts match answers you check by hand, its number holds still when nothing changes, it counts citations correctly, and you can export the raw answers at a price that fits your scale.

Key takeaways

  • An AI visibility tool trial is only useful if you freeze the prompt set, engines, countries and competitors on day one and judge the tool against that written plan.
  • Checking ten of the tool's stored answers by hand against the live engine catches most counting errors faster than reading any dashboard.
  • Ahrefs found that 14.4% of 66.6 million free, logged-out ChatGPT responses from August and September 2026 triggered a web search, so a trial should record whether each stored answer searched.
  • Across the 16 AI visibility tools with a US dollar monthly entry tier that we recorded on 5 and 6 October 2026, the median entry price was $79, so the real cost question is the tier your prompt set needs.
  • A tool that will not export raw answers with dates, engines and countries in the trial will not export them at renewal either.

What should a two-week AI visibility tool trial prove?

A two-week trial should prove that the tool measures your market the way you would measure it by hand, only faster. A dashboard that looks good proves nothing on its own. The test is whether the numbers behind it match the answers real buyers see, and whether they stay put when nothing has changed.

Most trials fail for a simpler reason. The team signs up, types in a few prompts, looks at the charts for a week and then decides on feel. By day fourteen nobody can say whether the tool was right, because nobody wrote down what right would look like. The plan below fixes that with a written test, ten hand checks and a scorecard.

It suits a marketing or search lead choosing a first tool, or a team replacing one at renewal. It assumes one country and one language. If you sell in several markets, run the same plan once per market, because each one is a separate sample with its own sources.

Prior work and a disclosure

Others have covered parts of this. Otterly, which sells AI search monitoring, published a stability study in September 2026 on how much brand mentions and cited sources move from day to day.1 MaxAEO, which sells AI visibility measurement, published guidance in July 2026 on sample size and why repeat runs of one prompt are clustered rather than independent.2 What this guide adds is the order of work for a fixed trial window. Sigzen AI sells audits and programmes that compete with tool subscriptions, so read the plan with that interest in mind.

Write the test down before day one

The first job is to write down what the tool will be tested on, before you log in. Choose the prompts, engines, countries and competitors, and freeze them for the whole trial. If the inputs change halfway, you are testing your own edits, not the tool.

Start with the prompt set. Take twenty to forty questions a buyer in your category would really ask, most of them without your brand name in them. Add a few comparisons with named rivals and a few that ask for alternatives. Our measurement spec explains why unbranded questions carry most of the signal.

Then list the engines your buyers use and the countries you sell to. Name three competitors. Write one sentence on what a pass looks like for each check in this guide. Keep the document short enough to read in two minutes, and send it to the vendor on day one so they know what you are testing.

The market is crowded enough that you will probably trial more than one product. G2 listed more than 150 products in its answer engine optimisation category by January 2026, up from 7 in March 2025.3 A written test is what lets you compare two of them fairly, because both get the same inputs.

What to freeze on day one, and what to leave to the tool

Frozen by you

  • Prompt set, word for word
  • Engines, countries and language
  • Three named competitors
  • Pass mark for each check

Left to the tool

  • Run schedule and repeat count
  • How mentions and citations are parsed
  • Charts, alerts and summaries

Recorded every day

  • Any engine or model release
  • Any change you made to your site
  • Any question you sent the vendor
Illustrative. A trial set-up the Sigzen AI editorial team uses when it tests measurement software.

Is the tool asking the engines the way your buyers do?

The first check is how the tool reaches each engine, because the route changes the answer. Some tools send prompts to a model through its developer interface. Others capture the consumer product, logged in or logged out, in a given country. Ask the vendor which route each engine uses, and write the reply into your test document.

Web search is the detail that matters most. Ahrefs, which sells SEO and AI visibility tools, studied 66.6 million free, logged-out ChatGPT responses from August and September 2026, across all countries, and 14.4% triggered a web search.4 An answer that searched can cite your pages this week. One that did not reflects what the model already learned. A tool should record which kind each stored answer is.

Engines also differ in how many sources they show. Semrush, which sells an AI visibility toolkit, analysed 126 million US prompts from January to April 2026 and found that ChatGPT cited about 15 sources per response, against about 3 for Gemini.5 A tool that reports one citation count for all engines hides that gap. Ask to see the count per engine.

Country is the last part of the route. Ask how the tool sets location for each engine and whether it can run your prompts in the language your buyers use. A tool that runs every prompt from one country will give a tidy number for a market you do not sell in.

How engines answer, 2026

Two numbers to put to any vendor

14.4%of free, logged-out ChatGPT responses triggered a web searchAhrefs, Aug–Sep 2026
~15sources cited per ChatGPT response, US promptsSemrush, Jan–Apr 2026
~3sources cited per Gemini response, US promptsSemrush, Jan–Apr 2026
Sources: Ahrefs ChatGPT query fan-out study, Semrush 2026 AI Visibility Index.

Check ten answers by hand

The fastest way to judge a tool is to check ten of its stored answers yourself. Pick ten prompts from the frozen set: five where the tool says you were named, five where it says you were not. Open the stored answer for each and read it in full.

For each one, ask three questions. Does the stored answer name you where the tool says it does? Did the tool count a rival correctly, including spelling variants and parent brands? Did it miss a mention that sits in a table, a list or a footnote? Note each disagreement with the prompt, the engine and the date.

Then run the same ten prompts yourself in the consumer product, in the same country, on the same day. Your answers will not match word for word, because engines vary between runs. They should agree on the broad picture: who tends to be named, and roughly where. If the tool says you lead and your own ten checks rarely find you, stop and ask the vendor why.

Two or three disagreements in ten are worth a question. Five or more mean the counting method is not fit for your market yet. Our guide to why a colleague's ChatGPT answer differs from the report covers the normal reasons two readings disagree, so you can tell them apart from real errors.

The ten-answer hand check

  1. Pick

    Five prompts the tool marks as a mention, five it marks as a miss.

  2. Read

    Open each stored answer in full and mark every brand it names.

  3. Compare

    Set your marks beside the tool's counts and log each disagreement.

  4. Repeat live

    Run the same ten prompts yourself in the same country on the same day.

  5. Ask

    Send the vendor every disagreement and judge the reply, not the apology.

Illustrative. A check the Sigzen AI editorial team runs on any measurement source, including its own.

Does the number hold still when nothing changes?

A good tool gives a similar number on two quiet weeks, and a range that tells you how similar to expect. Engines give different answers to the same prompt from run to run. A tool that reports a single percentage with no range invites you to read noise as progress.

Use the trial to measure the noise directly. Make no changes to your site during the two weeks, and log any change you cannot avoid. Ask the tool to run the frozen set at least twice, on separate days. Compare the two readings per engine. If the gap is wider than the range the tool reports, the range is too narrow.

Repeat runs of one prompt are not the same as more prompts. MaxAEO's guidance treats repeats of one prompt as clustered: they resemble each other, so each extra repeat adds less information than a new question would.2 Ask the vendor how it counts runs when it states a margin of error, and whether it treats repeats as clustered.

Engines can also change underneath a trial. OpenAI moved paid ChatGPT plans to GPT-6 Sol from 7 October 2026 and Free and Go users to GPT-6 Luna from 8 October.6 A release like that can move every brand in a category in one week. A good tool marks model changes on its charts. Our view: a vendor that cannot tell you which model answered each stored prompt is not ready to be your system of record.

Quiet-week test
Two runs of the frozen prompt set on separate days with no site changes in between. The gap between them is the tool's practical noise for your market.

Are the citations counted correctly?

Citations need their own check, because a mention and a citation are different things. A mention is your name in the answer text. A citation is a link to a page the engine drew on. A tool can count one well and the other badly.

Your own site should show up among the sources. Profound, which sells AI visibility tools, analysed 11.84 billion citations across eight engines from 16 April to 16 July 2026 and found that about 57% went to company-owned sites.7 If the trial tool shows almost none of your pages cited, check a few answers by hand before you accept it. The tool may be missing links in collapsed source panels, or it may be counting only one URL format.

Then check how the tool groups sources. It should separate your domain from subdomains, review sites and directories that list you, and it should let you see the exact URL. A source report you cannot open page by page cannot guide any fix.

There is one free cross-check for Microsoft's engine. Bing Webmaster Tools added an AI Performance report in public preview in February 2026, covering Microsoft Copilot, AI-generated summaries in Bing and select partner integrations.8 It counts how often your pages appear as sources. It is not a share-of-answer measure, but if the trial tool and Bing disagree on which of your pages get cited, ask why.

A figure with a magnifying glass compares two answer cards resting on the two pans of a balance scale.
A trial is a comparison: the tool's reading on one side, your own hand checks on the other.

Can you take your data with you?

Test the export in the trial, not at renewal. Download the raw answers for the frozen prompt set and open the file. If the trial tier blocks exports, ask for a sample file from your own runs before you sign.

The file should hold one row per answer, with the prompt text, engine, country, language, date and time, and the model or mode where the tool knows it. It should carry the full answer text and every cited URL. Counts alone are not an export, because you cannot recheck a count without the answer behind it.

Ownership matters as much as format. The prompt set you wrote is yours. Ask in writing whether you keep the raw answers if you cancel, and for how long they stay downloadable. Our guide to AI visibility tool contract terms lists the clauses that protect your history when a vendor changes owner.

A good export also lets you switch tools later without losing your baseline. Our view: if you cannot rebuild last month's number from the export with a spreadsheet, the tool is holding your measurement history hostage, however good the dashboard looks.

What will the tool cost at your real scale?

Price the tier your frozen prompt set needs, not the entry tier on the pricing page. Entry prices are low. In the published prices we recorded on 5 and 6 October 2026, the 16 AI visibility tools with a monthly entry tier in US dollars had a median entry price of $79, ranging from $19 to $800.9

The bill grows with what you track. Most tools price on some mix of prompts, engines, countries, runs and seats. Multiply your prompt count by the engines and countries you need, then by the runs per month. Compare that total with each tier's limits. A cheap entry tier that tracks only one engine, or runs monthly, may not cover the test you just designed.

Ask four pricing questions in the trial. Which engines cost extra? Is each repeat run counted against the limit? What happens when you exceed it mid-month? Does an annual plan lock the price for the full term? Write the answers into the scorecard beside the price.

Then set the yearly total against the hours the tool saves. A tool removes the repetitive part of measurement. It does not read lost answers, write pages or fix templates for you. Our guide to what GEO costs compares tools, agencies and in-house work on the same basis.

A two-week trial, day by day

  1. Send the vendor the written test; load the frozen prompt set, engines, countries and competitors.

  2. First full run; ask how each engine is reached; record web-search flags and citation counts per engine.

  3. Ten-answer hand check against stored answers and live runs; send every disagreement to the vendor.

  4. Second run on a separate day; quiet-week comparison; citation check against your own pages.

  5. Export test; pricing questions at your real scale; read the contract terms.

  6. Score the tool against the written test and decide.

Illustrative. A schedule the Sigzen AI editorial team would use for one country and one language.

A scorecard to decide on day fourteen

Decide against the written test, one line per check, and let any failed line block the purchase. A tool that wins on charts but fails the hand check is the wrong buy. A plain tool that passes every line is the right one, even if its dashboard is dull.

CheckPass looks likeFail looks like
Engine routeRoute stated per engine and country"We use the latest models" and no detail
Hand checkAt most two disagreements in ten, explainedFive or more, or no access to stored answers
Quiet weeksGap between runs inside the stated rangeOne percentage, no range, big swings
CitationsExact URLs per engine; your pages foundDomain counts only; your pages missing
ExportOne row per answer, full text and URLsCounts only, or export locked to top tier
Price at scaleFrozen set fits a named tier and priceLimits unclear until after you sign

Illustrative. Pass marks set by the Sigzen AI editorial team for a one-country trial; set your own in the written test.

If two tools both pass, pick the one whose answers you could defend to a sceptical colleague. That usually means the one with the clearer stored answers and the more honest range, not the higher score. Higher scores often come from easier prompts.

Keep the scorecard after you buy. At renewal, rerun the hand check and the quiet-week test on the same frozen set. A tool that passed in the trial and fails a year later has changed its method, and you will want to know before you sign again.

When is a tool trial the wrong test?

A trial is the wrong test when you do not yet know which prompts matter, or when nobody will act on the numbers. In both cases the tool will run well and change nothing. Fix the input or the owner first.

If you cannot write twenty buyer questions with confidence, start by reading answers yourself. Our guide to rerun schedules shows how often a small manual set needs checking, and a short manual pass will tell you which questions are worth paying to track.

If nobody has hours to read lost answers and fix pages, a tool becomes a monthly report that nobody opens. In that case, price the work, not the software. A one-off audit sets a baseline and a method that a later tool can inherit, and an outside programme covers the hours.

Our view: buy the tool after you know what you would do with its numbers. The best trial question is not which dashboard looks best. It is which tool would have told you something true that you then acted on.

Sources

  1. Otterly, AI search visibility stability study (Sep 2026)
  2. MaxAEO, AI visibility sample size and clustered runs (Jul 2026)
  3. G2, growth of the AEO software category: products listed on G2, Mar 2025–Jan 2026 (Jan 2026)
  4. Ahrefs, ChatGPT query fan-out study: 66.6M free, logged-out GPT-5.6 Luna responses, Aug–Sep 2026, all countries; factual subset 20,000 US prompts; citation ranks US, Mar–Sep 2026 (Oct 2026)
  5. Semrush, 2026 AI Visibility Index: 126M US prompts, Jan–Apr 2026 (Jun 2026)
  6. OpenAI, ChatGPT release notes: GPT-6 Sol for paid tiers from 7 Oct 2026 and GPT-6 Luna for Free and Go from 8 Oct 2026 (Oct 2026)
  7. Profound, where AI citations come from: 11.84B citations, 8 engines, 16 Apr–16 Jul 2026 (Jul 2026)
  8. Microsoft Bing, AI Performance in Bing Webmaster Tools, public preview (Feb 2026)
  9. Sigzen AI, published GEO prices: 34 of 41 agencies listed and 25 tools, seen 5–6 Oct 2026 (Oct 2026)

Questions readers ask

  • An AI visibility tool trial should test six things against a written plan: how the tool reaches each engine and country, whether its counts match ten answers checked by hand, whether two quiet runs agree within its stated range, whether citations are counted by exact URL, whether raw answers export in full, and what the tier for your real prompt set costs.

Keep reading

Free. Enter your domain and Sigzen AI checks how five engines answer your buyers' questions, against up to three competitors.

No account needed. Two minutes to submit; we email the full five-engine report when the check has run.