- GPTBot, ClaudeBot and PerplexityBot fetch your HTML without executing JavaScript. They do not wait for rendering and they do not try again.
- A client-side rendered store returns an empty shell to those crawlers, so the catalog an assistant would recommend from does not exist as far as it is concerned.
- Googlebot does render JavaScript, which is why this problem hides. Your Google traffic can look acceptable while AI visibility sits at zero.
- You can check it in two minutes by turning JavaScript off and reloading. Whatever survives is roughly what the crawler receives.
There is a version of your store you have never looked at.
It is the version that arrives before any JavaScript runs. Your browser receives it, executes the scripts, builds the page, and shows you the finished result in a few hundred milliseconds. You never see the intermediate state, so it is easy to assume it does not matter.
For a growing share of the software now deciding which businesses get recommended, that intermediate state is the whole page.
The Gap Nobody Checks
Take any store built on a modern JavaScript framework. Load it in Chrome and you get product images, prices, descriptions, reviews. Request the same URL the way a script does, reading only the HTML the server sends back, and you can get something close to this: a head section, an empty div, and a handful of script tags.
Both are the same page. The difference is who does the work of building it. Server-side rendering builds the document on the server and sends it finished. Client-side rendering sends a shell and instructions, and the browser assembles the page afterwards.
To a human the two are indistinguishable. To a crawler that executes JavaScript they are also roughly equivalent. To a crawler that does not, one is a product catalog and the other is nothing at all.
Which Crawlers Render JavaScript
This is the part that surprises people, because the search crawler everybody optimizes for behaves differently from the AI crawlers nobody has got around to yet.
| Crawler | Run by | Executes JavaScript | What it sees on a client-rendered page |
|---|---|---|---|
| Googlebot | Google Search | Yes, with delay | The rendered page, on a second pass, with partial coverage |
| GPTBot | OpenAI | No | The initial HTML only |
| OAI-SearchBot | OpenAI | No | The initial HTML only |
| ClaudeBot | Anthropic | No | The initial HTML only |
| PerplexityBot | Perplexity | No | The initial HTML only |
Published crawler analyses are consistent on this point. The AI crawlers consume the first server response and nothing else. There is no partial credit and no second attempt.
Googlebot renders, so your Google traffic can look perfectly healthy while your AI visibility sits at zero. Nothing in Search Console reports it. The first symptom most brands notice is that assistants recommend competitors and never mention them, which is easy to misread as a content problem rather than a rendering one.
Why This Got Expensive
Crawl volume tells the story. Across Vercel's network in December 2024, GPTBot generated 569 million requests and Anthropic's Claude crawlers generated 370 million. Separate analysis found GPTBot's crawl volume rose by 305% between May 2024 and May 2025, reaching roughly 30% of AI crawler traffic.
Those crawlers are building the answer layer that sits in front of your store. When somebody asks an assistant for a recommendation, the assistant works from what it managed to read. A store it could not read is not ranked lower. It is absent from the consideration set entirely.
How to Test Your Store in Two Minutes
You do not need a tool for the first pass.
Open a product page and view the source
View source, rather than the inspector. The inspector shows the page after JavaScript has run. View source shows the HTML as the server delivered it, which is what a non-rendering crawler receives.
Search that source for your product name and price
If both appear as plain text, the server is sending them. If you find an empty div and a list of script tags where the product should be, it is not.
Turn JavaScript off and reload
In Chrome, open DevTools, press Ctrl+Shift+P, and run "Disable JavaScript". Whatever still renders is approximately the crawler's view. A blank page is your answer.
Repeat on a category page and the homepage
Results often differ by template. A store can serve product pages correctly and still deliver category listings client-side, which breaks the path a crawler would follow to find the products.
The Four Fixes, Ranked
Ordered by how much they cost to implement, cheapest first.
Move critical content into the server response. Often the catalog renders server-side and only certain blocks do not, usually reviews, specification tabs, or descriptions loaded by an app. Moving those into the template solves most of the problem without touching the architecture.
Put your metadata in the HTML. Title tags, meta descriptions, canonical tags and structured data injected by JavaScript reach nothing that does not render. These belong in the document the server sends, every time.
Add prerendering. A prerendering layer generates static HTML snapshots and serves them to crawlers while human visitors get the dynamic version. It is a bolt-on rather than a rebuild, which makes it the practical option for a site already in production.
Move to server-side rendering. The real fix. Frameworks support it natively now, and it benefits human visitors too through faster first paint. It is also the largest project on this list, so it tends to arrive with a redesign rather than before one.
If You Are on a Hosted Platform
Most hosted store platforms server-render product pages by default, so the core catalog is usually safe. The gaps come from themes and apps.
Independent brands on hosted platforms, from Allured Seduction to any store you could name on the same stack, inherit whatever rendering behavior their theme and installed apps produce. That is worth knowing, because the decision is not usually one anybody made deliberately. A theme loads product descriptions after page load, a reviews app injects its content client-side, a quick-view feature replaces a static listing, and the store ends up with a crawler-visible surface far smaller than the one shoppers see.
The check is the same. Turn JavaScript off, walk your main templates, and note what disappears. Anything missing is a theme edit rather than a platform migration, which usually makes it a cheap afternoon rather than a project.
Need this diagnosed properly?
Browse vetted technical SEO agencies that audit rendering, crawl access and AI visibility rather than just keywords.
Compare agenciesReadable HTML is the entry requirement, not the whole job. A crawler has to be able to read the page before anything else matters, and being readable does not make the content worth citing. Fix the rendering, then earn the citation with information nobody else has.
Frequently Asked Questions
Most do not. Published crawler analyses show that GPTBot, ClaudeBot and PerplexityBot fetch HTML without executing JavaScript, so anything injected client-side is invisible to them. They do not wait for rendering and they do not make a second attempt. Googlebot does render JavaScript, with delay and partial coverage.
Disable JavaScript in your browser and reload the page. Whatever remains is roughly what a non-rendering crawler receives. You can also view the page source rather than the inspector, since source shows the HTML as delivered. If you see an empty div and script tags where your product information should be, your content is client-side rendered.
Less severely. Googlebot renders JavaScript, but it does so in a second pass that can lag the initial crawl, and coverage is partial. Client-side rendering tends to produce slower indexing and inconsistent results on Google rather than total invisibility. The total invisibility problem is specific to crawlers that never render at all.
Server-side rendering builds the full HTML on the server and sends a complete document. Client-side rendering sends a near-empty HTML shell plus JavaScript, and the browser builds the page after the fact. A human sees the same result either way. A crawler that does not execute JavaScript sees a finished page in the first case and an empty shell in the second.
Partly. Most hosted store platforms serve product pages as server-rendered HTML by default, so the core catalog is usually fine. Problems appear with themes and apps that inject content client-side, including reviews, specification tabs, and dynamically loaded descriptions. Check a product page with JavaScript off, and move anything missing into the theme template.
No. Readable HTML is the entry requirement rather than the whole job. A crawler has to be able to read the page before anything else you do matters, but being readable does not make the content worth citing. Original information, clear structure, and genuine expertise decide that.
Sources & Further Reading
- Agentic commerce readiness · Vercel
- JavaScript rendering and AI crawlers: can LLMs read your SPA? · Passionfruit
- Web crawler explained: GPTBot vs Googlebot · Bluetick Consultants
- Does ChatGPT and AI crawlers read JavaScript? · SEO.ai