The short answer
If ChatGPT, Gemini or Google's AI Overviews never mention your business, poor content is usually not the reason. Two things stop most small business websites in India from being cited, and both sit above the content layer: the AI crawler cannot fetch your pages at all, or it can fetch them but cannot find a clean answer to quote.
Both are free to check and take about twenty minutes. Do that first. Everything else in AI search optimisation is wasted effort until those two are true.
Why this is suddenly worth your afternoon
India is ChatGPT's second largest market. Sam Altman put the figure at 100 million weekly active users here in February 2026, and said India has the largest student user base of any country. Google's AI Overviews have been running in Indian results since 2024 in English and Hindi, and AI Mode has since arrived in several Indian languages.
The practical effect is easy to describe. A growing share of the questions that used to land someone on your website now get answered before they leave the results page.
"Best paediatric dentist in Banjara Hills." "WooCommerce developer in Hyderabad under 50k." "Is Shopify cheaper than WooCommerce for a 200 product store." Each of those now returns a paragraph of an answer with two or three businesses named inside it.
You are either one of the named businesses or you are not. There is no page two to fall back to.
The encouraging part is that this is a technical bar rather than a financial one. AI systems do not choose sources by ad budget or domain age. They choose pages they can fetch, parse and quote without much risk of being wrong. A five page site for a clinic in Kukatpally can get cited alongside a national chain if the plumbing is right, and plenty of expensive sites are invisible because nobody ever looked at the plumbing.
Check one: can the AI crawlers actually reach you?
This is the check almost nobody runs, and it is the most common hard blocker we come across.
Most small business sites in India sit behind Cloudflare, usually on the free plan, usually set up by the hosting company or a previous developer. Somewhere in that dashboard is a switch that blocks AI bots. It gets turned on for sensible reasons. Scrapers are a genuine nuisance, they eat bandwidth, and "block AI bots" reads like straightforward security hygiene.
It is not quite that. That one switch does not separate the bots that scrape your content for model training from the ones that fetch your page because a real person just asked a question about your industry. Flip it and you have removed yourself from the answer, not just from the training set.
A site can be perfectly optimised, schema and all, and still return a 403 to every crawler that would have cited it. We have opened client sites where the entire AI search problem was one toggle.
The test almost everyone runs, and why it lies to you
Search for how to check this and you will be told to run something like:
curl -A "GPTBot" -I https://yourdomain.comYou will very likely get a 403 back. That 403 usually does not mean what you have just been told it means.
Cloudflare and most serious bot protection do not decide anything based on the user agent string, because a user agent string is just text and anything can claim to be anything. They verify by IP range and reverse DNS lookup instead. So when a request arrives from your broadband connection in Hyderabad announcing itself as GPTBot, it gets blocked as a spoofer, which is exactly correct behaviour. Meanwhile the real GPTBot, arriving from OpenAI's own address ranges, may be getting through without any trouble at all.
We ran this test on our own site and spent the better part of a day acting on a result that was simply wrong. The only reliable way to answer the question is to stop probing from outside and read what the server itself recorded.
Where the real answer lives. In Cloudflare, open AI Crawl Control and look at the request log. It lists every AI crawler that hit you, how many times, and whether each request was allowed or blocked. If you are not on Cloudflare, your host's raw access logs will show the same thing. Bing Webmaster Tools is worth a look too, since ChatGPT's live web results have historically leaned on Bing's index.
Google Analytics will not help here. Crawlers do not run JavaScript tags, so none of this traffic ever appears in GA4.
Blocking training bots is not the same as blocking AI search
This distinction is where most of the damage happens, and it is worth being precise about. Every AI company runs more than one crawler, and they do different jobs.
| Crawler | Run by | What it does | Blocking it |
|---|---|---|---|
| GPTBot | OpenAI | Collects content to train models | Your call, no effect on citations |
| OAI-SearchBot | OpenAI | Builds the index ChatGPT searches | Removes you from ChatGPT results |
| ChatGPT-User | OpenAI | Fetches a page when a user asks about it | Your page cannot be opened or quoted |
| ClaudeBot | Anthropic | Collects content for training | Your call, no effect on citations |
| Claude-User | Anthropic | Fetches a page on a user's request | Your page cannot be opened or quoted |
| PerplexityBot | Perplexity | Builds Perplexity's search index | Removes you from Perplexity answers |
| Googlebot | Feeds Search, AI Overviews and AI Mode | Removes you from Google entirely |
Read that table once and the trade-off becomes obvious. If your objection to AI is that your writing is being used to train a model without payment, block the training crawlers and keep the retrieval ones. You stay quotable and you stay out of the training set. Blocking everything with one switch gives up the visibility for no additional protection.
One more wrinkle worth knowing about: on Cloudflare, a managed rule outranks the individual per crawler toggles. You can set a single bot to "allow" in the interface and still watch it get blocked, because a broader managed rule above it is still doing its job. The hover text says so, in small print, and it is easy to miss.
The robots.txt line that quietly says no
There is a newer signal in robots.txt worth checking while you are in there. It looks like this:
Content-Signal: search=yes, ai-input=no, ai-train=noThe trap is in the word search. Under the Content Signals policy, search covers building a traditional search index with links and short excerpts, and it explicitly excludes AI generated summaries. The permission that covers being used inside a live AI answer is ai-input, which is separate.
So a site with search=yes, ai-input=no has, without anyone intending it, said: index me, but do not use me in answers. If you want to be cited but not trained on, the honest version of that line is search=yes, ai-input=yes, ai-train=no.
Cloudflare can inject its own managed robots.txt above the file you wrote, and the two can end up contradicting each other on the same page. Do not check your file in your code editor. Open yourdomain.com/robots.txt in a browser and read what the server is actually serving.
Check two: can a model find an answer once it gets in?
Assume the crawler is now getting through. The next question is whether there is anything on the page worth lifting.
Language models do not read a page the way you skim one. They pull out passages that stand on their own, which means a paragraph that answers a specific question completely, without depending on the three paragraphs above it for context. If your service page opens with two hundred words about your journey and your passion for excellence, there is nothing there to quote.
- Answer in the first hundred wordsPut the direct answer near the top of the page, then explain. If someone asks what a WooCommerce store costs, the number goes early, with the caveats after it.
- Use real questions as headings"How long does a website take to build?" gets matched to a query. "Our Process" does not. Write the heading the way a customer would type it.
- One fact per paragraphShort, self-contained paragraphs survive extraction. Long ones with three ideas woven together get skipped because no clean chunk can be pulled out.
- Put the content in the HTMLIf your page renders its text through JavaScript after load, some crawlers will see an empty shell. View the page source, not the inspector, and confirm your actual words are in there.
- Keep the heading order cleanOne H1, then H2s, then H3s under them, with no skipped levels. This is what lets a model work out which answer belongs to which question.
- Add schema that matches the visible pageLocalBusiness or Organization details, FAQPage for a genuine FAQ, and every value in the markup identical to what a human sees. Invented schema values are worse than none.
Check three: does your business exist as a single, consistent entity?
An AI system recommending a business is making a small bet on being right. What it looks for is agreement. Your name, address and phone number should match exactly across your website, your Google Business Profile, Justdial, IndiaMART, Sulekha and anywhere else you are listed. Not roughly. Exactly.
This is where a lot of Indian businesses quietly lose. Three different phone numbers across four directories, "Pvt Ltd" on one listing and "Private Limited" on another, an old shop address that never got updated after a move. A human reads through all of that without noticing. A model treats it as low confidence and picks the competitor whose details agree with themselves everywhere.
Two things worth doing while you are on this:
- Claim Bing Places. Almost nobody in India bothers, and ChatGPT's live results have leaned on Bing. It takes about fifteen minutes and the field is close to empty.
- Check what your Google Business Profile website field points at. If it still points at an old domain that redirects, your strongest local signal is arriving second hand. We found exactly this on our own profile.
What we changed on our own site
We put our own site through this before writing any of it down, which is how we know the spoofed user agent test is misleading. The server logs told a different story from the one our first test suggested, and once we read them the actual problems turned out to be narrow and specific.
One assistant crawler was genuinely blocked by a managed rule that swept it in as a training bot even though its job is retrieval. Our Content-Signal line said search=yes and never declared ai-input, so retrieval was neither granted nor refused. And the robots.txt being served contradicted the one in our repository, because a managed version was being injected above it.
None of that was visible from the front end. All three were fixed in an afternoon once we stopped guessing.
What to expect once it is fixed
Being fetchable is a requirement, not a result. Fixing a blocked crawler does not put you in an AI answer next Tuesday, in the same way that fixing a broken sitemap does not put you at the top of Google.
What it does is make you eligible. From there the ordinary work applies: pages that answer real questions, a business whose details agree with themselves everywhere, and genuine proof that you have done the thing you say you do. Reviews, case studies with real numbers, named clients.
Anyone selling you a guaranteed spot in ChatGPT is selling something they cannot control, in exactly the way ranking guarantees have always worked. What is controllable is whether the door is open. Most of the time, for most small business sites we look at, it is not.
Common questions
How do I know if my site is blocking AI crawlers?
Read your server's own logs rather than testing from outside. On Cloudflare, open AI Crawl Control and check the request log, which shows each AI crawler that visited and whether it was allowed or blocked. On other hosts, the raw access logs show the same. Testing with a spoofed user agent from your own computer returns a false 403, because bot protection verifies crawlers by IP address and reverse DNS, not by the user agent string.
Should I block AI bots from my website?
Only the training crawlers, and only if you object to your content being used to train models. Blocking GPTBot or ClaudeBot has no effect on whether you get cited. Blocking the retrieval crawlers, such as OAI-SearchBot, ChatGPT-User, Claude-User and PerplexityBot, removes you from AI answers entirely. Most single switch "block AI bots" settings do not make that distinction.
Does getting cited by ChatGPT actually bring visitors?
Some, and they tend to be well qualified, because the person has already read a description of what you do and chosen to click through anyway. Treat it as a channel that supplements search rather than replaces it. Google is still where most of the volume is.
Do I need a separate strategy for AI search and normal SEO?
No. The overlap is close to total: clean semantic HTML, accurate structured data, content that answers real questions, and a consistent business identity across the web. The genuinely new parts are crawler access at the server level and writing passages that can be quoted on their own. Anyone selling AI search as a separate discipline with its own budget is mostly repackaging technical SEO.
My site is on WordPress with a security plugin. Could that be blocking crawlers?
Yes, and this is common. Wordfence, Sucuri and most rate limiting plugins can block or throttle crawlers they do not recognise, and some SEO plugins write their own robots.txt rules that override what you think is live. Check the plugin's blocked request log, then open yourdomain.com/robots.txt in a browser to confirm what is actually being served.
How long does it take to show up in AI answers after fixing this?
Unblocking a crawler makes you eligible, not visible. Retrieval based answers can pick up a page within days of it being fetched, while anything depending on a search index follows that index's own refresh cycle, which is usually weeks. Nobody can give you a date, and a firm promise of one is a warning sign.
If you would rather not do this yourself
Every check above is something a business owner can run without any special access, and we would rather you ran them than paid someone to tell you the results. If you get partway in and find something you do not want to touch, or you would like someone to go through the whole site properly, message us on WhatsApp and we will take a look.