Is your website blocking ChatGPT?
Many website builders and security plugins block AI crawlers without telling you. If ChatGPT can't read your site, it can't recommend you. Enter your address to check your robots.txt in seconds.
What this checks
Your robots.txt file tells crawlers which pages they may read. This tool reads yours and tests whether each of the eight AI-related crawlers is allowed to read your home page. Five of them decide whether you can be found in Google, Bing, ChatGPT and Perplexity. The other three are used only to train models, and allowing them is your choice.
Which crawlers does this check?
Googlebot (Google Search and AI Overviews), Bingbot (Bing and Copilot), OAI-SearchBot (ChatGPT search), ChatGPT-User (pages fetched during a chat), PerplexityBot, and the three training crawlers GPTBot, ClaudeBot and Google-Extended.
Should I block the training crawlers?
That's your choice. GPTBot, ClaudeBot and Google-Extended collect pages to train future models. Blocking them does not stop ChatGPT or Google from finding you through search, which is what matters for being recommended today.
My site has no robots.txt. Is that a problem?
Not for crawlers: with no file, everything is allowed. But a robots.txt that lists your sitemap helps search engines find all your pages, so it's worth adding one.
The tool says allowed, but ChatGPT still doesn't mention me. Why?
Being allowed in is only the first step. ChatGPT also needs clear facts on your pages and mentions of your business on other websites. Our guide on getting recommended by ChatGPT in Malaysia covers the rest.
Read next: How to get your business recommended by ChatGPT in Malaysia.