Files stay on your device Use Robots.txt Validator ↓

Robots.txt Validator Online — Free & Private

Check robots.txt groups and directive syntax. Free, private, and processed entirely in your browser — no upload, no account, no file ever leaves your device.

No sign-up required Files stay on your device Works offline once loaded

Check robots.txt groups and directive syntax.

Inputs and example for Robots.txt Validator

robots.txt
User-agent: *
Allow: /
Sitemap: https://example.com/sitemap.xml

Supported scope: Syntax checks only; does not simulate all crawlers, wildcard matching or CDN access.

Check robots.txt groups and directive syntax.

  1. Enter the labeled values or edit the example.
  2. Run the operation and inspect the result.
  3. Review and copy or download the result.

Privacy proof for Robots.txt Validator

This tool processes input in your browser. Open DevTools → Network while you run it to check the requests. Some tools need downloaded libraries on their first use.

Popular available tools for everyday tasks. Browse categories or search the full catalog.

Star any tool — it appears under Favorites on this device.

Available tools process your files and text on your device. Choose the operation you need, check its supported inputs and limits, and keep the original file until you have reviewed the result. PDF compression offers a lossless mode and optional image-based modes; image-based exports lose selectable text. Scanned PDFs need OCR for text extraction. OCR language models may need an initial download. Some catalog operations are still being completed and are clearly marked as unavailable. A current browser and sufficient device memory are required. Optional sign-in stores activity metadata, not your documents.

What inputs and assumptions does this operation use?

Syntax checks only; does not simulate all crawlers, wildcard matching or CDN access.

Is my input uploaded?

No. The operation runs in this browser tab.

Paste the text of your robots.txt file into the validator. Run the check and look at the issues list. Each message identifies the line to edit. To check a published file, open /robots.txt on the exact host you are investigating and copy its contents.

Try a broken file

Disallow: /drafts/
User-agent: *
Sitemap: /sitemap.xml

The checker reports Line 1: rule before user-agent. and Line 3: invalid sitemap URL. Move the crawler group above its rule and supply the full sitemap URL. For example, this version returns an empty issues list:

User-agent: *
Disallow: /drafts/
Sitemap: https://example.com/sitemap.xml

Replace the example host and paths with your own before using them. If the file references a sitemap, you can also check its XML.

What can pass this check and still block my pages?

User-agent: *
Disallow: /

These two lines pass the syntax check. Together they ask crawlers covered by the group to stay out of the whole site. Review the paths you intended to block before publishing. The validator reads pasted text. It cannot fetch the deployed file, predict wildcard matches or tell you whether Google indexed a URL.

Check each host separately. A rule on www.example.com does not cover tools.example.com. For Google’s matching rules and how it handles a missing or unavailable file, see Google’s robots.txt specification.

From our blog: How to Use Robots.txt Validator Online (Free & Private)