WebForAgents
HomeAll sitesOpen a fast laneBlog
English▾
WebForAgents
WebForAgents
WebForAgents

Open an AI agent fast lane on your website and stay a step ahead of your competitors in the age of agents.

Product

Free scanOpen a fast laneAll sites

Resources

BlogLive test reportOur scanner

Legal

Terms of ServicePrivacy PolicyLegalContactCookie settings
© 2026 Funovate Technology LLC. All rights reserved.
WebForAgents

Open an AI agent fast lane on your website and stay a step ahead of your competitors in the age of agents.

Product
Free scanOpen a fast laneAll sites
Resources
BlogLive test reportOur scanner
Legal
TermsPrivacyLegalContactCookie settings
© 2026 Funovate Technology LLC. All rights reserved.

The WebForAgents scanner

WebForAgents measures how well AI agents can read and use a website. When someone enters a site on webforagents.org, our scanner visits a few public pages of that site, checks what an agent would receive, and publishes a report. It does not log in, submit forms, open accounts or bypass any verification.

How to recognise our requests

The first phase fetches pages over HTTP with this User-Agent:

Mozilla/5.0 (compatible; WebForAgents/1.0; +https://webforagents.org/bot)

Pages that need a browser are opened in Chrome exactly as a browser-based AI agent (Playwright, browser-use, agentic browsers) would open them: Chrome’s standard User-Agent, with no name added and no extra headers, so that we see what those agents see. You can recognise these visits by the IP addresses below:

<Chrome’s standard User-Agent>

HTTP requests (phases 1 and 2) also carry From: [email protected]. First-phase requests carry an X-WebForAgents-Scan header with the scan’s ID; quote it when you contact us about a specific visit.

Our requests come from these IP addresses: 43.130.47.235

How much we visit

  • Phase 1: typically 20 to 35 requests per scan (including robots.txt and redirects), over one connection per host at a time (at most 3 requests in flight on HTTP/2), finished within 8 seconds.
  • Phase 2 (wayfinding estimate): reads the sitemap listed in your robots.txt (or /sitemap.xml, at most 5 files), then follows links from the homepage up to 4 clicks deep. At most 100 pages, plain GET over one connection, one request per second (we follow your Crawl-delay up to 5 seconds), usually finished within 5 minutes. Pages whose content appears only after JavaScript runs are opened in the browser instead, at most 20 per scan. It never follows logout, account, checkout, add-to-cart, download, search-result or one-time-token links, and stops as soon as it gets a 429 or 503.
  • Phase 3: at most 15 pages per scan (the pages on the way to the tasks people typically bring to your site), opened one at a time like an ordinary visitor. Images, media and fonts are not downloaded, and nothing is clicked or typed.
  • A site is scanned at most 3 times per day on request, at least 10 minutes apart. Results are reused for 7 days. Viewing a report never triggers a new scan.

robots.txt

We read robots.txt first and follow it under RFC 9309. Our product token is WebForAgents; if there is no group for it, we follow the * group. Paths you disallow are not fetched and the checks that need them are marked as not applicable.

Opting out

The simplest way is robots.txt. This stops all scans of your site:

User-agent: WebForAgents Disallow: /

The report then says that the site does not allow testing, shows no score and is not indexed. If you would rather not change robots.txt, email [email protected] from an address at your domain, or ask us for a DNS TXT record or a file under /.well-known/ to prove you run the site. We can opt out a single host or your whole domain. Existing reports stay available, marked with the date you opted out, and are no longer indexed.

Contact

Questions, complaints or a visit you want explained: [email protected].

Notifications
No notifications yet