Terms of use

Last updated 31 October 2026.

1. The tools are provided free, as is, without warranty. Check generated files before publishing them; a sitemap does not guarantee crawling or indexing.
2. Only crawl websites you own or are authorised to crawl. Do not use the crawler to overload, probe or scrape sites, or to bypass access controls. We may block abusive use.
3. The crawler identifies itself, honours robots.txt by default and is rate-limited. Turning off robots.txt respect is for sites you control.
4. You are responsible for the content of the lists and files you process.
5. These terms may change; the date above shows the latest version.