TuneAIBot Crawler Information

TuneAIBot Crawler Information

User-Agent:
TuneAIBot/1.0 (+https://www.tuneaibot.com/crawler-info)

Purpose

TuneAIBot is a web crawler operated by TuneAIbot.com, a conversational AI platform that powers knowledge-based chat assistants for client websites.
The crawler’s purpose is to collect publicly available web content — such as FAQs, service descriptions, and blog posts — to help website owners build and update their AI-driven knowledge bases.
TuneAIBot does not index pages for public search and does not display or redistribute crawled content.

How It Works

TuneAIBot visits URLs specified by site owners or by users of our SaaS system.

It fetches only publicly accessible HTML or PDF content.

It obeys all directives in your site’s /robots.txt file and standard meta robots tags.

Crawl frequency and depth are limited to avoid server overload.

Identification

You may identify our requests by the User-Agent header:

TuneAIBot/1.0 (+https://www.tuneaibot.com/crawler-info)

Controlling Access

To prevent TuneAIBot from accessing your site, you can use the following rule in your robots.txt file:

User-agent: TuneAIBot
Disallow: /

Contact

If you have any questions or concerns regarding our crawler’s activity, please contact:
📧 crawler@tuneaibot.com