MCOAI-SiteBot
A small crawler that checks website health for MCOAI Business Growth customers.
What is it?
MCOAI-SiteBot is a program that opens website pages to check for technical problems such as broken pages, missing titles or slow pages. The results are shown to the website owner only.
Verified websites only
We only scan websites whose owners proved ownership (with a meta tag, a file or a DNS record) and pressed "Scan now" themselves. There are no scheduled scans and no scans of other people's websites.
Polite and obeys robots.txt
MCOAI-SiteBot obeys robots.txt and Crawl-delay, opens one page at a time at least 1 second apart, stays on the same domain and stops at a page limit. It does not keep page content.
How to recognise it
Our User-Agent:
MCOAI-SiteBot/1.0 (+https://creator.mcosuper.app/bot)
How to block it
Add these lines to your website's robots.txt file:
User-agent: MCOAI-SiteBot
Disallow: /