# robots.txt --- index my site for search but don't archive it User-agent: * Disallow: # Tell Alexa Internet not to keep a copy User-agent: ia_archiver Disallow: / # desactivation HTTrack User-agent: HTTrack Disallow: / User-agent: Microsoft URL Control - 6.00.8862 Disallow: / # desactivation Pompos User-agent: pompos Disallow: /