# This is the robots.txt file for Carrel.ORG # You may be seeing this file instead of the usual content if your subnet was being # abusive, for example running a crawler that does not honor robots.txt. # 219.133.39.0/24 - Chinese bot does not honor robots.txt User-agent: Mediapartners-Google* Disallow: # This bot will attempt to load 8-10 images at a time and flood my link, so it gets nothing. User-agent: Yahoo-MMCrawler Disallow: / # Bad manners graphics downloads also at MSN. User-agent: msnbot-media Disallow: /gallery Disallow: /albums # Bad manners graphics downloads also at Yeti. User-agent: Yeti Disallow: /gallery Disallow: /albums # Stupid bots need to learn not to download 3MB images on my link User-agent: * Disallow: /admin/ Disallow: /admin Disallow: /accounts/ Disallow: /images/today.mp4 Disallow: /images/yesterday.mp4