# /robots.txt file for http://www.caprino.no/ # mail webmaster@caprino.no for constructive criticism # Sitemap: http://www.caprino.no/sitemap.xml.gz User-agent: * Disallow: /shop/no/orderview.asp Disallow: /shop/da/orderview.asp Disallow: /shop/en/orderview.asp Disallow: /user/ #Private Disallow: /mail/ #Private Disallow: /private/ #Private Disallow: /temp/ #Private # Disallow: /forums/ #Web crawlers are currently unable to traverse directory correctly User-agent: Eniro Crawler Disallow: /media/doc #Stop downloading files you cannot read Disallow: /media/video Disallow: /forums/ User-agent: dotbot User-agent: Java User-agent: findlinks User-agent: findfiles User-agent: LiteFinder User-agent: MJ12bot User-agent: proodleBot User-agent: ShopWiki User-agent: CentiverseBot User-agent: CazoodleBot User-agent: BPImageWalker Disallow: / #Welcome back when you know how to crawl relative paths