Tuesday, December 21, 2010

ShopWiki still ignores robots.txt

I seem to remember that this was a problem before (Oct/Nov 2006) that went away after putting the appropriate entry in robots.txt - this is what the ShopWiki help for webmasters suggests.

However, they seem to be ignoring the robots file again!

I have placed a Rewrite condition and rule in my server config. Let's see if that works - I won't publish what I did until I confirm that it works - there is more than enough of this on the Internet as it is!

No comments: