Tuesday, December 21, 2010

ShopWiki still ignores robots.txt

I seem to remember that this was a problem before (Oct/Nov 2006) that went away after putting the appropriate entry in robots.txt - this is what the ShopWiki help for webmasters suggests.

However, they seem to be ignoring the robots file again!

I have placed a Rewrite condition and rule in my server config. Let's see if that works - I won't publish what I did until I confirm that it works - there is more than enough of this on the Internet as it is!

No comments:

Post a Comment

Please do not enter links that are just promoting your own website or ones to those that you are shilling.
Comments with hyperlinks (that are not relevant to the original post) will be re-posted or deleted.
Sorry, this is to discourage "blog spam"