30

Stories about: Robots Meta Tags

Methods to Block Spiders There are situations when you do not want that certain URLs of your website to be indexed by search enginesrobots. Fortunately, solutions exist if you need to remove outdated content like entire web pages, individual images and more. The general methods for blocking search engines robots use robots.txt files, specif... 10 January 2008 12:35 GMT