Robots.txt Notes

A good page on how to configure a robots.txt file. It gives enough examples, like how to disable a single page from being indexed, while remaining pretty easy to follow.


http://www.searchtools.com/robots/robots-txt.html
Related Scribbles:


ID: 351
Author:
leonard
Date Updated:
2003-11-08 14:18:29
Date Created:
2003-11-08 14:18:03

Edit

Comments?
 >> Leonard Chan's Homepage  >> Scribble Web  >> Robots.txt Notes
leonard.lotus-land.ca is hosted by Perceptus Solutions Inc.