Portal Home > Knowledgebase > Articles Database > Robot.txt question
Robot.txt question
| Posted by alsoRan, 07-23-2008, 12:20 PM |
| I want to stop cataloging of all PDF files on my site, can this be done using wild cards through Robot.txt?
What is a good way to test to make sure that the instructions in the Robot.txt are working?
Thanks
Mark
|
| Posted by TheSimpleHost-Nathan, 07-23-2008, 01:24 PM |
| There's a little information here:
http://www.mcanerin.com/EN/search-engine/robots-txt.asp
and to validate:
http://tool.motoricerca.info/robots-checker.phtml
|
| Posted by alsoRan, 07-23-2008, 02:36 PM |
| Thank you, apparently the wild card does work, but for only particular bots.
|
Add to Favourites 
Print this Article
Also Read
Cold Fusion (Views: 452)
