Portal Home > Knowledgebase > Articles Database > Keeping Files from coming up on google with IIS
Keeping Files from coming up on google with IIS
| Posted by reddem0n, 10-28-2009, 12:02 PM |
| Hey all,
I was doing a search on google and retrieved some files on it with some sites that should not be available to the public. I investigated the site a little bit and it looked like they are running ASP. I know with Linux servers you can place a .htaccess file which can restrict bots from accessing particular directories, but how can you do it with a windows server running IIS? I would like to get in contact with these companies and let them know about the issues I ran into with their site.
|
| Posted by NeWeb, 10-28-2009, 04:21 PM |
| All files have the same extension?
All in the same directory?
Code or documents files?
|
| Posted by Secter, 10-28-2009, 04:24 PM |
| Configure your robots.txt file, password protect private files, block the bot IPs in IIS, or block the bot IPs in IPSEC.
|
| Posted by reddem0n, 10-28-2009, 09:24 PM |
| NeWeb, it could be multiple directories or even one directory. It could be a few .xls files or it could be .html files. Is there a set way to do it NeWeb?
Secter, I totally forgot about the robot.txt files. How the heck would you find IP addresses for Googlebot, yahoobot and all the other millions of bots? Do they all use static IP's?
|
| Posted by Scott.Mc, 10-29-2009, 12:54 AM |
| reddem0n there is lists published of most of the major ones (Search for cloaking) but you shouldn't need that. Just use robots.txt which all good search engines will honor.
|
Add to Favourites 
Print this Article
Also Read
