http://www.addedbytes.com/article/robots-txt-file/Latest comments on robots.txt File on AddedBytes.comhttp://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Ami Daniel ( <a href=""></a> )<br /><br />Great Article!<br /> Simple, understandable and practical. <br /> Thanks!http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Muffy ( <a href=""></a> )<br /><br />I just made a site with some amazon ads on it which are iframes (and they don't validate). Do they hurt my site ranking? And if so how can I use a robot.txt file to tell them not to index the iframe? <br /> <br /> Sorry if this is a dumb question I am a total newbie.http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Christian ( <a href="http://christian.project-groove.com">http://christian.project-groove.com</a> )<br /><br />Thanks for this article. Now I'm ready to make my own robots.txt. Just because I'm curious: can someone answer the question of brent?http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by johan ( <a href="http://telefonix.co.uk">http://telefonix.co.uk</a> )<br /><br />caren, google ".htaccess rippers". It's another file you put in your root, it stops bad boys ripping off your site.http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Brent Lagerman ( <a href="http://www.mimoymima.com">http://www.mimoymima.com</a> )<br /><br />first off great site, I've stumbled upon you many times in the past and have always found great info here. <br /> <br />secondly can you help me with this problem: <br /> <br />I'm trying to disallow a directory /clients/ but allow only one file in the directory /clients/index.php <br /> <br />is this possible with <br /> <br />allow /clients/index.php <br />disallow /clients <br /> <br />does it matter the order or will this just not work at all. <br /> <br />brent <br />@ <br /><a href="http://www.mimoymima.com">mimoymima.com</a>http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Caren ( <a href=""></a> )<br /><br />I have had people stealing from and downloading my entire website. I have exact copies of it with a few minor changes and them claiming it as thier own. I have put a great amount of time and money into creating it. I am looking for something to lock the pages or encrpt them so they can not be stolen. I understand that if you encrpt the html with a program that your page or website can not be crawled or index. Is this true? Any suggestions on how to protect my work and stil be indexed so my site can be seen? Are there any programs that you would suggest? Thanks!http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Julian ( <a href="http://www.toporganicdiet.com">http://www.toporganicdiet.com</a> )<br /><br />I instructed robot.txt to disallow indexing my website, but google still picked up and indexed all the pages even src and images forbidden directories. Did anybody have the same experience?http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Anonymous ( <a href=""></a> )<br /><br />can i know where to put robot.txt?http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by James Ryddel ( <a href=""></a> )<br /><br />Fix that link Dave. <br /> <br />http://www.addedbytes.com/seo/robots-txt-file/ <br /> <br />That's a free tip :0)http://www.addedbytes.com/article/robots-txt-file/comments/http://www.addedbytes.com/article/robots-txt-file/comments/Comment by Haris khan ( <a href=""></a> )<br /><br />This article really helps me in writing robots.txt file.