Ok just a quick note for those not aware of what a robots.txt file is.....

Baically robots.txt is a text file a webmaster places in the root directory of his site which instructs some search engines as to what areas of the site he is ok for them to index.

Q. Do I have to have a robots.txt file to get ranked on a search engine??
No you do not - if no robots.txt file is present the search engine or spider will just assume that it is ok for it to index your whole site

Q. Will a robots.txt file help my site get looked at by more search engines??
A. No - in order to view the file they must already be at your site

Q. So what do I need it for then??
A. Well it allows you to designate certain areas which you do not wish to be indexed by the search engine - perhaps you have a test folder where you simply try out new things that you dont want the public to be aware of etc

Q. Ok so how do I make one then?
A. Easy just open a txt editor like notepad and type the following

User-agent: *
Disallow: /test/

you can add as many dir to this as you wish

Q. Ok have written my file but how do i know it is working?
A. simply upload it into the root dir of your site and that should be it if you wish to check the syntax - and make sure everything is correct you can visit here
or for more info on robots.txt try here

thanks

v_Ln