MyBBThemes Community

Here's a guide to not getting indexed in major search engines, and to remove yourself from these sites.

Why would you want this?
Some people make personal sites for a small group of people, and do not want their site being indexed, and their valuable work all over the internet.

Okay, so let's say you do not want any search engine to index any part of your site. The first part is to create a file called

Quote:robots.txt

Then put the following in that file:


All pages won't get indexed. Now, let's say you only want the search engine to not index only one page. Then you must put the following in the robots.txt page.


This means those 3 directories will not be indexed by any search engine, good for work you want to show off to people, but don't want it lying around on the web.


Now let's say you do not want to have only a particular search engine to index your site, like Google, or Yahoo. Then use the following, and substitute "bot" with the search engine of your choice.



Now, here's the opposite, if you want to only have one search engine index your site, use the following and substitute "bot"


That's all, hope that helps :thumb1:

Thanks Killz, don't know why you wouldn't want your website indexed, but for those who don't, this will come in handy, great tutorial

Like I said, people with personal sites that don't want thier work all over the internet. For example, my personal site has family pictures on it, so why would I want those all over the search engines? There merely there to show to other friends and family.