# Don't want no search engines

**URL:** <https://forum.kirupa.com/t/dont-want-no-search-engines/133081>\
**Category:** Uncategorized\
**Created:** [January 5, 2005, 3:55pm UTC](https://forum.kirupa.com/t/dont-want-no-search-engines/133081 "2005-01-05T15:55:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![RabBell](https://yyz1.discourse-cdn.com/flex011/user_avatar/forum.kirupa.com/rabbell/32/340_2.png) [@RabBell](https://forum.kirupa.com/u/RabBell)\
**Post date:** [January 5, 2005, 3:55pm UTC](https://forum.kirupa.com/t/dont-want-no-search-engines/133081/1 "2005-01-05T15:55:17Z")

</div>

sorry this is probably in the wrong forum but I know several search engines use spider software to go through directories in your site and searches the content (I think google does this)

I have pages from old sites on my site and rather than delete them I’d rather have them not be searched. I understand theres something about adding a text file to a directory and calling it ROBOTS.txt or something

can anyone tell me more, like I am way off the mark, is this true, what should be in the file, etc.

thanks :thumb: