Search Engine Tips & Tricks: Create a Robots Text File for Your Web Site
Search engines index millions of web sites to generate the search results they return for key words. They do this using “spiders”.
Most search engines have their own spider that crawls around the web looking for web pages. Spiders are also known as “robots” because they are simply tiny little programs that run automatically, looking for […]
Written by on March 31st, 2007 with no comments.
Read more articles on Search Engine Optimization.