How
do search engines differ from directories, announcement sites,
and guides on the Web?
Well a Search Engine is a service that uses
robots or spiders. What these programs do is continually "crawl"
the web looking for new or updated pages. A real surfers dream!
but they go from URL to URL until they have visited every site
on the Internet.
At the site, an agent records the full text
of the pages (home and sub pages) within the site. Sounds like
a lot of work but it don't stop there! it then go and searches
every external link and follows them to. This how the Search Engine
can find your site
Following these external links is
how search engines are able to find your site regardless of whether
or not you register your URL with them. Submitting your URL, however,
does speed up the process. It notifies an agent to visit and index
your site instead of waiting for it to eventually locate you through
one of your external links.
Robots will then revisit your site
periodically to refresh the recorded information. The revisiting
of links,if the find a dead link, their robot would try unsuccessfully
to update the information on a dead link and realize it no longer
exists.
The easy way to tell whether a
Web index is a search engine as opposed to another type of directory
is by the information it requires when adding your URL. A true
search engine will only need the Web address. The indexing agent
takes care of the rest.
