Search Engine Indexer

After a Search Engine spider or robot has found, downloaded and stored the page another program, called an indexer, processes the page using its own proprietary algorithm.
A search engine indexer will carve the page up into its various components removing all the HTML tags and store links in a queue. The contents of the meta keyword tag, description tag, titles, headings, links, body copy, bold, and italic are analysed. Some indexers compress the page by removing stop words; finally the much cut down page is stored in an online searchable database.
The indexer uses a complex mathematical formula or algorithm to analyse and weight key areas of a page. Every indexer uses a unique algorithm and these algorithms are constantly being modified and bettered by the search engines to filter out Spam and to ensure that the most relevant pages get better rankings.
Because of the unique nature of these algorithms a page 1 listing in Lycos does not guarantee a similar listing in Google or any other search engine.