If not a FAQ:
How does an indexing robot decide what to index?
If an indexing robot knows about a document, it may decide to parse it,
and insert it into its database. How this is done depends on the robot:
Some robots index the HTML Titles, or the firs few paragraphs, or parse
the entire HTML and index all words, with weightings depending on
HTML constructs, etc. Some parse the META tag, or other special hidden tags.
In other words, it depends :-)
> I want to know like I could create the html files in way that the
> robot links the pages in relation to the keywords choices from me.
Which the robot may or may not want you to do, as you might put in
irrelevant words to have your page come up under more searches...
-- Martijn
Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html