My problem is deciding exactly *which* words are important to index, and
how to store such a huge amount of data in a manner that will be easily
accessible for a search engine.
Has anyone got any suggestions as to how to go about this? Should I maintain
a list of keywords which my spider will index, or should I index every single
word (including small ones such as if, the, and, but, etc...)?
I'm currently developing the application in C
=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=
David Reilly,
Computer Programmer, dodo@fan.net.au
http://www.fan.net.au/~dodo s1523@sand.it.bond.edu.au
=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=