about robots.txt content errors

savron@world-net.sct.fr
Sat, 17 Feb 1996 10:15:19 +0000

Messages sorted by: [ date ][ thread ][ subject ][ author ]
Next message: joseph williams: "Re: Robots and search engines technical information."
Previous message: John D. Pritchard: "Re: URL measurement studies?"

Why not doing some sort of 'fuzzy' or 'loose' scan to compensate for
typos and others errors ?

Next message: joseph williams: "Re: Robots and search engines technical information."
Previous message: John D. Pritchard: "Re: URL measurement studies?"