robots.txt syntax
Fred K. Lenherr (lenherr@tiac.net)
Fri, 11 Oct 1996 07:15:17 -0400
Is there a way to allow access to a subdirectory, while disallowing
everything else (short of exhausively listing every file)?
E.g, something like:
Disallow: /
Allow: /public/