I seem to remember an early version of the "Guidelines for Robot Writers"
page discouraging depth first and I'm wondering why.
Would somebody please explain to me why it is "unfriendly" to implement
a depth first traversal. I know this effects the stack locally but I don't
quite appreciate how it effects the remote sites I'm visiting.
NOTE: I don't intend to write a robot I'm just interested in the area :-)