> For now a few cron jobs could switch between a few robots.txt files during > appropriate times of the day.
How often is a robots.txt supposed to be checked? I generally only check it at the first request of the day.