robots.txt
Posted: 7/12/1999 8:43:23 AMBy: Comfortably Anonymous
Times Read: 2,651
Likes: 0 Dislikes: 0
Topic: Test-Only
Make sure and add /books to robots.txt so that crawlbots do not suck down all the books in the library as seems to be happening right now!
Rating: (You must be logged in to vote)