Topic: Test-Only robots.txt Posted: 7/12/1999 1:43:23 PM By: Guest Times Read: 2,616 Likes: 0 Dislikes: 0 Topic: Test-OnlyMake sure and add /books to robots.txt so that crawlbots do not suck down all the books in the library as seems to be happening right now! Rating: (You must be logged in to vote) Reply MessageBase Terms and Conditions
