Quote:
Originally posted by -IAN-
It is seven, I can change it to zero. I looked in the website directory (like the page you linked recommended ex:http://www.foobar.com/robots.txt) and there wasnt a robots.txt file. Should there be one?
|
Not necessarily. It's really only needed if you want portions of your site not to be spidered.
Quote:
|
Note: I just retried it with the LIMIT_DAYS set to zero, and I still got the same result.....
|
What search depth did you choose? If you left the default at zero, only the root will be indexed. That might possibly be the problem.