I just installed PhpDig...am impressed with its search speed, but have one problem. It spidered only two of my subdirectories, and when I try to spider these, I see the message that this domain was spidered recently.
http://mydomain.com - root - was spidered
http://mydomain.com/directory1&2 were spidered....
but
http://mydomain.com/directory3,4 etc. were not spidered. How do I accomplish this?
Thanks for your time.