How long?

[sjSJ]

I have installed PhpDig on my winxp box running Apache .
I have installed a robots.txt in the root dir.
When trying to spider my site I get a list of all the directories which are off limits and the URL becomes locked.

How long will "Spidering" take ? Must be less than 500kb of html pages. Should this take minutes or longer?

Regards
Steve

[vinyl-junkie]

Welcome to the forum, sjSJ. We're glad you joined us. :D

At the risk of stating the obvious, the length of time it takes to spider will be dependent on how many pages there are to spider. ;) That said, it really should take only a few seconds per page.

Is nothing happening at all for you when the spider starts? You should be getting displays every few seconds of the each page that phpdig is processing.

It's possible you don't have your folder permissions set correctly. Check the documentation to see if that might be the problem. Another possibility is that the spidering simply got hung. That happened to me the first time I tried to spider one of my sites. However, I cleared all the tables, started the spider again, and everything turned out okay.

Hopefully, one of these suggestions will be the key. Let us know if you still have problems. We'll be glad to help.