Spider Directories

[clearwater]

I just installed PhpDig...am impressed with its search speed, but have one problem. It spidered only two of my subdirectories, and when I try to spider these, I see the message that this domain was spidered recently.

http://mydomain.com - root - was spidered
http://mydomain.com/directory1&2 were spidered....

but http://mydomain.com/directory3,4 etc. were not spidered. How do I accomplish this?

Thanks for your time.

[Charter]

Hi. Are there links from mydomain.com to directories three and four? Do you get the message that the site was recently indexed? If so, you might try changing define('LIMIT_DAYS',7); to define('LIMIT_DAYS',1); in the config file.

[clearwater]

Hi Charter,

Thanks for your prompt response.

Yes, I'm getting the message the site was recently indexed, so I changed define('LIMIT_DAYS',7); to define('LIMIT_DAYS',1); in the config file....still same result...site was recently indexed.

But you bring up an interesting point...many of these subdirectories are linked from the home page, but only through a dhtml/javascript menu navigation. Perhaps that is part of my problem?

[Charter]

Ah, Javascript. Perhaps this thread (http://www.phpdig.net/showthread.php?threadid=86) might help.

Also, if you want to try reindexing from scratch, you might find this thread (http://www.phpdig.net/showthread.php?threadid=78) useful.

You could also try setting define('LIMIT_DAYS',1); to define('LIMIT_DAYS',0);

[clearwater]

Thanks again for the help.

I was able to spider the remaining directories, except for one that does not have an html link. I could just create the link temporarily if it is that important.

Thanks for the good utility! - it's nice to be off of the web-based, outsourced search. :)