PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 05-11-2004, 04:07 AM   #1
sjSJ
Green Mole
 
Join Date: May 2004
Posts: 1
How long?

I have installed PhpDig on my winxp box running Apache .
I have installed a robots.txt in the root dir.
When trying to spider my site I get a list of all the directories which are off limits and the URL becomes locked.

How long will "Spidering" take ? Must be less than 500kb of html pages. Should this take minutes or longer?

Regards
Steve
sjSJ is offline   Reply With Quote
Old 05-11-2004, 06:09 PM   #2
vinyl-junkie
Purple Mole
 
Join Date: Jan 2004
Posts: 694
Welcome to the forum, sjSJ. We're glad you joined us.

At the risk of stating the obvious, the length of time it takes to spider will be dependent on how many pages there are to spider. That said, it really should take only a few seconds per page.

Is nothing happening at all for you when the spider starts? You should be getting displays every few seconds of the each page that phpdig is processing.

It's possible you don't have your folder permissions set correctly. Check the documentation to see if that might be the problem. Another possibility is that the spidering simply got hung. That happened to me the first time I tried to spider one of my sites. However, I cleared all the tables, started the spider again, and everything turned out okay.

Hopefully, one of these suggestions will be the key. Let us know if you still have problems. We'll be glad to help.
vinyl-junkie is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Long Time Spidering Crashes mikiwiki Troubleshooting 1 10-26-2006 09:25 AM
Long Call help. GunMuse How-to Forum 0 07-26-2006 10:11 AM
Long post, possible issues, and maybe the data to solve some? Paul D. Buck Troubleshooting 4 02-03-2005 11:59 AM
PDFs in directory listed as one long entry motopsycho External Binaries 4 03-09-2004 05:07 PM


All times are GMT -8. The time now is 01:06 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.