PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 11-20-2006, 02:18 AM   #1
lentilsoup
Green Mole
 
Join Date: Nov 2006
Posts: 2
Unhappy Spidering not completing

I used PHPDig on my local machine and successfully spidered and searched my website. I then uploaded this code to a webserver and it stopped working properly.

At first it would not find anything. On 'digging' a new site it would say it had completed indexing very quickly and then on return to the admin page it would say 1 host and zero for everything else (keywords etc..) in the statistics. After fiddling around I eventually indexed some pages, but not all. I'm sure I didn't really change anything. It seems a little sporadic.

So the problem is: the spider doesn't seem to be following all links as it has only found around 9 pages whereas the site has around 100 pages. I change nothing since last night and now it won't find any links again.

Here's a checklist of things I have done based on previous posts and/or the documentation:

- 'safe-mode' is off: Check
- 'allow_url_fopen' is on: Check
- Site is linked up entirely by simple <a href> tags: Check
- Tried the DEFINES as follows:
define('ABSOLUTE_SCRIPT_PATH',correct with no trailing slash);
define('LIMIT_TO_DIRECTORY',false);
define('PHPDIG_SESSID_REMOVE',true);
define('PHPDIG_IN_DOMAIN',true);
Check
- The three directories that require it are CHMOD 777: Check
- Tried the $pu['port'] = ''; bugfix - both as '' and '0': Check

Could this be something to do with the commercial server I am running this on? Or is it more likely the code?
lentilsoup is offline   Reply With Quote
Old 11-20-2006, 03:25 AM   #2
Dave A
Purple Mole
 
Dave A's Avatar
 
Join Date: Aug 2004
Location: North Island New Zealand
Posts: 170
This could be a problem with the way the new sever balances it loading, some odd things have turned up depending on how the laod balancing is set up
Dave A is offline   Reply With Quote
Old 11-20-2006, 09:24 AM   #3
lentilsoup
Green Mole
 
Join Date: Nov 2006
Posts: 2
Is there any way around this if I have no control over the server myself?
lentilsoup is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT -8. The time now is 08:42 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.