PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 10-23-2003, 07:00 AM   #1
DanBUK
Green Mole
 
Join Date: Oct 2003
Location: UK
Posts: 6
Angry ARGH - Indexing only ever gets 1 link! HELP

Hi,
I have installed phpdig onto my server, but i cannot seem to figure out why its not indexing.
I have checked mysql privs, file privs and looked at the HTTP headers that are sent, but they do look correct. All i get from any domain is the following:

Spidering in progress...
SITE : http://freebox.mine.nu/
Exclude paths :
- @NONE@
1:http://freebox.mine.nu/
(time : 00:00:00)
No link in temporary table
links found : 1
http://freebox.mine.nu/
Optimizing tables...
Indexing complete ! [Back] to admin interface.

Help

Regards,
Daniel.
__________________
--------------------
DanBUK
dan@lockedbox.net
http://dans.lockedbox.net
http://livecd.lockedbox.net
DanBUK is offline   Reply With Quote
Old 10-23-2003, 08:48 AM   #2
Rolandks
Purple Mole
 
Rolandks's Avatar
 
Join Date: Sep 2003
Location: Kassel, Germany
Posts: 119
Re: ARGH - Indexing only ever gets 1 link! HELP

Quote:
Originally posted by DanBUK
... my server ...
And now we all must advise what operating System and which PHP Version you use on "your server"

Please post OS and PHP-Version - it is important !
Rolandks is offline   Reply With Quote
Old 10-23-2003, 08:57 AM   #3
DanBUK
Green Mole
 
Join Date: Oct 2003
Location: UK
Posts: 6
Sorry my foobar

Sorry I havent slept in 36 hours...

Running:
Vanilla Kernel 2.4.22
Apache 2.0.47
MySQL 4.0.13-r4
PHP 4.3.3-r2

I have also added the l_time FLOAT into the db.
And rectified the \n -> \r\n for the headers

Cheers,
Dan.
__________________
--------------------
DanBUK
dan@lockedbox.net
http://dans.lockedbox.net
http://livecd.lockedbox.net
DanBUK is offline   Reply With Quote
Old 10-24-2003, 07:08 AM   #4
DanBUK
Green Mole
 
Join Date: Oct 2003
Location: UK
Posts: 6
Also

Also, I have testing on many different domains. The ones that are hosted on the same box as the phpdig will due to my bind setup be told the internal IP not the global one, so thats not causing it.
Im really confused as to why its not working...
__________________
--------------------
DanBUK
dan@lockedbox.net
http://dans.lockedbox.net
http://livecd.lockedbox.net
DanBUK is offline   Reply With Quote
Old 10-24-2003, 04:23 PM   #5
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Does anything in this thread help?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 10-25-2003, 02:51 AM   #6
DanBUK
Green Mole
 
Join Date: Oct 2003
Location: UK
Posts: 6
Nope

I'd allready read that one, that was why I pointed out that Bind(DNS Server) has two "views", if an external ip askes about my domain, it gets the 'Real' ip, if an internal ip asks about the domain it get the internal 192.168.x.x address.
I have tried setting PHPDIG_DEFAULT_INDEX as well.

I'm lost!
__________________
--------------------
DanBUK
dan@lockedbox.net
http://dans.lockedbox.net
http://livecd.lockedbox.net
DanBUK is offline   Reply With Quote
Old 11-12-2003, 08:16 AM   #7
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Oops, this thread got lost.

Has anything changed?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 11-12-2003, 08:23 AM   #8
DanBUK
Green Mole
 
Join Date: Oct 2003
Location: UK
Posts: 6
Nope

No change, ive had a bit of a fiddle further to no joy.

I ended up installing htdig instead.
__________________
--------------------
DanBUK
dan@lockedbox.net
http://dans.lockedbox.net
http://livecd.lockedbox.net
DanBUK is offline   Reply With Quote
Old 11-12-2003, 08:41 AM   #9
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. I can't see to duplicate the problem. I crawled your site at level one and found nine links. Did anything in this thread help?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 11-13-2003, 03:36 AM   #10
schade
Green Mole
 
Join Date: Nov 2003
Posts: 5
Same problem here

Same problem here

Im running PHP Version 4.3.2
BUT Server API is CGI (Could this be a problem?)
And running on an AIX.

The output is:

Spidering in progress...

--------------------------------------------------------------------------------
SITE : http://www.kvis.org/
Exclude paths :
- @NONE@
No link in temporary table

--------------------------------------------------------------------------------

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

And I've testet tons of sizes - same result.

Can I turn some kind of debugging on?
schade is offline   Reply With Quote
Old 11-14-2003, 05:22 PM   #11
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. There seems to be a problem with PhpDig and CGI mode, but I currently don't have access to PHP in CGI mode. If someone out there with PHP in CGI mode, who is having problems like those posted above, can offer access, then I could try to locate the problem.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 11-15-2003, 02:24 PM   #12
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. To schade: I purchased a hosting account that runs PHP in CGI mode, and I also set an open_basedir restriction. I was able to crawl several sites using this account without incident, except for your site where I received the same results as you did. Now I no longer think that there is a CGI mode problem. Rather, it seems the PhpDig problem is related to something site specific. Are you able to setup a plain demo page without JavaScript on your account and crawl it?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 11-19-2003, 01:32 AM   #13
schade
Green Mole
 
Join Date: Nov 2003
Posts: 5
Hi,

I just created a simple test-page

http://www.kvis.org/test/

But I still have the same problem.
schade is offline   Reply With Quote
Old 11-19-2003, 05:04 AM   #14
schade
Green Mole
 
Join Date: Nov 2003
Posts: 5
I've been digging into robot_functions.php and found the reason to my errror fsockopen() fails.

This small program demonstrates the error:

<?php
$fp = fsockopen ("www.schade.dk", 80, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br>\n";
} else {
fputs ($fp, "GET / HTTP/1.0\r\nHost: www.schade.dk \n\r\n");
while (!feof($fp)) {
echo fgets ($fp,128);
}
fclose ($fp);
}
?>


Returning this errormessage:

----

Warning: fsockopen(): php_network_getaddresses: getaddrinfo failed: System error returned in errno (is your IPV6 configuration correct? If this error happens all the time, try reconfiguring PHP using --disable-ipv6 option to configure) in /home/www/php/test.php on line 2

Warning: fsockopen(): unable to connect to www.schade.dk:80 in /home/www/php/test.php on line 2
No such file or directory (2)

----

hmmm, searching the net I found:

http://bugs.php.net/bug.php?id=11058

... thats all for now, but I'll keep digging :-)
schade is offline   Reply With Quote
Old 11-19-2003, 08:15 AM   #15
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. When I run your snippet, I get the following output:
Code:
HTTP/1.1 200 OK
Date: Wed, 19 Nov 2003 17:12:24 GMT
Server: Apache/1.3.27 (Unix)  (Red-Hat/Linux) Chili!Soft-ASP/3.6.2
        FrontPage/4.0.4.3 mod_auth_pgsql/0.9.12
Last-Modified: Tue, 11 Nov 2003 13:23:10 GMT
ETag: "1ba40ca-1177-3fb0e2be"
Accept-Ranges: bytes
Content-Length: 4471
Connection: close
Content-Type: text/html
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
again : Indexing problems : no link flloweb sylvain Troubleshooting 1 10-26-2005 10:33 PM
Full Link Exploration with Selective Content Indexing Xavian How-to Forum 8 10-11-2004 08:37 PM
Too many duplicate link, someone help please! warrence Troubleshooting 1 09-07-2004 04:26 PM
more indexing problems: No link in temporary table boomboom100 Troubleshooting 9 03-23-2004 09:33 AM
no link found... again moutonrebelle Troubleshooting 3 12-30-2003 03:12 AM


All times are GMT -8. The time now is 01:34 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.