PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)
-   -   Less hosts than I put (http://www.phpdig.net/forum/showthread.php?t=2234)

noel 11-06-2005 09:36 AM

Less hosts than I put
 
Hi CHARTER,

I put 773 urls and after the spider stops without problems, phpdig finds just 588 host. I don't understand why, all the urls works well, all the sites exists.

Thank you

Noël

Charter 11-06-2005 01:08 PM

Maybe certain sites block spiders using a robots.txt or .htaccess file. Maybe the encoding could not be understood in the phpdigMakeUTF8 function. Maybe the status from the phpdigTestUrl function returned NOHOST, NOFILE, LOOP, or NEWHOST. Maybe something else.


All times are GMT -8. The time now is 04:04 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.