|
11-06-2005, 10:36 AM | #1 |
Orange Mole
Join Date: Aug 2005
Posts: 44
|
Less hosts than I put
Hi CHARTER,
I put 773 urls and after the spider stops without problems, phpdig finds just 588 host. I don't understand why, all the urls works well, all the sites exists. Thank you Noël |
11-06-2005, 02:08 PM | #2 |
Head Mole
Join Date: May 2003
Posts: 2,539
|
Maybe certain sites block spiders using a robots.txt or .htaccess file. Maybe the encoding could not be understood in the phpdigMakeUTF8 function. Maybe the status from the phpdigTestUrl function returned NOHOST, NOFILE, LOOP, or NEWHOST. Maybe something else.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
Thread Tools | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Maximum Hosts? | pgzn | The Mole Hole | 0 | 07-13-2006 04:35 PM |
Blocking particular hosts or domains? | cewyattjr | How-to Forum | 0 | 06-09-2006 12:49 PM |
FTP Put Error | cash4read | Troubleshooting | 6 | 05-28-2004 11:45 AM |
Limiting the search to certain hosts | lighthouse | Mod Requests | 1 | 03-11-2004 03:30 PM |
Put text_content in DB? | renehaentjens | Mod Requests | 0 | 11-27-2003 07:45 AM |