PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 11-06-2005, 09:36 AM   #1
noel
Orange Mole
 
Join Date: Aug 2005
Posts: 44
Less hosts than I put

Hi CHARTER,

I put 773 urls and after the spider stops without problems, phpdig finds just 588 host. I don't understand why, all the urls works well, all the sites exists.

Thank you

NoŽl
noel is offline   Reply With Quote
Old 11-06-2005, 01:08 PM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Maybe certain sites block spiders using a robots.txt or .htaccess file. Maybe the encoding could not be understood in the phpdigMakeUTF8 function. Maybe the status from the phpdigTestUrl function returned NOHOST, NOFILE, LOOP, or NEWHOST. Maybe something else.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Maximum Hosts? pgzn The Mole Hole 0 07-13-2006 03:35 PM
Blocking particular hosts or domains? cewyattjr How-to Forum 0 06-09-2006 11:49 AM
FTP Put Error cash4read Troubleshooting 6 05-28-2004 10:45 AM
Limiting the search to certain hosts lighthouse Mod Requests 1 03-11-2004 02:30 PM
Put text_content in DB? renehaentjens Mod Requests 0 11-27-2003 06:45 AM


All times are GMT -8. The time now is 06:30 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.