PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 08-03-2004, 06:12 AM   #1
lighthouse
Green Mole
 
Join Date: Mar 2004
Posts: 20
Command line spider ignoring "filelist.txt"

I can run a command line spider by doing the following:

>> php - f search.php
The above indexes everything...

However - I want to just update a few sites, so I have a file called "filelist.txt" which contains 2 URLS's (each on a diffent line)...

When I do the following:

>>php - f search.php filelist.txt
It makes no difference... phpdig proceeds to update everything!!!!

Any ideaS?...
I need to resolve this so i can setup a cron....

Also - FYI:
>> php -f search.php http://www.anydomain.com
This works as expected... It's just with specifying a filelist that there is problems.

Thanks in advance
LH
lighthouse is offline   Reply With Quote
Old 08-03-2004, 11:29 AM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. What version of PhpDig are you using?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 08-09-2004, 03:31 AM   #3
lighthouse
Green Mole
 
Join Date: Mar 2004
Posts: 20
1.6.x (as far as i can see - taken from "spider.php" file)
lighthouse is offline   Reply With Quote
Old 08-09-2004, 05:08 AM   #4
lighthouse
Green Mole
 
Join Date: Mar 2004
Posts: 20
Quote:
Originally posted by lighthouse
1.6.x (as far as i can see - taken from "spider.php" file)

Update - I upgraded to latest version, and STILL it ignores the filelist.txt and simply begins indexing everything!!!

Please help!!
LH
lighthouse is offline   Reply With Quote
Old 08-10-2004, 05:09 AM   #5
lighthouse
Green Mole
 
Join Date: Mar 2004
Posts: 20
Update:
Have tried and tried with this, but no luck.

I've opted to just run 6 crons (instead of 1), in the format:
php - f path/to/dir/spider.php http://theurliwanttoindex.com

Not pretty, but it works....
Still open to any suggestions if you have any...

LH
lighthouse is offline   Reply With Quote
Old 08-15-2004, 03:03 PM   #6
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Try updating to 1.8.3.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 08-16-2004, 05:32 AM   #7
lighthouse
Green Mole
 
Join Date: Mar 2004
Posts: 20
Quote:
Originally posted by Charter
Hi. Try updating to 1.8.3.

Charter - I did upgrade - and still the same result....
lighthouse is offline   Reply With Quote
Old 08-17-2004, 02:27 PM   #8
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Is the tempspider table empty?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 08-18-2004, 01:01 AM   #9
lighthouse
Green Mole
 
Join Date: Mar 2004
Posts: 20
That seems to have done the trick!!!!
Does the Temporary table always need to be empty then?...
Only I see that it's often got entries in it...

LH
lighthouse is offline   Reply With Quote
Old 08-18-2004, 07:35 AM   #10
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. The tempspider table should try and empty after PhpDig is done, but if it doesn't (for example, the spidering process prematurely ends) then just go and empty that table.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
shows blank page if "Search All" and "exact phrase", timeout? alokjain9 Troubleshooting 2 03-07-2006 07:08 AM
"search depth" and "links per" features laurentxav How-to Forum 1 01-12-2005 07:27 PM
"from" line in email hollycrj Coding & Tutorials 2 06-29-2004 12:34 PM
PhpDig Ignoring Something in robots.txt Destroyer X Troubleshooting 2 06-18-2004 01:57 PM


All times are GMT -8. The time now is 09:05 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.