Command line spider ignoring "filelist.txt"

[lighthouse]

I can run a command line spider by doing the following:

>> php - f search.php
The above indexes everything...

However - I want to just update a few sites, so I have a file called "filelist.txt" which contains 2 URLS's (each on a diffent line)...

When I do the following:

>>php - f search.php filelist.txt
It makes no difference... phpdig proceeds to update everything!!!!

Any ideaS?...
I need to resolve this so i can setup a cron....

Also - FYI:
>> php -f search.php http://www.anydomain.com
This works as expected... It's just with specifying a filelist that there is problems.

Thanks in advance
LH

[Charter]

Hi. What version of PhpDig are you using?

[lighthouse]

1.6.x (as far as i can see - taken from "spider.php" file)

[lighthouse]

Originally posted by lighthouse
1.6.x (as far as i can see - taken from "spider.php" file)


Update - I upgraded to latest version, and STILL it ignores the filelist.txt and simply begins indexing everything!!!

Please help!!
LH

[lighthouse]

Update:
Have tried and tried with this, but no luck.

I've opted to just run 6 crons (instead of 1), in the format:
php - f path/to/dir/spider.php http://theurliwanttoindex.com

Not pretty, but it works....
Still open to any suggestions if you have any...

LH

[Charter]

Hi. Try updating to 1.8.3.

[lighthouse]

Originally posted by Charter
Hi. Try updating to 1.8.3.


Charter - I did upgrade - and still the same result....
:(

[Charter]

Is the tempspider table empty?

[lighthouse]

That seems to have done the trick!!!! :)
Does the Temporary table always need to be empty then?...
Only I see that it's often got entries in it...

LH

[Charter]

Hi. The tempspider table should try and empty after PhpDig is done, but if it doesn't (for example, the spidering process prematurely ends) then just go and empty that table.