Problem spidering sites at in .txt over 20 address

[joshuag200]

I have trouble spidering sites from a text file containing websites that has over a certain amount of address.

when I enter the following command with 300 sites in the file sites.txt

php -f spider.php sites.txt

I get
spidering in progress...

When I check only 50 websites are added as HOST and in the TEMP tables but not as entires.

When I spider the with only 20 sites in the file sites.txt it works fine.

how can I place huge amounts of sites in a .txt file and have all spidered

[Charter]

Hi. Do the URLs start with http and are they listed one per line in the text file? If so, how long do you wait before you check the tables?

[joshuag200]

Yes I have the address begin with http:// .....and one line for each address.

for 300 address I wait about 20 second then I check the tables

[Charter]

Hi. Not sure of the exact time frame with 300 sites, but try giving it a couple of hours, checking the tables intermittently, but don't stop PhpDig while checking.