View Single Post
Old 04-22-2004, 10:09 AM   #2
marb
Green Mole
 
Join Date: Mar 2004
Posts: 19
I have install spider serveral times, in different dirs use one db.
Think this the fasted way to run more than one spider.
Have spider more than 19000 urls in 3 weeks, take 1 gb traffic.
More/to much spiders make the server slow.
Search at the same, during spidering with more than one spider at the same time, results show can come slow also.
I have experience, use more than 5 spider at the same time the script hang.
I could save the db, trough copy to a other db, but must reinstall the script and delete the old db.

Marten
marb is offline   Reply With Quote