PDA

View Full Version : Crawl mutiple urls


Jeff
02-03-2004, 12:28 PM
I would like to crawl more then one url at a time.

Can Dig do this?

Jeff

bloodjelly
02-03-2004, 03:13 PM
You can run two instances of the script from the shell (see http://www.phpdig.net/navigation.php?action=doc#toc8) or you could open two browser windows and crawl a different site in each.

Jeff
02-03-2004, 03:47 PM
Well i could write a macro too, but i was woundering if it could do a lot more then the one or two, more on the order of 2 - 3 hundred.

David J Harmon
02-03-2004, 06:47 PM
I've uses the 2-4 browser way, but I would like to see if you could make a macro and do more...

Jeff
02-03-2004, 07:59 PM
creat a file called url.txt on your desktop, have the urls line spaced

http//www.msn.com
http://dogpile.com
so on
so on


let me know how it works out

it will tie up the pc your on, that's why i don;'t want this way

jeff