Crawl mutiple urls
 

I would like to crawl more then one url at a time.

Can Dig do this?

Jeff

You can run two instances of the script from the shell (see http://www.phpdig.net/navigation.php?action=doc#toc8) or you could open two browser windows and crawl a different site in each.

Well i could write a macro too, but i was woundering if it could do a lot more then the one or two, more on the order of 2 - 3 hundred.

I've uses the 2-4 browser way, but I would like to see if you could make a macro and do more...

creat a file called url.txt on your desktop, have the urls line spaced

http//www.msn.com
http://dogpile.com
so on
so on


let me know how it works out

it will tie up the pc your on, that's why i don;'t want this way

jeff