PhpDig.net - View Single Post - Spider External links to a depth of 1 (1.8.3)

kenazo · 10-20-2004, 06:28 AM

I just upgraded to 1.8.3 and even after reading a ton of posts before (1.6.3) and again now I can't seem to get a clear answer on this question.

Basically what I want to do is spider external links from the spidered domain to a depth of one. The page I want to spider is basically a collection of links pointing to other pages. I want to crawl that page, find all those links, and follow them but only spider the page the link points to. I don't want to pick up external links of that linked page as then the spider just keeps going on and on.

Is this possible?

Thanks

10-20-2004, 06:28 AM	#1
kenazo Green Mole Join Date: Mar 2004 Posts: 5	Spider External links to a depth of 1 (1.8.3) I just upgraded to 1.8.3 and even after reading a ton of posts before (1.6.3) and again now I can't seem to get a clear answer on this question. Basically what I want to do is spider external links from the spidered domain to a depth of one. The page I want to spider is basically a collection of links pointing to other pages. I want to crawl that page, find all those links, and follow them but only spider the page the link points to. I don't want to pick up external links of that linked page as then the spider just keeps going on and on. Is this possible? Thanks