View Single Post
Old 01-16-2007, 05:30 AM   #1
griemer
Green Mole
 
Join Date: Jan 2007
Posts: 1
Index some, but spider all pages

Hi all,
Have a Question:

The index page of the domain i want to spider, contains links to say level1.php?id=xx and level1.php?id=yy

level1.php?id=xx contains links to level2.php?id=aa and level2.php?id=bb

level1.php?id=yy contains links to level2.php?id=cc and level2.php?id=dd

The pages i want to index are the level2.php pages. But i only can reach them trough index -> level1.php -> level2.php.

So the spider needs to follow all the links from the root.

What would be the best way to index some, but spider all pages?
__________________
Regards,
Griemer
Onderdelenzoeker.nl
griemer is offline   Reply With Quote