PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)
-   -   Index some, but spider all pages (http://www.phpdig.net/forum/showthread.php?t=2740)

griemer 01-16-2007 05:30 AM

Index some, but spider all pages
 
Hi all,
Have a Question:

The index page of the domain i want to spider, contains links to say level1.php?id=xx and level1.php?id=yy

level1.php?id=xx contains links to level2.php?id=aa and level2.php?id=bb

level1.php?id=yy contains links to level2.php?id=cc and level2.php?id=dd

The pages i want to index are the level2.php pages. But i only can reach them trough index -> level1.php -> level2.php.

So the spider needs to follow all the links from the root.

What would be the best way to index some, but spider all pages?


All times are GMT -8. The time now is 02:50 PM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.