PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)
-   -   Not spidering mod rewritten pages (http://www.phpdig.net/forum/showthread.php?t=2381)

wineo 02-22-2006 07:38 PM

Not spidering mod rewritten pages
 
I have two sub domains for my site and phpdig spiders the one nicely (including mod rewrites). BUT, it is not spidering further than this new page on the other sub domain. It will not go deeper than the page http://www.wineography.com/wine_reviews.htm . Here is the report:

Code:

SITE : http://www.wineography.com/
Exclude paths :
- @NONE@
- cellar_wine_search.htm
- member_details.htm
Duplicate of an existing document
1:http://www.wineography.com/wine_reviews.htm
(time : 00:00:05)
No link in temporary table
links found : 1
http://www.wineography.com/wine_reviews.htm
Optimizing tables...
Indexing complete !

Any ideas on how to force it?!

wineo 02-22-2006 07:53 PM

I physically indexed one of the mod pages () and it managed to index it and found one other, but still did not continue further. I had the link depth set to 20 and the limit set to 0...

SITE : http://www.wineography.com/
Exclude paths :
- @NONE@
- cellar_wine_search.htm
- member_details.htm
1:http://www.wineography.com/USA/country/212
(time : 00:00:06)
+
level 1...
2:http://www.wineography.com/top_wineries.htm
(time : 00:00:19)

No link in temporary table
links found : 2
http://www.wineography.com/USA/country/212
http://www.wineography.com/top_wineries.htm
Optimizing tables...
Indexing complete !


All times are GMT -8. The time now is 01:35 PM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.