Selective Indexing of URL Containing a <keyword>
Hi Forum,
I'm trying to customize phpDig to do the following: - I want to spider all links in a particular domain. - Only specific URL's should be indexed, especially URL's containing the keyword - <rest of URL>"showproperty"<rest of URL>. - My first attempt was to modify the "phpdigRewriteURL" function inside the "robot_functions.php" API, in the following way: Code: if (!eregi("showproperty", $eval)) { return -1; } so that any URL containing the keyword is not indexed. This, however, results in all links not being followed, and only links containing the keyword... - My second attempt involves playing with the "$ok_for_index" parameter in the "spider.php" API: //let's go for indexing the content if ($ok_for_index == 1) { ... so that the "$ok_for_index" is only set to 1 if the URL being indexed contains the "showproperty" keyword. My question is; am I going in the correct direction with this, or is there a more optimal approach? Thanks, Leith |
All times are GMT -8. The time now is 05:00 AM. |
Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.