PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   How-to Forum (http://www.phpdig.net/forum/forumdisplay.php?f=33)
-   -   Selective Indexing of URL Containing a <keyword> (http://www.phpdig.net/forum/showthread.php?t=2849)

Leith 01-21-2008 02:16 AM

Selective Indexing of URL Containing a <keyword>
 
Hi Forum,

I'm trying to customize phpDig to do the following:

- I want to spider all links in a particular domain.

- Only specific URL's should be indexed, especially URL's containing the keyword - <rest of URL>"showproperty"<rest of URL>.

- My first attempt was to modify the "phpdigRewriteURL" function inside the "robot_functions.php" API, in the following way:
Code:
if (!eregi("showproperty", $eval)) {
return -1;
}
so that any URL containing the keyword is not indexed. This, however, results in all links not being followed, and only links containing the keyword...

- My second attempt involves playing with the "$ok_for_index" parameter in the "spider.php" API:
//let's go for indexing the content
if ($ok_for_index == 1) {
...
so that the "$ok_for_index" is only set to 1 if the URL being indexed contains the "showproperty" keyword.

My question is; am I going in the correct direction with this, or is there a more optimal approach?

Thanks,
Leith


All times are GMT -8. The time now is 05:00 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.