View Full Version : Indexing outside root domain
Hello,
could anybody provide me the info how to allow the spider to jump domain names aka following all links.
Would be great if somebody could let me know how to do this :)
Charter
10-28-2003, 07:19 PM
Hi. Do you mean that your page has links to a third party domain that you'd lik to crawl?
yes, that's what i mean. I would like to be able to crawl third party sites too (I am aware of the high traffic / sql load this will cause)
jimigisme
10-31-2003, 08:20 AM
In the robots_function.php
Change phpdigCompareDomains($url1,$url2)
-
-
-
-
-
else {
return true;
Change the last line from False to True.....
Charter
03-14-2004, 02:57 PM
Hi. Also see this (http://www.phpdig.net/forum/showthread.php?postid=2879#post2879) post.
vBulletin® v3.7.3, Copyright ©2000-2025, Jelsoft Enterprises Ltd.