View Full Version : specific links not spider
flanders
10-06-2004, 11:02 PM
hi,
i have a problem! how i can say phpdig, that he dont spider specific links. with :
<!-- phpdigExclude --> and <!-- phpdigInclude -->
it dosnt work. he spider the "text" not, but the link :bang:
have anyone an idea?
Charter
10-06-2004, 11:12 PM
The phpdigExclude and phpdigInclude tags work like this:
<html>
<body>
<a href="http://www.this_link.com/is_followed.html">This text is indexed.</a>
This text is indexed.
<!-- phpdigExclude -->
<a href="http://www.this_link.com/is_followed.html">This text is ignored.</a>
This text is ignored.
<!-- phpdigInclude -->
<a href="http://www.this_link.com/is_followed.html">This text is indexed.</a>
This text is indexed.
</body>
</html>
Note that the phpdigExclude and phpdigInclude tags need to be on their own lines. If dynamically including a portion of a page, you may need to add a \n newline before and/or after the phpdigExclude and phpdigInclude tags, so that in the HTML source, the tags appear on their own lines.
To exclude specific links from being indexed, try using a robots.txt file, editing FORBIDDEN_EXTENSIONS in the config file, using a no index no follow META tag, or crawl the links and then exclude directories from the admin panel. The FORBIDDEN_EXTENSIONS option requires some regex knowledge.
vBulletin® v3.7.3, Copyright ©2000-2025, Jelsoft Enterprises Ltd.