PDA

View Full Version : specific links not spider


flanders
10-06-2004, 11:02 PM
hi,

i have a problem! how i can say phpdig, that he dont spider specific links. with :
<!-- phpdigExclude --> and <!-- phpdigInclude -->

it dosnt work. he spider the "text" not, but the link :bang:

have anyone an idea?

Charter
10-06-2004, 11:12 PM
The phpdigExclude and phpdigInclude tags work like this:

<html>
<body>
<a href="http://www.this_link.com/is_followed.html">This text is indexed.</a>
This text is indexed.
<!-- phpdigExclude -->
<a href="http://www.this_link.com/is_followed.html">This text is ignored.</a>
This text is ignored.
<!-- phpdigInclude -->
<a href="http://www.this_link.com/is_followed.html">This text is indexed.</a>
This text is indexed.
</body>
</html>

Note that the phpdigExclude and phpdigInclude tags need to be on their own lines. If dynamically including a portion of a page, you may need to add a \n newline before and/or after the phpdigExclude and phpdigInclude tags, so that in the HTML source, the tags appear on their own lines.

To exclude specific links from being indexed, try using a robots.txt file, editing FORBIDDEN_EXTENSIONS in the config file, using a no index no follow META tag, or crawl the links and then exclude directories from the admin panel. The FORBIDDEN_EXTENSIONS option requires some regex knowledge.