tricky indexing problem
 

hello,

wow this engine is really great, my testpage is at yoohoo.lu, but now i have a problem, if i set define('LIMIT_TO_DIRECTORY',false); the spider finds all documents in the folder www.bighost.com/userfolder/ and the documents in www.bighost.com/userfolder/subfolder/ but then he starts to spider www.bighost.com/ which is not good cause there are thousends of "userfolders". If i set define('LIMIT_TO_DIRECTORY',true); the spider only goes to www.bighost.com/userfolder/ but he does not include www.bighost.com/userfolder/subfolder01/ or www.bighost.com/userfolder/subfolder02/ ... which is a problem ... any idea?

thx

chris

ups i had not read this http://www.phpdig.net/forum/showthread.php?t=1749 it helped me :)