chris614164
02-08-2005, 02:15 PM
hello,
wow this engine is really great, my testpage is at yoohoo.lu, but now i have a problem, if i set define('LIMIT_TO_DIRECTORY',false); the spider finds all documents in the folder www.bighost.com/userfolder/ and the documents in www.bighost.com/userfolder/subfolder/ but then he starts to spider www.bighost.com/ which is not good cause there are thousends of "userfolders". If i set define('LIMIT_TO_DIRECTORY',true); the spider only goes to www.bighost.com/userfolder/ but he does not include www.bighost.com/userfolder/subfolder01/ or www.bighost.com/userfolder/subfolder02/ ... which is a problem ... any idea?
thx
chris
wow this engine is really great, my testpage is at yoohoo.lu, but now i have a problem, if i set define('LIMIT_TO_DIRECTORY',false); the spider finds all documents in the folder www.bighost.com/userfolder/ and the documents in www.bighost.com/userfolder/subfolder/ but then he starts to spider www.bighost.com/ which is not good cause there are thousends of "userfolders". If i set define('LIMIT_TO_DIRECTORY',true); the spider only goes to www.bighost.com/userfolder/ but he does not include www.bighost.com/userfolder/subfolder01/ or www.bighost.com/userfolder/subfolder02/ ... which is a problem ... any idea?
thx
chris