PhpDig.net - View Single Post

Charter · 12-30-2003, 02:23 AM

Hi. PhpDig tries to grab links from anchor, frame, javascript, and meta tags if the links match the regular expression in the phpdigExplore function and the links gets through the phpdigRewriteUrl function. As the inital page is small and the redirect page to the main site has a fair amount of javascript, you might try setting the CHUNK_SIZE in the config file to something fairly small as well. If this doesn't work, then you might want to bypass the intro pages and crawl http://www.ski-lesarcs.com/fr/index.php and http://www.ski-lesarcs.com/gb/index.php directly.

12-30-2003, 02:23 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. PhpDig tries to grab links from anchor, frame, javascript, and meta tags if the links match the regular expression in the phpdigExplore function and the links gets through the phpdigRewriteUrl function. As the inital page is small and the redirect page to the main site has a fair amount of javascript, you might try setting the CHUNK_SIZE in the config file to something fairly small as well. If this doesn't work, then you might want to bypass the intro pages and crawl http://www.ski-lesarcs.com/fr/index.php and http://www.ski-lesarcs.com/gb/index.php directly. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.