|
Hi. Is http://www.mansfield-tx.gov/ the domain you are trying to crawl? If so, the redirect has JavaScript in the middle and end of the HTML so try changing define('CHUNK_SIZE',2048); to define('CHUNK_SIZE',200); in the config file. Does this change make it pick up the links?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
|