PhpDig is set to crawl links from one site using the admin panel. By indexing from shell a list of URLs can be specified, one per line in a text file. To crawl links from site to external site, set PHPDIG_IN_DOMAIN to true in the config file and apply the code change in this thread.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
|