View Single Post
Old 07-08-2004, 12:13 PM   #5
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
>> I also started a spider with the exec() command, which ran for a while and then stopped with links still in the temporary table and without unlocking. Most likely this is because I didn't set links per for this spider?

Hi. Hmm, not sure on this one. In spider.php is the following:
PHP Code:
if (!isset($linksper) or (int)$linksper LINKS_MAX_LIMIT) {
 if (
$run_mode != 'cgi') {
    
$linksper RELINKS_LIMIT;
 }
 else {
    
$linksper LINKS_MAX_LIMIT;
 }

In 1.8.1 the links per is either set via the browser interface or by values in the config file. One of the new tables has links per for each site, but utilizing this table didn't get done for version 1.8.1, so for now, links per will be the same for all sites crawled via shell.

Anyway, back to your exec issue, I'm not sure why the spider quit. Maybe r****m noise, maybe not. Anything in your error logs?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote