Hi. The below is untested, but you might try making the following changes in the spider.php file. Of course, another alternative is to avoid crawling such sites or use a search depth of zero or one.
PHP Code:
$sum_of_tempfilesize = 0;
// Spidering ...
while(($level <= $limit) && ($sum_of_tempfilesize <= X)) {
// $tempfilesize is text filesize, not the actual page size
// set X to be the maximum number of bytes allowed
...
$sum_of_tempfilesize = $sum_of_tempfilesize + $tempfilesize;
//Retrieve meta-tags for this page
...
// clean the tempspider table
$query = "DELETE FROM ".PHPDIG_DB_PREFIX."tempspider WHERE site_id=$site_id";