PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > External Binaries

Reply
 
Thread Tools
Old 08-12-2004, 12:26 PM   #16
rom
Green Mole
 
Join Date: Jan 2004
Posts: 25
Hi Charter,

I read through the memory thread. Looked up my memory_limit, which is 10 M. Tried also this code for memory_get_usage from the php.net site:

<?php
// This is only an example, the numbers below will
// differ depending on your system
echo memory_get_usage() . "\n"; // 36640
$a = str_repeat("Hello", 4242);
echo memory_get_usage() . "\n"; // 57960
unset($a);
echo memory_get_usage() . "\n"; // 36744
?>

My server returned this:
16704 38000 16784

I know now, based on when the spidering ends, that it is getting hung up on one 4.6 M pdf.

From the memory thread, I wasn't sure what else to do, because at the end of the thread Tomas says nothing worked. Is there something that can be done to skip over that one file?

Thanks again.

Rom
rom is offline   Reply With Quote
Old 08-15-2004, 03:15 PM   #17
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Did you try something like in this post?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 08-25-2004, 04:05 PM   #18
rom
Green Mole
 
Join Date: Jan 2004
Posts: 25
hi charter,

tried your suggestion above. the indexing just stops part way through. seems to be when it encounters a 4.6 M file. it doesn't want to skip over it.

thanks,

rom
rom is offline   Reply With Quote
Old 08-25-2004, 07:10 PM   #19
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Did you try this too?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 08-26-2004, 09:02 AM   #20
rom
Green Mole
 
Join Date: Jan 2004
Posts: 25
Hi Charter,

Tried that also, but again it stops part way through indexing, when it reaches the 4.6 M file.

Thanks,

Rom
rom is offline   Reply With Quote
Old 08-26-2004, 10:21 AM   #21
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Assuming you are using 1.8.3, try moving this code:
PHP Code:
if (memory_get_usage() + 1000000 3000000) {
    return array(
'tempfile'=>0,'tempfilesize'=>0);

to be right after the following in the robot_functions.php file:
PHP Code:
// $file_content = @file($uri); ///////////////////////////////////////////////// 
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 08-26-2004, 01:55 PM   #22
rom
Green Mole
 
Join Date: Jan 2004
Posts: 25
i'm using 1.8.0. should i upgrade first?
rom is offline   Reply With Quote
Old 08-27-2004, 04:11 PM   #23
rom
Green Mole
 
Join Date: Jan 2004
Posts: 25
I tried moving the lines as directed. Still stops indexing part way through at the same spot.
rom is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
PDF indexing Probelm (pdftotext) ripchen External Binaries 9 10-20-2005 11:14 AM
Can phpdig index Japanese PDF file??? mynamesucks External Binaries 3 02-22-2005 09:59 PM
can phpdig index PDF server-side served from php? Sybolt How-to Forum 1 02-18-2005 12:16 PM
not indexing with pdftotext davideyre External Binaries 2 03-30-2004 12:55 PM
How to index a directory with pdf files simonced How-to Forum 3 02-13-2004 10:41 AM


All times are GMT -8. The time now is 09:05 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.