|
12-13-2003, 06:06 PM | #1 |
Green Mole
Join Date: Nov 2003
Posts: 5
|
Limit number of spidered pages
I would like to put a limit on the number of pages that are indexed, save on some space. LIMIT 15?
thanks, Robert |
12-14-2003, 03:27 PM | #2 |
Head Mole
Join Date: May 2003
Posts: 2,539
|
Hi. The below has not been tested. When I get a chance I can test it, but for now it gives you something to try.
In spider.php find the following: PHP Code:
PHP Code:
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
12-15-2003, 05:51 PM | #3 |
Green Mole
Join Date: Nov 2003
Posts: 5
|
Sorry to say that didn't work
I've tried to limit the number of times the tempspider is written but can't figure it out. Any ideas? Thanks for the help! |
12-16-2003, 11:18 AM | #4 |
Head Mole
Join Date: May 2003
Posts: 2,539
|
Hi. Please undo the above changes and then see if this thread helps.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
12-16-2003, 03:10 PM | #5 |
Green Mole
Join Date: Dec 2003
Location: Lyon, France
Posts: 17
|
Charter : A bug in you code =>
// Spidering ... $my_spider_limit = 0; while(($level <= $limit) && ($my_spider_limit < 15)) { $my_spider_limit++; You must change && by OR : // Spidering ... $my_spider_limit = 0; while(($level <= $limit) OR ($my_spider_limit < 15)) { $my_spider_limit++; Spider must stop when $level <= $limit OR when $my_spider_limit <15 Isn't it ? |
12-16-2003, 04:03 PM | #6 | |
Head Mole
Join Date: May 2003
Posts: 2,539
|
Quote:
Hi. Thanks, but in that code I wanted the 'and' in place. It was meant as when $level <= $limit and when $my_spider_limit < 15 do the loop, but it was the wrong piece of code to edit because all it does is basically limit the seach depth to fourteen without affecting the number of links found in a page. Silly me. Please see this thread instead.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
|
Thread Tools | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Number of sites locked/being spidered | CentaurAtlas | Mod Submissions | 0 | 12-09-2006 03:38 AM |
Show number of indexed pages on homepage | Freddie | Mod Submissions | 5 | 01-18-2005 05:36 PM |
Number of pages indexed | claudiomet | How-to Forum | 0 | 08-30-2004 03:26 PM |
Set time limit on spider.php or number of pages | paullind | Troubleshooting | 1 | 05-01-2004 08:25 AM |
pages number limited indexing | JÿGius³ | Mod Requests | 4 | 01-13-2004 12:17 PM |