PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 12-09-2003, 03:53 PM   #1
fr :: anonymus
Green Mole
 
fr :: anonymus's Avatar
 
Join Date: Dec 2003
Location: Lyon, France
Posts: 17
When a very lot of links...

Hi,

When there are a very lot of links (most than 3000) in the same page, spider record all, and scan them.
But it's very long between 2 links :

17:http://www.*****.net/docs/php/annote...properties.php
(temps : 00:49:41)

18:http://www..*****.net/docs/php/annot...es-by-list.php
(temps : 00:54:05)

19:http://www..*****.net/docs/php/annot...-by-regexp.php
(temps : 00:57:36)


and time, in (at) the end, is very important. (3-5 days ?)

Do you know how to make it faster, without mode localhost ?

Thanks, Anonymus.
fr :: anonymus is offline   Reply With Quote
Old 12-09-2003, 03:54 PM   #2
fr :: anonymus
Green Mole
 
fr :: anonymus's Avatar
 
Join Date: Dec 2003
Location: Lyon, France
Posts: 17
**** = nexen.
fr :: anonymus is offline   Reply With Quote
Old 12-10-2003, 07:51 AM   #3
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Are you indexing http://www.nexen.net/docs/php/annotee/indexes.functions.php at level one? If not already done, perhaps try to be selective when indexing a site by choosing the most relevant page to index at level one. Also, if available, perhaps try to index from shell rather than with the browser interface. If indexing from shell, the following can be set in the config file to limit the level to one.
PHP Code:
define('SPIDER_MAX_LIMIT',1);           //max recurse levels in sipder
define('SPIDER_DEFAULT_LIMIT',1);       //default value
define('RESPIDER_LIMIT',1);             //recurse limit for update 
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 12-10-2003, 12:38 PM   #4
fr :: anonymus
Green Mole
 
fr :: anonymus's Avatar
 
Join Date: Dec 2003
Location: Lyon, France
Posts: 17
That's for ..../php/annotee/indexes.functions.php

All are in level 1.

Ok, thanks.

Anonymus.
fr :: anonymus is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Phpdig find a lot of 404 errors Danydan Troubleshooting 3 12-18-2005 01:11 AM


All times are GMT -8. The time now is 09:01 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.