PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 12-19-2004, 06:46 AM   #1
rafarspd
Orange Mole
 
rafarspd's Avatar
 
Join Date: Nov 2003
Location: High Wycombe, Bucks. UK
Posts: 35
Indexing a directory with many files

This old boy's brain has stopped functioning - HELP please.

Original version was 1.8.0.
PHPDIG was removed from the PHP area and the SQL data dropped.
I started afresh with V1.8.6.
My problem was only apparent after installation of the latest version.

I have a sub-directory which contains 79 *.txt files (2K to 13K each) which are sequentially numbered. [Newsletters, generated and uploaded monthly].
The search results only display 17 of them and it appears that they are not in number order but in the order stored on the ISP's HDD.

I have changed the following (as an experiment):

define('SPIDER_MAX_LIMIT',50);
define('RESPIDER_LIMIT',50);
define('LINKS_MAX_LIMIT',50);
define('RELINKS_LIMIT',50);

with no difference.

Obviously I am looking in the wrong area so any ideas!
__________________
Regards
Bob Finch
rafarspd is offline   Reply With Quote
Old 12-19-2004, 09:02 AM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Set "search depth" to a large number, set "links per" to zero, set LIMIT_TO_DIRECTORY to false.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 12-21-2004, 01:49 AM   #3
rafarspd
Orange Mole
 
rafarspd's Avatar
 
Join Date: Nov 2003
Location: High Wycombe, Bucks. UK
Posts: 35
Thanks, it help a bit but I now have to confess to the main reason for the problem.

I forgot a basic principle ?
If you want PHPDIG to find a file then it must have a URL somewhere pointing to it.

I had removed some references to older text files from a page.

Problem is now cured by adding the URL's (with no text) at the bottom of the html just above </body></html>.
They are not seen when viewing but PHPDIG finds and indexes them.
__________________
Regards
Bob Finch
rafarspd is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to limit indexing to a specific directory? Beans How-to Forum 1 08-15-2005 10:03 AM
How to index only local directory files? sf44 How-to Forum 0 01-28-2005 02:56 AM
Can only index files in a single directory gcrachel Troubleshooting 5 09-28-2004 06:23 AM
Not indexing my files Spider Troubleshooting 5 07-22-2004 05:12 PM
How to index a directory with pdf files simonced How-to Forum 3 02-13-2004 10:41 AM


All times are GMT -8. The time now is 05:26 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.