View Single Post
Old 05-29-2007, 04:11 AM   #1
histrenact
Green Mole
 
Join Date: May 2007
Posts: 3
Upgrade and Update

Hi all,

Been looking at this site for ages but only really needed to post now. Have been using phpdig almost since the beginning but have been running 1.6.2 for about two years.

A bit about the site. I am one of the reference sites for phpdig and currently have approximately 500 sites indexed comprising just over twelve million index entries and just short of one and a half million keywords. The database is approximately 300Mb running on Linux Slackware 11, Apache 2.2.3 and Mysql 5.1.11 and php 5.2.0 all is fine with the current installation.

I have a couple of questions

1, Due to the fact that this is very much a production environment, the main website that I run which uses the search engine gets about a million hits a year or so, I tend to be very careful about updates. So I have a test envorinment which I am currently using to create the procedure to update the production site.

Running al of the update_to_xxx.sql from 1.6.2 upwards it appears that 1.8.6 and the most recent update_db.sql both have ALTER TABLE excludes DROP INDEX ex_id statements. Is this in error? After running 1.8.6 the index should have been removed?

2, I have seen reference here to the ability to run multiple spiders. I appreciate that this can be done with several lists of urls and several PCs so that the text areas do not clash but I have also seen reference to wrapper but the link to the download sesms to have gone. Can anyone help with the link?

3, MySQL NDB..... I am trying to get the database to an NDB cluster. Does anyone have any experience of this? Getting NDB working is no problem but with 12,000,000 three field records in the engine table, for example, is this the best way forwards? I get the impression from the documentation that illions of very small records may not be the best for using NDB, especially as the reason is to increase the search speed. Any suggestions for further "research" on my part would be very useful.

Finally, I do have all my own kit here, servers, software, complete control &c so if there is anything that *I* can do to help please feel free to ask.

David D.
The Historical Reenactment Web Site
http://www.histrenact.co.uk/
histrenact is offline   Reply With Quote