why is a temptable so big? (phpdigtempspider)
 

Hiho.

my phpdigtempspider table takes 4.6MB. Is that ok ore did something go wrong with the spider?

And my phpdigengine table takes 8.1MB and my phpdigkeywords only 2.8.

My computer crashed some times when indexing (hardware) so I'm not shure if something gone wrong.

Hi. Each keyword is associated with an id in the keywords table. These ids are associated with the text files in the engine table. With a large engine table, it seems like there are a lot of the same keywords showing up in different pages. The number of rows in the sites table is how many sites were crawled. The number of rows in the spider table is how many pages were crawled.

The tempspider may contain rows when the spidering process is stopped prematurely. These rows may be removed when the associated site is reindexed. The tempspider table may also contain rows even after a site has been indexed. The latter is something that needs further investigation. In the meantime, just make a backup of your database, and then afterwards empty the tempspider table if you want.