renehaentjens
02-12-2004, 02:58 AM
I managed to get indexing/spidering into a loop. In a very small branch of the site, with no links to elsewhere, it kept repeating the same URL over and over again, with a red cross in front and "Was recently indexed" at the end.
When I stopped the process, the site was locked (of course), and MySQL gave error 145 on table tempspider: cant open file tempspider.MYI.
I had to delete the table and create it back with phpMyAdmin.
As I have long URLs in the site, I first thought that this might have caused the problem. Why, by the way, are fields 'file' and 'path' limited to 127 chars in the spider table? That is not going to be enough for my site! Can't they be TEXT fields like in tempspider?
Anyway, my long URLs are not yet long enough, I have around 50-60 characters in path and file currently.
So, something else must have been the cause of this looping.
When I stopped the process, the site was locked (of course), and MySQL gave error 145 on table tempspider: cant open file tempspider.MYI.
I had to delete the table and create it back with phpMyAdmin.
As I have long URLs in the site, I first thought that this might have caused the problem. Why, by the way, are fields 'file' and 'path' limited to 127 chars in the spider table? That is not going to be enough for my site! Can't they be TEXT fields like in tempspider?
Anyway, my long URLs are not yet long enough, I have around 50-60 characters in path and file currently.
So, something else must have been the cause of this looping.