Hi. Between runs, check that the tempspider table is empty. If it's not empty, then empty it. You can do this by clicking the delete button from the admin panel
without selecting a site, or run the following query:
Code:
DELETE FROM tempspider;
Sometimes things can get left in the tempspider table when there is no error but the corresponding page hasn't been indexed. This can happen if the spidering process is terminated prematurely.