Edomondo
11-30-2005, 12:38 AM
Hi there,
I have a cron that refresh my index every night.
Last morning, the hosting company contacted me because the log file was growing 0.5 Go every half-hour.
I realized the cron launched the day before was still active.
So when the current day cron was trying to index a site locked by the first cron, the log was returning an error message, causing the file to grow dramatically.
I'd like to know what can I do to stop any spidering before launching a new one. Should I include the link to stop_spider.php?stop=1 on the top of spider.php?
Or do you know of other way to do so?
I have a cron that refresh my index every night.
Last morning, the hosting company contacted me because the log file was growing 0.5 Go every half-hour.
I realized the cron launched the day before was still active.
So when the current day cron was trying to index a site locked by the first cron, the log was returning an error message, causing the file to grow dramatically.
I'd like to know what can I do to stop any spidering before launching a new one. Should I include the link to stop_spider.php?stop=1 on the top of spider.php?
Or do you know of other way to do so?