View Single Post
Old 04-01-2004, 11:48 PM   #5
jerrywin5
Orange Mole
 
Join Date: Mar 2004
Posts: 48
Hi Charter,

This still isn't clear to me. If you set a cron job to index URLs in a file list, once the spider has indexed the list, when the cronjob runs again what method will the spider use to index these URLs? Will it index all pages found regardless if the update date has not been reached or will it skip files that have been recently indexed and are not due to be index yet?
jerrywin5 is offline   Reply With Quote