Hi Charter,
This still isn't clear to me. If you set a cron job to index URLs in a file list, once the spider has indexed the list, when the cronjob runs again what method will the spider use to index these URLs? Will it index all pages found regardless if the update date has not been reached or will it skip files that have been recently indexed and are not due to be index yet?
|