Restrict Spidering to new pages only
 

My site has utilities for adding new content and once added the content remains static.

I would like to modfiy the publish scripts to spider the page that has just been uploaded.

Something along the lines of

$article_id = $sql_post_newarticle[aid];
$url_to_spider = "./view_article.php?aid=$article_id";
$spider_depth = 1;
$spiderresult = spider($url_to_spider, $spider_depth);

Is this possible with PHPDig?

Hi. Perhaps try this thread and then index only the new pages.