PhpDig.net - Different search depths for different cron jobs

PhpDig.net (http://www.phpdig.net/forum/index.php)

- How-to Forum (http://www.phpdig.net/forum/forumdisplay.php?f=33)

- - Different search depths for different cron jobs (http://www.phpdig.net/forum/showthread.php?t=746)

lighthouse

03-30-2004 01:50 AM

Different search depths for different cron jobs

I want to setup cron jobs to do the following:

1) spider a set of URLS every 7 days to a search depth of 2
2) spider a set of URLS every 28 days to a search depth of 5

Can this be done?... Can you specify the search depth paramater in the cron job?...

Also - can I spider 1 particular page of a website (the news page) without spidering the rest of the site (I'm pretty sure this is possible, but just want to confirm!!)

Thanks in advance
Gary

Charter

03-30-2004 08:15 AM

Hi. Perhaps try the following.

In spider.php find:

PHP Code:


		
			
if ($argc > 1) {

and afterwards put:

PHP Code:


		
			
$limit = argv[2];

Usage: php -f spider.php [option] [number]
where option and number must be specified.

All times are GMT -8. The time now is 10:45 PM.