PhpDig.net - index via shell

PhpDig.net (http://www.phpdig.net/forum/index.php)

- How-to Forum (http://www.phpdig.net/forum/forumdisplay.php?f=33)

- - index via shell (http://www.phpdig.net/forum/showthread.php?t=326)

I am timing out in the browser so I want to index via the shell. The script is located outside my cgi-bin and in the regular httpdocs folder.

I want to index 10 levels deep and have allready run it in the browser a couple of times and have a few directories indexed.

How do I do this? I read a similar question in the forum but it I didn't quite get it.

By the way I tried the documentation and nothing happens when I run what is in the docs in my command prompt?

Hi. In the config file change the following to ten:

PHP Code:


		
			
define('SPIDER_MAX_LIMIT',10);          //max recurse levels in sipder

define('SPIDER_DEFAULT_LIMIT',10);      //default value

define('RESPIDER_LIMIT',10);            //recurse limit for update

Then place the full URLs in a list.txt file, one per line.

Finally run the following command at the shell prompt, changing the paths and slashes as needed:

Code:

/path/to/php -f /path/to/spider.php /path/to/list.txt >> spider.log

The spider.log file will contain the indexing information.