PDA

View Full Version : index via shell


Nosmada
12-23-2003, 11:23 AM
I am timing out in the browser so I want to index via the shell. The script is located outside my cgi-bin and in the regular httpdocs folder.

I want to index 10 levels deep and have allready run it in the browser a couple of times and have a few directories indexed.

How do I do this? I read a similar question in the forum but it I didn't quite get it.

Nosmada
12-23-2003, 12:10 PM
By the way I tried the documentation and nothing happens when I run what is in the docs in my command prompt?

Charter
12-23-2003, 09:23 PM
Hi. In the config file change the following to ten:

define('SPIDER_MAX_LIMIT',10); //max recurse levels in sipder
define('SPIDER_DEFAULT_LIMIT',10); //default value
define('RESPIDER_LIMIT',10); //recurse limit for update

Then place the full URLs in a list.txt file, one per line.

Finally run the following command at the shell prompt, changing the paths and slashes as needed:

/path/to/php -f /path/to/spider.php /path/to/list.txt >> spider.log

The spider.log file will contain the indexing information.