View Full Version : indexing from command line with text file

Wayne McBryde
01-08-2004, 05:26 PM
I can reindex from the command line with no problem, but it will only reindex the sites allready in the database.
From shell and in the phpdig-admin folder, I enter:
"php -f spider.php cronlist.txt"
It will reindex the sites allready in the database but not the sites in the text file.
If I enter:
"php -f spider.php http://www.domain.com"
I get the same result.

PHP ver is 4.1.2

I also tried to setup the cron job but it's not working. However I would perfet to use the command line.

01-09-2004, 03:54 AM
Hi. When you run php -f spider.php http://www.newdomain.com what output does PhpDig give?

Wayne McBryde
01-10-2004, 05:04 PM
The atached file (graphic) shows the command and output. When I enter php -f spider.php http://www.150cam.com
If I enter:
php -f spider.php all
php -f spider.php text_file.txt
php -f spider.php
php -f spider.php http://www.150cam.com
The result is the same. It will reindex the sites in the database but thats all.

If the graphic does not show up, it is at lakenormansweb.com/phpdig_index.jpg

Also Charter, I'm sorry I put this post in the wrong form.

Thanks for your help.

01-11-2004, 12:47 PM
Hi. Are you able to index that site via the browser interface? A test run via shell gave the following output. Curious. Maybe this is related to the FP issue (http://www.phpdig.net/showthread.php?threadid=190) experienced previously?

> php -f spider.php http://www.150cam.com
Spidering in progress...
SITE : http://www.150cam.com/
Exclude paths :
- @NONE@
(time : 00:00:03)
level 1...
XDuplicate of an existing document
(time : 00:00:05)

No link in temporary table
links found : 2
Optimizing tables...
Indexing complete !

Don't worry about posting in the wrong forum. I've moved things around to differentiate between troubleshooting and how-tos. ;)

Wayne McBryde
01-11-2004, 04:52 PM
No problems from the browser interface. This site and many more spidered on my main site. I have phpdig installed on 2 servers, 1 for production and 1 for testing. Spidering from the command line has the same problem on both. Both also have Front Page installed. I'll try to setup a site w/o Front Page and see what happens. Any ideas about what to try if Front Page does apear to be the problem?

Wayne McBryde
01-11-2004, 06:27 PM
I setup a new site with a new domain name. I did not install Front Page. It has the same problem as the site with Front Page installed.

01-11-2004, 06:31 PM
Hi. Are you getting any errors in your PHP logs when you try and index via shell? Also, can you post a link to your PHP info?

01-11-2004, 07:35 PM
Hi. Try setting register_argc_argv to On in the php.ini file.

Wayne McBryde
01-12-2004, 05:56 PM

Changing register_argc_argv to On in the php.ini file did the trick.