command line and ABSOLUTE_SCRIPT_PATH

[drywall]

I've had phpdig up and running on my site for a while, but I've gotten tired of manully spidering, so I decided to look into setting up cron -- which largely appears to be really easy, bravo!

The problem I'm running into is that php spits out a bunch of include warnings when I try to run from the command line... I've no idea why, though I'm pretty sure I've got it set up properly.

My config.php looks like:


define('ABSOLUTE_SCRIPT_PATH','/Library/WebServer/Documents/tools/phpdig');

if ((!isset($relative_script_path)) || (($relative_script_path != ".") &&
($relative_script_path != "tools/phpdig") && ($relative_script_path != "..") && ($relative_script_path != ABSOLUTE_SCRIPT_PATH)))

My search.php isn't located in my phpdig directory, obviously. Anyway, these settings have everything on the website working well. But when I try the following from the command line [yep, I know php's in a slightly unusual place]:


/usr/local/php/bin/php -f /Library/WebServer/Documents/tools/phpdig/admin/spider.php http://www.mydomain.com


I get the following output:


PHP Warning: main(../includes/config.php): failed to open stream: No such file or directory in /Library/WebServer/Documents/tools/phpdig/admin/spider.php on line 91
PHP Warning: main(): Failed opening '../includes/config.php' for inclusion (include_path='.:/usr/local/php/lib/php') in /Library/WebServer/Documents/tools/phpdig/admin/spider.php on line 91
PHP Notice: Use of undefined constant USE_RENICE_COMMAND - assumed 'USE_RENICE_COMMAND' in /Library/WebServer/Documents/tools/phpdig/admin/spider.php on line 93
PHP Warning: main(../admin/robot_functions.php): failed to open stream: No such file or directory in /Library/WebServer/Documents/tools/phpdig/admin/spider.php on line 112
PHP Warning: main(): Failed opening '../admin/robot_functions.php' for inclusion (include_path='.:/usr/local/php/lib/php') in /Library/WebServer/Documents/tools/phpdig/admin/spider.php on line 112
PHP Fatal error: Call to undefined function: phpdigprnmsg() in /Library/WebServer/Documents/tools/phpdig/admin/spider.php on line 128


Which seem like the kind of messages I might get if I had ABSOLUTE_SCRIPT_PATH incorrectly set. But I've looked at it a million times and retyped it and I swear it's correct. Any idea what I'm doing wrong here?

Thanks for your help.

[Charter]

The output is saying that PHP cannot find various files for inclusion WRT this location:

/Library/WebServer/Documents/tools/phpdig/admin/spider.php

As you seem to have a different directory structure, try setting the following (it's a guess):

define('ABSOLUTE_SCRIPT_PATH','/Library/WebServer/Documents/tools/phpdig');

[drywall]

Okay, so you got me... I was actually trying to obscure a little bit the actual path I'm using (just paranoia on my part about disclosing full directory structures). Made the change in what I posted as config but not in the php warnings. But the cat's out of the bag, so I might was well confess... *actual* config.php:


define('ABSOLUTE_SCRIPT_PATH','/Library/WebServer/Documents/tools/phpdig');

if ((!isset($relative_script_path)) || (($relative_script_path != ".") &&
($relative_script_path != "tools/phpdig") && ($relative_script_path != "..") && ($relative_script_path != ABSOLUTE_SCRIPT_PATH)))


And *actual* command I'm using is:


/usr/local/php/bin/php -f /Library/WebServer/Documents/tools/phpdig/admin/spider.php http://www.mydomain.com


Sorry to be misleading and by doing so waste anyone's time. I'm an idiot.

[Charter]

OIC, ha ha, anyway... where are the config.php and robot_functions.php files relative to the following:

/Library/WebServer/Documents/tools/phpdig/admin/spider.php

[drywall]

The usual...

/Library/WebServer/Documents/tools/phpdig/admin/spider.php
/Library/WebServer/Documents/tools/phpdig/includes/config.php
/Library/WebServer/Documents/tools/phpdig/admin/robot_functions.php

So they seem so to be where they're supposed to be...

[Charter]

Are you sure "/Library/WebServer/..." is the full path? Maybe you have something like "/dir1/dir2/Library/WebServer/..." for the full path? Try the following script. Does it echo "/Library/WebServer/..." or something else?

<?php
echo $_SERVER['DOCUMENT_ROOT']."<br>";
echo $_SERVER['SCRIPT_FILENAME']."<br>";
echo $_SERVER['PATH_TRANSLATED']."<br>";
?>

[drywall]

Yep, I'm pretty darn sure (I'm basically co-sysadmin for the machine, so I know the dir structure pretty well). The above script outputs:

/Library/WebServer/Documents
/Library/WebServer/Documents/tools/phpdig/test.php
/Library/WebServer/Documents/tools/phpdig/test.php

[Charter]

The only other thing I can think of is that it's a permission/ownership/group issue, where it runs from the web interface but not from shell. Check that the PhpDig files have "-rw-r--r--" permission and see if /usr/local/php/bin/php is set with "-rwxr-xr-x" permission. The crontab executable can have "-rws--x--x" permission.

Also, I did think about the php.ini file, perhaps changing:

include_path = .:/usr/local/php/lib/php

To something like the following:

include_path = ./:/usr/local/php/lib/php:/Library/WebServer/Documents/tools/phpdig/admin:/Library/WebServer/Documents/tools/phpdig/includes

But that doesn't really seem to be the problem, as you can run PhpDig from the web interface, so it's back to a permission/ownership/group issue.

[drywall]

Those both sounded like very good, promising ideas.

Unfortunately, my perssions were already set to those values, so that wasn't the problem.

Edited my php.ini just to give that a try, and it didn't change anything at all -- still get the same errors.

If I didn't know any better, I'd say that it seems like from the command line spider.php is still trying to use $relative_script_path instead of falling back on the ABSOLUTE_SCRIPT_PATH. That's the only way I can figure it'd work fine from the website but not from the command line... but I could be way off on that one.

[Charter]

The $relative_script_path is used throughout the script, but it should be relative to wherever you are in the script. The ABSOLUTE_SCRIPT_PATH is so you can call PhpDig from shell while being outside the admin directory, but once called, the $relative_script_path should be page relative... unless... what's your "doc_root" set to in your php.ini file?

[drywall]

I had to check and... It's not set to anything.

[Charter]

Hmm, is your /usr/local/php/bin/php running under user root?

[drywall]

I believe so... it's running as whatever the default is -- pretty sure that's root (but I have to admit I don't know how to check or change that). /usr/local/php/bin/php is owned by root....

[Charter]

What's a link to your PHP info?

[drywall]

Don't wanna be a jerk, but there's some sensitive info on that page... (namely PHP_AUTH_PW) but PHP_AUTH_USER is "xxxxxx" -- anything in particular you want to see on that page?

[Charter]

Nothing particular, but without seeing the page, I'm out of ideas. As the page has sensitive information, of course, don't post a link.

[drywall]

I moved the page to a non-cookie area...

http://www.freepress.net/phpinfo.php

Thanks for all the help on this.

[Charter]

Ho hum... nothing stands out, so more thoughts. First try doing the following echo and see what it prints for full path:

<?php
echo __FILE__; // two underscores FILE two underscores
?>

Maybe there is a hidden "/Local" that should start "/Local/Library/WebServer..." or PHP wants "/usr/local/etc/httpd..." and not "/Local/Library/WebServer..." or something is set for "DocumentRoot" in the httpd.conf file.

Maybe change the owner of the PhpDig files: sudo chown -R [user]:[group] [file]

Maybe give up and hard code the full path where needed in the PhpDig files.

Maybe finally figure out next year what one flag needed to be changed. ;)

[drywall]

No luck on on the __FILE__ approach. I played with permissions before but maybe will do so again. Sigh... thanks for the help, though.

Out of curiosity, how hard would it be to modify spider.php (or create a new file altogether) that took info as GET arguments, so a cron job could just be set up with a "curl"? Something like...


curl http://www.mydomain.com/phpdig/spider_curl.php?d=www.mydomain.com&depth=6


(Or something like that). Seems like that might be a reasonably easy workaround, but I dunno...

[Charter]

http://www.phpdig.net/forum/showthread.php?t=1538 :confused:

[drywall]

I'll check that out, it may be the workaround I need.

FYI, I just did a clean install of 1.8.7 and the problem persists (with very slightly different messages from PHP). So I don't think it's the result of anything I may have changed anywhere...

[Charter]

I did a Google search to figure out WTF, and while I didn't see a solution for your OS/setup, what I did find is that there were several instances of people on Mac having the same 'relative' problem. Who knows...

[drywall]

Wow, so you think it's platform-specific? Weird -- I never would have guessed that. Maybe Apple hid a weird document root thing somewhere in the Apache files that I need to find.

[drywall]

Looks like DocumentRoot is set -- both for the virtual host and in general to /Library/WebServer/Documents
ServerRoot is "/Library/WebServer"
Nothing too unusual there, I don't think...

[Charter]

Searched again: check this (http://aldoblog.com/blog/395) out. Permissions, permissions...

[drywall]

Took a look at the link, thatnks for digging that up. At first I was excited because I thought you might have found it, but the message is mostly about when trying to do includes off of a user directory... and I'm not in a user directory. But I took the advice to heart, went bonaza and changed permissions to 766 on "phpdig" and every dir within that. Also changed it on robot_functions.php and config.php since those seemed to be the offending parties. Still no luck.

I just noticed something weird though. Here's the first (of several) errors I get when running from the command line.

PHP Warning: main(): Failed opening '../includes/config.php' for inclusion (include_path='.:/php/includes
:/Library/WebServer/Documents/includes
:/Library/WebServer/Documents/admin/phpdig/admin
:/Library/WebServer/Documents/admin/phpdig/includes') in /Library/WebServer/Documents/admin/phpdig/admin/spider.php on line 96

The thing is, though, I reset my include path back to just :/usr/lib/php ! How is it that my command line php could have a different include path than when through Apache? Might this lead us to something...?

Very confused...

[Charter]

There's "DocumentRoot (http://httpd.apache.org/docs/mod/core.html#documentroot)" in httpd.conf (Apache) and there's "include_path (http://www.php.net/manual/en/ini.sect.path-directory.php#ini.include-path)" in php.ini (PHP) and there's all the rest of the settings. What's your DocumentRoot set to?

[drywall]

DocumentRoot /Library/WebServer/Documents

[drywall]

I decided I'm tired of whatever this weird problem is. So I went into spider.php and changed line 94 from


$relative_script_path = '..';


to


$relative_script_path = '/Library/WebServer/Documents/admin/phpdig';


and that seems to have gotten everything working. Still dunno what was wrong with '..' since it seems like it should, but whatever.

Thanks for all your help. This was driving me nuts too.

[Charter]

What happens if you comment out the $relative_script_path = '..'; line, the one that appears before the USE_RENICE_COMMAND constant?

[olivier]

Same problem here with FreeBSD 4.9

su-2.05b# /usr/local/bin/php -f /usr/local/www/data/phpdig/admin/spider.php /var/cron/phpdig.txt >> /var/log/spider.log
PHP Warning: main(../includes/config.php): failed to open stream: No such file or directory in /usr/local/www/data-dist/phpdig/admin/spider.php on line 96
PHP Warning: main(): Failed opening '../includes/config.php' for inclusion (include_path='.:') in /usr/local/www/data-dist/phpdig/admin/spider.php on line 96
PHP Notice: Use of undefined constant USE_RENICE_COMMAND - assumed 'USE_RENICE_COMMAND' in /usr/local/www/data-dist/phpdig/admin/spider.php on line 98
PHP Warning: main(../admin/robot_functions.php): failed to open stream: No such file or directory in /usr/local/www/data-dist/phpdig/admin/spider.php on line 115
PHP Warning: main(): Failed opening '../admin/robot_functions.php' for inclusion (include_path='.:') in /usr/local/www/data-dist/phpdig/admin/spider.php on line 115
PHP Fatal error: Call to undefined function: phpdigprnmsg() in /usr/local/www/data-dist/phpdig/admin/spider.php on line 131


I used the same trick and it is working now... Thanks Drywall and Charter!

Olivier.