Go Back > PhpDig Forums > Bug Tracker

Thread Tools
Old 11-30-2004, 09:24 AM   #1
Green Mole
Join Date: Oct 2004
Posts: 3
I was experiencing the same problem as Ensim, namely an attempt to spider from the command line would always return "No link in temporary table". I took the time to trace spider.php, and found that setting LIMIT_TO_DIRECTORY to false in config.php solved the problem.

The code in question is at or about line 270 of spider.php (my line numbers may be a little off since I tidied the code up a bit to aid in reading). Specifically:

PHP Code:
    if (
$links_per_lev == 0) {
$query_tempspider "INSERT INTO ".PHPDIG_DB_PREFIX."tempspider (site_id,file,path) SELECT site_id,file,path FROM ".PHPDIG_DB_PREFIX."spider WHERE site_id=$site_id $andmore_tempspider";
    else {
$query_count_lev mysql_query("SELECT COUNT(*) as cnt FROM ".PHPDIG_DB_PREFIX."tempspider WHERE site_id = $site_id and level = 0",$id_connect);
$query_count_arr mysql_fetch_array($query_count_lev);
$query_count_num $query_count_arr['cnt'];
        if (
$query_count_num $links_per_lev) {
$level_lim $query_count_num $links_per_lev;
$query_tempspider "DELETE FROM ".PHPDIG_DB_PREFIX."tempspider WHERE level = 0 LIMIT $level_lim";
$flag_for_inserts_check1 1;
        elseif ((
$links_per_lev $query_count_num) &&
$flag_for_inserts_check1 == 0)) {
$level_lim $links_per_lev $query_count_num;
$query_tempspider "INSERT INTO ".PHPDIG_DB_PREFIX."tempspider (site_id,file,path) SELECT site_id,file,path FROM ".PHPDIG_DB_PREFIX."spider WHERE site_id=$site_id $andmore_tempspider LIMIT $level_lim";

It seems that if LIMIT_TO_DIRECTORY is set to true, the tempspider table is never populated and spidering never begins. Please correct me if I'm wrong, though, since I only studied it enough to get it working for me .

For the record, I'm running spider.php as follows:
php -f /path/to/my/site/dig/admin/spider.php all
I have a single site in the database with the site_url formatted '' (with the trailing slash)
indeh is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell command no indexing noel Troubleshooting 3 10-27-2005 11:22 AM
LIMIT_TO_DIRECTORY Rolandks How-to Forum 1 08-24-2004 05:32 AM
Shell Spidering CrazyCanuck Troubleshooting 3 04-20-2004 10:56 AM
Shell mode not working bloodjelly Troubleshooting 8 04-08-2004 05:43 PM
index via shell Nosmada How-to Forum 2 12-23-2003 10:23 PM

All times are GMT -8. The time now is 02:41 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.