PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 07-20-2004, 01:00 AM   #1
e_talis
Green Mole
 
Join Date: Jul 2004
Posts: 5
Question Spidering returns very few keyword results

Hi all,
I am using phpDig 1.6.1 on my site. At first when I installed it it did a good spider and returned many keywords. Later when I deleted some link entries and tried to respider the site, the number of keywords became very few and now when someone does a search most words return "No Result" but the words were returning many results before I went on the deleting spree. I have tried to clear the whole database to see if I can remove any Logs but it stiill gives few or no results. What can I do to get back my many keywords?
__________________
You Digg!?
e_talis is offline   Reply With Quote
Old 07-20-2004, 01:32 PM   #2
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
Hi e_talis -

First, it couldn't hurt to upgrade to 1.8.3 before you get started again, as it offers lots of new features and a few bug fixes.

As for increasing the number of keywords found, try 1) setting the search depth higher, and 2) making sure all your pages are linked to on your site so that phpDig can find them. You might want to set up a site map with links to the main parts of your site. Just make sure phpDig can find a link to your site map. Good luck.
bloodjelly is offline   Reply With Quote
Old 07-20-2004, 09:43 PM   #3
e_talis
Green Mole
 
Join Date: Jul 2004
Posts: 5
Hi bloodjelly,

Okay, I will upgrade by the end of the day and see how 1.8.3 works. I am using a search depth of 5, which was working quite fine when I first installed the mole. I will try and test it with 10 and see how things work out. About the site map, the spidering is detecting all the pages+links in my site but it is not picking up enough keywords from the pages it finds (it was in the begining before I started deleting links).
Thanks.
__________________
You Digg!?
e_talis is offline   Reply With Quote
Old 07-21-2004, 01:58 AM   #4
e_talis
Green Mole
 
Join Date: Jul 2004
Posts: 5
1.8.3 slower that 1.6.1?

bloodyjelly,
I have installed 1.8.3 as you had suggested...
I still get the same proble and it is spidering at a very slow rate. Any suggestions on how to boost it up?
__________________
You Digg!?
e_talis is offline   Reply With Quote
Old 07-21-2004, 08:54 AM   #5
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
Hmm...as for the spidering being slow, check out this thread: http://www.phpdig.net/showthread.php?s=&threadid=1066

Why it's not gathering as many keywords I'm not sure. The new version is supposed to get them differently from what I understand, but it should still get a good number. Can you post a URL where we can check it out?
bloodjelly is offline   Reply With Quote
Old 07-21-2004, 09:16 AM   #6
e_talis
Green Mole
 
Join Date: Jul 2004
Posts: 5
I have changed to
PHP Code:
sleep(1); 
in the spider.php and the spidering is a bit faster now.
Is there a way of excluding php errors in the summary text? ... maybe I should silence the functions that give the errors eg.
PHP Code:
@setcookie("...."
__________________
You Digg!?
e_talis is offline   Reply With Quote
Old 07-21-2004, 10:02 AM   #7
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
You could silence the errors, or set a lower level of error reporting (http://www.php.net/error_reporting), but why are you getting php errors in the first place?
bloodjelly is offline   Reply With Quote
Old 07-21-2004, 04:40 PM   #8
vinyl-junkie
Purple Mole
 
Join Date: Jan 2004
Posts: 694
To suppress 404 errors, make sure you have this set like so in config.php:
PHP Code:
define('SILENCE_404S',true); 
Note: This is a new feature of 1.8.3.
vinyl-junkie is offline   Reply With Quote
Old 07-22-2004, 07:53 AM   #9
e_talis
Green Mole
 
Join Date: Jul 2004
Posts: 5
Hello,
I have silenced the 404s, that's ok. It seems while the mole is spidering, it outputs something therefore when it reaches a page with setcookie or session_start, a header cannot be sent becoz of the output from phpdig. The same pages are working fine when I browse them.
__________________
You Digg!?
e_talis is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Search results returns blank page Anton Troubleshooting 1 02-11-2006 10:20 PM
Search results returns blank page Anton Troubleshooting 0 01-05-2006 08:56 PM
Spidering returns no results toddvirgil Troubleshooting 1 11-30-2005 03:38 PM
Spidering from shell - returns immediately, with nothing ciaran@clissman Troubleshooting 1 06-17-2005 03:14 AM
Limit search to this path returns no results mixle Bug Tracker 2 05-30-2005 10:24 PM


All times are GMT -8. The time now is 11:09 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.