PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 03-13-2004, 11:21 PM   #1
yapuka
Green Mole
 
Join Date: Mar 2004
Location: France
Posts: 2
How can I restart spidering after crash?

Hello,

I started spidering a large mailing list archive Friday, and everything seemed to be going fine, until my browser crashed yesterday, after 3000 pages.

When I relaunched PhpDig, the sites was marked as 'Locked'. Since I didn't want to restart individually all the lines, I changed the value to of 'locked' to '0' in table 'sites'.

But now, PhpDig simply will not spider the site, saying it's been done recently!

How can I start it again?
yapuka is offline   Reply With Quote
Old 03-14-2004, 03:08 PM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. To unlock a site from the admin panel, choose the site, click the update button, and click the unlock link. Reindex can be based on meta tag or LIMIT_DAYS as set in the config file. You may find this thread useful. Also for larger sites it might be better to use shell when indexing if available.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 05-17-2004, 08:48 PM   #3
roger
Green Mole
 
Join Date: May 2004
Location: Cornwall, UK
Posts: 20
Hi.

I have the same problem and have to enter the database to remove the '1' lunlock value. The unlock link doesn't work. Can you help me get it working.

Roger
__________________
facts invariably distort the truth
roger is offline   Reply With Quote
Old 05-18-2004, 02:27 AM   #4
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
Try emptying the temp spider table, killing the spider process if it's still going, and unlocking the site, then spider again.
bloodjelly is offline   Reply With Quote
Old 05-18-2004, 07:58 AM   #5
roger
Green Mole
 
Join Date: May 2004
Location: Cornwall, UK
Posts: 20
Hi, how do I
Quote:
emptying the temp spider table
without going into the database which is what I'm trying to avoid by asking how to get the link to work
__________________
facts invariably distort the truth
roger is offline   Reply With Quote
Old 05-18-2004, 10:13 AM   #6
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
In the admin panel, click "delete site" without any site selected, and it will empty the temp spider table.
bloodjelly is offline   Reply With Quote
Old 05-18-2004, 01:30 PM   #7
roger
Green Mole
 
Join Date: May 2004
Location: Cornwall, UK
Posts: 20
Hi. I don't want to press the issue as I've not slept for two days but after executing the delete 'button' I still have 2 locked sites ???
__________________
facts invariably distort the truth
roger is offline   Reply With Quote
Old 05-18-2004, 02:15 PM   #8
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
Hi - no worries. Did you unlock them like Charter said above? Does the temporary table keep filling up after you empty it? If yes to the second question, you're still running a spider process that you should kill (through the shell, with the "kill PID" command)

Sometimes this happens to me, when I unlock a site only to have it keep relocking itself. If I can't kill the process for some reason I usually just keep emptying the tempspider table a few times until PhpDig chokes and it stops filling up. I know that's not the best way to do it but it's worked for me. Good luck.
bloodjelly is offline   Reply With Quote
Old 05-19-2004, 12:58 AM   #9
roger
Green Mole
 
Join Date: May 2004
Location: Cornwall, UK
Posts: 20
I'm not sure what's going on at the moment except that I can unlocj vis the database. So my thoughts are how to make a or the link work to execute an SQL command that will do it for me.

I imagine that's what's supposed to happen any way.

I've never doen any SQL so it's probably about time I did ??????

Thanks for your support BJ
__________________
facts invariably distort the truth
roger is offline   Reply With Quote
Old 05-19-2004, 01:58 AM   #10
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
Sorry Roger I guess I was answering yapuka's question and kind of confusing you. If the link doesn't work, maybe you can tell us exactly what happens. Does it unlock briefly and then lock again? When you unlock it directly through the DB does it stay unlocked? The SQL would be something like this:

"UPDATE sites SET locked='0' WHERE site_url = 'http://www.searched_website.com';"

But if that worked, the unlock link in the admin panel should work as well. Hmm...maybe someone else can throw out some ideas.
bloodjelly is offline   Reply With Quote
Old 05-19-2004, 02:14 AM   #11
roger
Green Mole
 
Join Date: May 2004
Location: Cornwall, UK
Posts: 20
Hi. Thanks

When I do a direct chenge via 'phpmyadmin' and just change the 1 to 0 it unlocks and there's no problem, it stays unlocked.

Nothing else works. The <update> <unlock> does nothing nor does <deleteing> with no site highlighted.

It shouldn't be a big problem as it only crashes, so far, when I try to quit a spider as I can see it's doing a never-ending forum with even more 'sessions' or some other reason - then it just locks the site and that's it. maybe the spidering it just going on - somehere - somehow ?????

Excuse my ignorance but how do I impliment the SQL code via an html link? If I can get that to work I'll then look at why the <unlock> link doesn't do the same. I will have a look at the current code once I've found the page.

Thanks again for your help
__________________
facts invariably distort the truth
roger is offline   Reply With Quote
Old 05-19-2004, 02:31 AM   #12
roger
Green Mole
 
Join Date: May 2004
Location: Cornwall, UK
Posts: 20
Hi Again.

My new thoughts are as follows:

As I couldn't initiate a database when I tried to personally install Version 1.8 I resorted to having an automatic insatall of version 1.6 provide by my webhost's add-on.

Then i found that the <update> link didn't work and found that there was no <update.php> file so I ftp'd across the one in version 1.8 from my local machine.

So maybe either there more wrong with my 1.6 version or the 1.8 <update.php> is incompatible with v1.6.

Maybe I should try and transfer all the other 1.8 files ???
__________________
facts invariably distort the truth
roger is offline   Reply With Quote
Old 05-19-2004, 03:13 AM   #13
roger
Green Mole
 
Join Date: May 2004
Location: Cornwall, UK
Posts: 20
Oops! I've transfered all the new files v1.8 except configure.php and connect.php and now I get no results although the database seems intact ???
__________________
facts invariably distort the truth
roger is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
MySQL crash when requesting word that is not in the index and encoding is ucs2_unicod SABsearch2 Bug Tracker 1 07-27-2006 02:40 AM
Restart spider and index urls in temptable jerrywin5 Troubleshooting 1 04-06-2005 01:18 PM
double quotes crash Phpdig Rolandks Bug Tracker 8 10-24-2003 03:36 PM


All times are GMT -8. The time now is 05:11 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.