PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)
-   -   Quick continue after accident and regular spiderind time. (http://www.phpdig.net/forum/showthread.php?t=1041)

sf44 07-07-2004 02:17 AM

Quick continue after accident and regular spiderind time.
 
Hi,
How to quick continue spidering/indexing after break?
My computer have crashed during spiderind/indexing proces. I am using Mozilla browser.
The site is locked - I m able to unlock it in admin, but what next ?
Update takes a lot of time, I just want to continue.
I am trying to spider more then 2000 pages ~ 30kB - I have download them to local computer.
But spidering sites takes too long - about 400 pages - 24 hours and it takes longer and longer.
Is it regular ?

Slavomir Slovakia.

sf44 07-07-2004 09:03 AM

I have solved this problem. Is is nescessary to work with database.
After some testing I have found that at my computer script which run from console is about 10 times more quick then browser based.

Slavomir.

Charter 07-07-2004 03:59 PM

Hi. Here are some answers, although you may have already figured them out. :)

>> How to quick continue spidering/indexing after break?

If spidering ends prematurely, there may be index info remaining in the tempspider table. If so, just enter the URL again or choose the site and update.

>> The site is locked - I m able to unlock it in admin, but what next ?

See above.

>> But spidering sites takes too long - about 400 pages - 24 hours and it takes longer and longer.
Is it regular ?

Maybe update to version 1.8.1 or make sure that the tempspider table is empty between runs. To empty the tempspider table, just click the delete button without selecting a site.

>> Is is nescessary to work with database.

Some people using Win/IIS servers have had to fiddle with the database tables, but if you are using Apache, then it shouldn't be necessary to work directly with the tables.

>> After some testing I have found that at my computer script which run from console is about 10 times more quick then browser based.

Welcome to shell. :D


All times are GMT -8. The time now is 01:22 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.