PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   How-to Forum (http://www.phpdig.net/forum/forumdisplay.php?f=33)
-   -   how to continue spidering when connection was been broken (http://www.phpdig.net/forum/showthread.php?t=1583)

zaartix 12-07-2004 09:23 PM

how to continue spidering when connection was been broken
 
how to continue spidering when connection was been broken?
i have modem line to internet, in new version of phpdig if i click on 'stop spidering' script deleting all tempspider table, but how to continue previus spidering?

zaartix 12-07-2004 10:09 PM

i trying to update site and click on reindex, but phpdig only reindexing old links without new links collecting.
the max number of links per depth per site is 0
the max search depth per site is 8, but indexed only 1 and 2

Charter 12-08-2004 10:55 AM

Update is meant to update what is there. To restart the spider, use the textbox on the admin/index.php page.

zaartix 12-08-2004 07:19 PM

how to continue previus spidering? script cleaning tempspider table, and using textbox is not working.

for example:
i have a page with a lot of links (70). It have URI http://domain.com/index.php/action/viewNews
I'm using a textarea for adding it and paste into textfield "http://domain.com/index.php/action/viewNews"
PHPDIG is beginning spidering, finding 70 new links, and we are close a browser.
So, on admin page we can see a "Temporary table : 70 Entries" and a looked site http://domain.com. If i click on "Update" site and unlooked it, i can't index other pages, just one by one through textbox. If add or update page "http://domain.com/index.php/action/viewNews" it does'nt find any links.



sorry for bad english

Edomondo 12-09-2004 12:17 AM

Hi.

I have the same problem.

Instead of respidering your site through the update site section, I recommend you to enter the URL of the page that contains the more links (menu page or index...) in the textbox and spider this way, just like you would do to index a new site.
Phpdig will find more links than with the updating tool.

Charter 12-09-2004 02:51 AM

The update section is meant to update what is already there, whether updating a page or directory. To get a site-wide reindex including new pages, just use the textbox. If you empty the tempspider table, then you cannot pick up where the spider left off. If your connection breaks, and there is content left in the tempspider table, you might want to look at the table and start the indexing again by using a link formed from the tempspider table where indexed is zero.

zaartix 12-09-2004 06:31 AM

that's a problem!
phpdig cleaning tempspider table when i try to update site, or unlock (i don't know)

zaartix 12-09-2004 07:18 AM

i've solve this problem, just comment in update.php on 88,89 strings.
now it's real to continue indexing use a textarea


All times are GMT -8. The time now is 09:50 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.