PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 08-28-2005, 07:21 AM   #1
traill
Green Mole
 
Join Date: Aug 2005
Posts: 5
Lightbulb Help, I'm so nearly there! Index after unlocking?

Hi everyone, I hope someone out there can perhaps help me out, I'm very new to all this but I'm also excited if I can get it to work properly:

I was indexing my site from the root, and trying to make it go through all the links, and it would have indexed every page on my site had my internet connection not dropped. When I came back, the site was locked, so I selected it, clicked on "update form" and clicked on the unlock link.

Now I have a nice tree of my indexed site (update_frame.php page) but clearly because indexing was stopped half way through, it hasn't yet discovered all my subfolders, because it didn't get that far. How can I force the spider to find them? I've tried clicking on the green ticks in the folders that I know hold these unindexed folders - I know they're in there - but the result is merely:

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !

I'm really sorry if the solution is obvious, I have searched this forum for hours and couldn't find anything, and I've no idea how to code or program

Many thanks!
traill is offline   Reply With Quote
Old 08-28-2005, 10:50 AM   #2
traill
Green Mole
 
Join Date: Aug 2005
Posts: 5
I've worked out maybe it's because it's already been indexed, it won't let me re-index it again... or at least not follow any links in it. need some help here guys
traill is offline   Reply With Quote
Old 09-03-2005, 12:53 PM   #3
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Is there content in the tempspider table?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 09-05-2005, 12:06 AM   #4
traill
Green Mole
 
Join Date: Aug 2005
Posts: 5
Hey Charter thanks so much 4 replying.
Well, I don't know if there's content in the tempspidertable, how do I clear it? I tried clicking on "Delete Site" without selecting anything, then tried to re-index a subfolder but no luck:

So what I've done is (after unlocking the site), I've gone to the admin interface and found the folders that it didn't finish indexing the first time, clicked on its green tick and it just comes back after 10-20 seconds with listing my robots.txt file then:
File date unchanged
1:[url]http://www.myurlhere.com/folder
(time : 00:00:20)
No link in temporary table

---------------------------------

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !:

If I can get this to work it will be invaluable to the site, so I'm keen to be patient and work at it! My site has about 500 html pages and my dodgy broadband hasn't yet stayed connected long enough for the indexing to complete from scratch, though I don't mind re-indexing a few folders manually (which is guess is what I'm trying to do here)

Perhaps something isn't working correctly because it hasn't got the right permissions? Just a thought... I know I can change those permissions in cPanel... maybe I haven't set the links depth correctly.... I wanted it to find the pages and follow every link on that page, and keep going, so I set it to 20-20, is that correct?
Don't know what else to do! What do you reckon, any suggestions?
Thanks so much for your help again
traill is offline   Reply With Quote
Old 09-05-2005, 01:01 AM   #5
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Once PhpDig starts back up, it should try and use any content in the tempspider table according to the config and admin panel settings, so you don't want to empty that table at this point. If the table is already empty, then there are no links to be indexed being stored in the table from any already indexed pages. When you click a green check icon, whether on the left or right side of the screen, only that particular content is reindexed, i.e., it reindexes what has already been indexed to update just those pages. If you wish to reindex, and follow new or yet to be indexed links, use the textbox on the index page of the PhpDig admin panel. Set LIMIT_TO_DIRECTORY to true in the config file to keep the reindex within a certain directory, assuming you don't want to reindex the entire site, and choose no if you want to change the drop down numbers between indexes. As your Internet connection is prone to drops, try LIMIT_TO_DIRECTORY and then use the textbox for the incompleted folders you'd like to index. There is also this thread that you may find useful, and if you are using PhpDig 1.8.7, this thread may be helpful too.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 09-08-2005, 06:03 AM   #6
traill
Green Mole
 
Join Date: Aug 2005
Posts: 5
Ah right ok so the answer is that in order to index the folders that never got indexed, I can just type the URL into the text box, that should be fine.
When I do that, say, for a particular folder, does the spider try to index just that folder and not any subfolders if Limit_to_directory is set to "true"? And if I set it to false, will it follow links out of that instead? Sorry if it's obvious, just want to make sure

The only other thing is that I don't quite understand the "Search depth" and "links per" numbers that I can select. Ideally I want it to spider as much as it can and just follow links everywhere until it beleives it has already indexed everything, so what's the best setting?

Cheers for all your help, really appreciate it! I'm so pleased this is working out and I am beginning to understand now cos this is exactly what my website needs
traill is offline   Reply With Quote
Old 09-09-2005, 08:42 AM   #7
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Your understanding is correct. To index more content rather than less, set LIMIT_TO_DIRECTORY to false and PHPDIG_IN_DOMAIN to true (both in the config file) and then, from the admin panel, use a large search depth, set links per to zero, and use the no option.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 10-03-2005, 05:50 AM   #8
traill
Green Mole
 
Join Date: Aug 2005
Posts: 5
Hi Charter, sorry for such a late reply. That's great, I think I have this sorted now, thanks to you.

I really appreciated your time and help, so thank you again.
traill is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT -8. The time now is 02:44 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.