PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 09-15-2004, 01:28 PM   #1
bbenson
Green Mole
 
Join Date: Dec 2003
Location: Charlotte, NC
Posts: 2
For Those With Password Protected Sites

I've been struggling with phpdig for quite some time, thinking I MUST be doing something wrong.

Some history: I made a website, which is restricted through apache's auth mechanism in httpd.conf, for friends and family some years ago. Last year, I decided I needed a search engine. After fiddling around with a few that didn't work so well, I came across phpdig. I loved it, but couldn't for the life of me get it to work. So, over the last year, when I had time and wasn't too busy with work and family, I would come back to trying to get it to work.

FINALLY, after scouring posts today something clicked and I thought I would share in case anyone was as stupid as I have been... possibly preventing them -- too late for me -- from going bald with frustration!

What I found out was that phpdig was happily going along and not complaining when it couldn't get to the password-protected content of my site. It would just skip over it and LOOK like it had indexed it. Only, when I went to search things nothing was there!! I couldn't figure this out and it was driving me nuts!!

Finally, a post I saw today made me look in the access logs for apache and, lo and behold, I noticed that every time I ran the spider, I was getting 401 errors in the log for the content that was password protected.

To solve, I temporarily removed the password protection from the server, reloaded the httpd.conf, and let phpdig go to work. NOW it indexed everything and I'm happy as a clam.

I hope this helps someone else someday.
bbenson is offline   Reply With Quote
Old 09-15-2004, 05:33 PM   #2
vinyl-junkie
Purple Mole
 
Join Date: Jan 2004
Posts: 694
Hi, bbenson. Welcome to the forums!

That's certainly one way of getting around the authentication problem. However, your site might be vulnerable to unauthorized activity while you're indexing like this.

Have you seen this thread? The processes described there would provide the means to index your site, while keeping phpdig security in place.

Hope this helps.
vinyl-junkie is offline   Reply With Quote
Old 09-16-2004, 07:03 AM   #3
bbenson
Green Mole
 
Join Date: Dec 2003
Location: Charlotte, NC
Posts: 2
Hi Vinyl-Junkie. Thanks for the welcome.

Thanks for the info. My site, however, is a personal site that is pretty small -- and not widely available: no DNS, just hanging off my cable modem . My removing the security while I index isn't a big deal... as I have it set, it only takes about 6 minutes to index what I want of the site.

I am very impressed with this software and many thanks to the forums, I have gotten many answers here over the last year or so... I'm just shy about posting and showing my ignorance.

Thanks again!!

Last edited by bbenson; 09-16-2004 at 07:06 AM.
bbenson is offline   Reply With Quote
Old 09-16-2004, 04:25 PM   #4
vinyl-junkie
Purple Mole
 
Join Date: Jan 2004
Posts: 694
Quote:
Originally Posted by bbenson
I have gotten many answers here over the last year or so... I'm just shy about posting and showing my ignorance.
Please don't ever be shy about asking a question here. If you haven't been able to find an answer to a question in the archives, it just might be something that not only you but a few other people would like to know the answer to.
vinyl-junkie is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Hiding Password Protected Pages joannelee How-to Forum 2 03-15-2005 10:07 AM
Highlight password-protected content with padlock? mr_ruskin How-to Forum 1 12-15-2004 03:59 AM
admin password-protected - cron job? manute Troubleshooting 3 05-15-2004 11:35 AM
problem indexing password-protected directories alexp Troubleshooting 29 12-05-2003 08:34 AM
install password protected? mistafeesh Script Installation 3 11-10-2003 11:54 PM


All times are GMT -8. The time now is 10:22 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.