Searching UBB Message Board

[Beans]

First let me say that I really dig PhpDig. FOr the people who wine about spending $5 for access to the grand poobahs sage advice, consider their time and effort to develop this product before you complain about $5!

Now on to my question....

I want to index a message board running UBB, and PHPDig just won't do it.

I thought at first it was because the /bboard/cgi-bin/ directory that contains the ultimate.cgi file was listed in the Robots.txt for this server, and the managment of this board removed the disallow for that directory.. but PHPDig still won't index the threads in the message board.

So now i'm wondering if it's because this particular message board requires you to be registered to see the threads? For whatever reason, PhpDig is able to index other message boards set up that way, but not this one.

So if it's the registration thing that is my obstacle.. is there a way to get PhpDig to act as me when indexing? (i.e. mimick a cookie so when it hits the website, that website thinks it's me, a registered user, browsing the fourms?)

Thanks in advance!

[Charter]

PhpDig sees what an unauthenticated user sees, though you can have PhpDig spider htaccess protected content by indexing http://username:password@www.domain.com although it's not recommended, as the username and password are sent in plain text. For some ideas on how to index session and cookie protected content, check out this (http://www.phpdig.net/forum/showthread.php?t=366) thread.

[Beans]

Is there any way to fool the message board into thinking PhpDig is me using my browser.. i.e., imbed a cookie session ID in it?

If not, sounds like the only way i'll be able to index that message board is if they make it readable to un-registered users.

[Charter]

Cookies are domain specific treats. Both UBB and PhpDig would need to be on the same domain for cookie integration, and even then you'd probably be better off using a reserved variable (http://www.php.net/reserved.variables) like in this (http://www.phpdig.net/forum/showthread.php?t=366) thread.