Taking Requests
 

Hi. If you really, ReaLLy, REALLY want something in the next release of PhpDig, NOW is the time to make the request, as I'm almost done with the next release. Note: CONSENSUS RULES!!! So the more requests for a particular item, the better the chances of seeing that item in the next release.

2 requests:

1 - small fix for the plurality of the errors

"you", are very common words and were ignored.

to

"you", is a very common word and was ignored.

etc...

2 - not sure if it's worth doing, but utilise the google api for phrase/spelling suggestions. It's not very complicated, and I've used it in other php based applications (using nusoap). If enough people ask for it, let me know and I'll help!

Proper Cookie Support in GET requests
 

Hi,

I would like to see cookies being passed properly in all page requests. Currently, in versions of PHP < 4.3, cookies are only being sent properly in HEAD requests but not in the GET requests.

This is a huge problem when you have a site that depends on a sessionID cookie as an example.

Personally I have already fixed this bug in the versions of PHPDig that I use for my clients sites.

What I did was use this small library to do it:
http://snoopy.sourceforge.com

Its an extremely simple piece of code and I fixed the PHPDig bug in 6 lines of code. (in the robot_functions.php file). Mind you, to properly support authentication and such you will need to add a few more lines :)

I will be more then happy to share my code fixes, simply let me know.

PHPDig rocks!

multiple catalog support
 

Not sure if this is being done yet, but it would be extremely useful to spider a site based on a full url path instead of just a domain name.

PHPDig currently does this, but what it does not do is store the search results based on the starting url. It instead stores them based on domain name.

This is a useful feature when you have a site in multiple languages. Example:

http://www.phpdig.net/en/index.php
http://www.phpdig.net/fr/index.php
http://www.phpdig.net/es/index.php

Those are entry points into the website that set a session language variable. All concurrent pages in the site are then rendered in the proper language.

The only way to support this in phpdig at the moment is to have 3 different installations which is a pain to maintain and adds unnecessary complexity/bloat to the sites code base.

I'm a linux guy personally, but what the best related example that I can think of is Microsoft's Indexing server concept of a catalog. It supports multiple catalogs with each catalog having a starting url. You can then write a search form that queries the catalog for results.

I hope this is somewhat clear, if not please let me know and I'll try to explain further :)

Did I mention that PHPDig rocks? Excellent work thus far!

First I want to say on behalf of everyone that you're doing an awesome job, Charter, both on development and support. Thanks also for taking requests! How cool.

Here are mine, in order of preference:

1) The ability to run multiple spider processes from the Admin panel, so that the whole will finish faster.

2) A limit on the total links per site spidered

3) Full URLs stored in the database as typed in when spidering...e.g. "http://www.site.com/folder" instead of "http://www.site.com"

:D

Hi, yes I really like phpDig : a nice tool, congratulation.

I have a directory of website, as many people, and a search web engine as phpDig visit them and indexes them. I would have the possibility to enhance the pertinence of one website or another depending of MY HUMAN judgement.

I think it will be great to set keywords to a website and giving these keywords a value (1 to 5).

Hi,

I really need Unicode-Support.. :)

my vote
 

my vote is for multibyte/unicode support

thanks for such great software

phpDig already rocks but have to agree with the two others above, unicode-support.

phrase/spelling suggestions would be nice to, but not critical.

Hi all.

I'd like to see in the next release some additions we have made
(me and alinin70); between others sponsored links.

We have added other features that we want to see in the next
release (a complete list is coming soon). First and foremost, integration with the google api using nusoap.

Ciao.

:DJyGius:D

> utilise the google api for phrase/spelling suggestions.

As Gooseman, JyGius and JanneJava suggested ... this would be most useful for me, in maintaining the search facilities for the Society of Indexers' mailing list archives.

Also, BloodJelly suggested

> The ability to run multiple spider processes from the
> Admin panel, so that the whole will finish faster.

Yes, please! I have to index 9000 pages on a monthly basis, and no matter how I do it, the indexing process still takes ~30+ hours to do!

Okay, again my plan (post some month ago) and request for : Intelligent Php-Dig Fuzzy
I have see at my statistic, user have many write errors or words which are not found, because there are other words which looks like this.

My Request is a "Did you mean Tag": See: Google - Did you mean:


My plan is:
- ideas to create better results
- create a Tag: phpdig: phonetic
- add phpdig: phonetic to index.php
- add this to the templates as: do you mean: ... (in all languages) and display to 3 TOP-hits if searchresult is empty
- add this seach to statistic --- found by phonetic ...

Problems are:
- different letter in front
- special language character: ü. ö, ä, ß - German "Straße" is in Database but "Strasse" isn't found anything (engl. street.
- a little slow (perhaps create a Table-Field for SOUNDEX and add at indexing or a Link in Admin-Menu.

You can view my source at: Test-Search-Page for (little) Intelligent Php-Dig Fuzzy

Try this search: "autentication" "deskription" "hiperlinks" on my Test-page.

Roland

I like the fact that the google implementation is tried and tested and very easy to implement.

It will work really well for spelling corrections and suggestions*, but the phrase suggestions (spelling correct, but you'll get more hits with a similar suggested phrase) works on the statistical analysis of searches and common phrases on its indexed pages.

So, in my opinion, for spelling corrections, google API is really good. For phrase suggestions, you'd probably want something more customised to your individual site (or indexed pages).

For now, I'd say go with the google api, as it's there and simple to implement. A more custom solution should be targetted to suggesting phrases based on your actual page data set.

*it's interesting that google still doesn't return results for us/uk spelling crossovers or gives you the option for this - organisation and organization return completely different results. You have to search for 'organisations OR organizations' (Then is suggests - did you mean 'organizations OR organizations') - lol

Charter - Thanks for your efforts, much appreciated here.

If you enter more than one search word can each one be highlighted in a different colour (like Google).

I do not think is is necessary to have the ability to switch them on and off (like Google).

I think all the suggestions are good, and I have been happy with phpdig.

Two things I would like to see:

1. meta-data for ranking pages.
2. Ability to crawl Web sites developed in Lotus Notes

-- John Chadwick