PhpDig.net

What is PhpDig?
PhpDig is a PHP MySQL based
Web Spider & Search Engine.
 


Download problems? Turn off any download manager/accelerator and then download.

Note that only PhpDig versions v.1.8.8 and v.1.8.9 RC1 are available for download.

Last experimental version (REQUIREMENTS)
2005-11-06 : Minor Release :
search_function(s).php links fix (thanks to alex)
phpdigDetectDir bug fix (thanks to raddanesh)
AElig entity case fix (thanks to Edomondo)
Shell script path fix (thanks to xdaniel)
phpdigCompareDomains flex added (thanks to dhorwitz)
search_function(s).php space in path fix (thanks to mixle)
Statistics/List query string order fixed (thanks to xmsmmgrs)
Remove hard-coded snippet length (thanks to quadisweb)
Account for more chars in robots.txt (thanks to fluxx)
Fix undefined user_agent (thanks to Noel)
Fix config date format use (thanks to Noel)
Fix undefined previous_link

zip file phpdig-1.8.9-rc1.zip 564.44 kb (direct link)
Last stable version (REQUIREMENTS)
2005-11-06 : Minor Release :
search_function(s).php links fix (thanks to alex)
phpdigDetectDir bug fix (thanks to raddanesh)
AElig entity case fix (thanks to Edomondo)
Shell script path fix (thanks to xdaniel)
phpdigCompareDomains flex added (thanks to dhorwitz)
search_function(s).php space in path fix (thanks to mixle)
Statistics/List query string order fixed (thanks to xmsmmgrs)
Remove hard-coded snippet length (thanks to quadisweb)
Account for more chars in robots.txt (thanks to fluxx)
Fix undefined user_agent (thanks to Noel)
Fix config date format use (thanks to Noel)
Fix undefined previous_link

zip file phpdig-1.8.8.zip 280.18 kb (direct link)
License
GNU/GPL, not less, not more. See the License file.
Archive Content in the Last stable version
clickstats.php
custom_rss.php
custom_search.php
list.php
phpdig_logo_2.gif
phpdig_logo_2.png
phpdig_powered_2.gif
phpdig_powered_2.png
search.php

admin/cleanup_common.php
admin/cleanup_dashes.php
admin/cleanup_engine.php
admin/cleanup_keywords.php
admin/debug_functions.php
admin/deny.gif
admin/details.gif
admin/files.php
admin/fill.gif
admin/index.php
admin/install.php
admin/limit_update.php
admin/logout.php
admin/no.gif
admin/README.txt
admin/robot_functions.php
admin/spider.php
admin/statistics.php
admin/stop_spider.php
admin/update.php
admin/update_frame.php
admin/yes.gif

admin/temp/cronfile.txt
admin/temp/keepalive.txt

documentation/phpdig-api-en.html
documentation/phpdig-api-en.txt
documentation/phpdig-doc-en.html
documentation/phpdig-doc-en.txt
documentation/README.txt

includes/_connect.php
includes/common_words.txt
includes/config.php
includes/de_en_fr_common_words.txt
includes/it_common_words.txt
includes/README.txt
includes/style.css

information/CHANGELOG
information/COPYING
information/CREDITS
information/FILELIST
information/LICENSE
information/README

libs/auth.php
libs/authold.php
libs/function_phpdig_form.php
libs/htmlheader.php
libs/htmlmetas.php
libs/mysql_functions.php
libs/phpdig_functions.php
libs/search_function.php

locales/ca-language.php
locales/cs-language.php
locales/da-language.php
locales/de-language.php
locales/de-language-old.php
locales/en-language.php
locales/es-language.php
locales/fr-language.php
locales/gr-language.php
locales/it-language.php
locales/it-language-old.php
locales/nl-language.php
locales/no-language.php
locales/pt-language.php
locales/README.txt

rss/keepalive.txt

sql/init_db.sql
sql/update_db.sql
sql/update_db_to_1_4_4.sql
sql/update_db_to_1_4_5.sql
sql/update_db_to_1_6.sql
sql/update_db_to_1_6_1.sql
sql/update_db_to_1_8_1.sql
sql/update_db_to_1_8_2.sql
sql/update_db_to_1_8_4.sql
sql/update_db_to_1_8_5.sql
sql/update_db_to_1_8_6.sql
sql/README.txt

templates/black.html
templates/bluegrey.html
templates/corporate.html
templates/gaggle.html
templates/green.html
templates/grey.html
templates/lightgreen.html
templates/linear.html
templates/newspaper.html
templates/phpdig.html
templates/simple.html
templates/terminal.html
templates/yellow.html

text_content/keepalive.txt

tpl_img/aq_background.gif
tpl_img/bar_left.gif
tpl_img/bar_middle.gif
tpl_img/bar_right.gif
tpl_img/barre-verte.gif
tpl_img/barre-verte-bas.gif
tpl_img/barre-verte-bas-courte.gif
tpl_img/barre-verte-courte.gif
tpl_img/coininfd.gif
tpl_img/coininfg.gif
tpl_img/coinsupd.gif
tpl_img/coinsupg.gif
tpl_img/cprgo.gif
tpl_img/cprgradient.jpg
tpl_img/cprlogo.jpg
tpl_img/cprtitle.jpg
tpl_img/d.gif
tpl_img/fond.gif
tpl_img/g.gif
tpl_img/horzfeuil.gif
tpl_img/inf.gif
tpl_img/left.gif
tpl_img/orbar-center.gif
tpl_img/orbar-left.gif
tpl_img/orbar-right.gif
tpl_img/redbar_left.gif
tpl_img/redbar_middle.gif
tpl_img/redbar_right.gif
tpl_img/right.gif
tpl_img/space15_15.gif
tpl_img/sup.gif
tpl_img/vertfeuil.gif
tpl_img/weight.gif
Old versions and changelog
2005-01-30 : Minor Release :
Multiple and multibyte support available (thanks to Mikolaj Jedrzejak for the ConvertCharset class).
Searching and highlighting content stored in a table works similar to the way it works for files.
The amount of content stored from each indexed page can be specified.
Indexing can be performed within entire subdirectory (thanks to td234).
The title displayed in search results can be limited to a certain length.
Renamed file and other miscellaneous edits/corrections.

zip file phpdig-1.8.8-rc1.zip 563.38 kb (direct link)

2005-01-16 : Minor Release :
Added ability to view previous search queries with links to search page.
Chunk encoding improvement in phpdigGetUrl function (thanks to attriel).
Improved indexing of links with special characters (thanks to zaartix).
Added ability to turn off click logging (thanks to vinyl-junkie).
Included some custom code to make own RSS and search templates.
Renamed some files and other miscellaneous edits/corrections.

zip file phpdig-1.8.7.zip 279.43 kb (direct link)

2004-12-15 : Minor Release :
Added a constant based security check.
Conformed search output to standards (see http://www.php.net/manual/en/faq.html.php#faq.html.encoding).
Removed predefined server global from functions.
Fixed case in HTML entities (thanks to salzbermat).
Do not show "did you mean" if words not available.
Other miscellaneous edits/corrections.

zip file phpdig-1.8.6.zip 274.75 kb (direct link)

2004-12-12 : Minor Release :
Highlight fixed for databased content.
Major security fix (thanks to zaartix).
CHANGE YOUR PASSWORDS AND THEN UPGRADE REGARDLESS OF YOUR VERSION!

zip file phpdig-1.8.5.zip 269.50 kb (direct link)

2004-12-06 : Minor Release :
Ability to stop spider from browser added.
Search menu now supports search all option.
Can set different depths and links per site.
Text box available for multiple links via browser.
Explore path links with query string added (thanks to blueyed).
Return of update one page or directory (thanks to vinyl-junkie).
Fuzzy "did you mean" now by word not phrase (thanks to Rolandks).
Remove session variable fixed (thanks to Edomondo, indeh).
Relaxed cleaning regex in function (thanks to pavel).
Close connection added to requests (thanks to vital).
Limit to directory fixed for shell (thanks to indeh).
Remove duplicate log information (thanks to ChadK).
Encoding typo fix (thanks to kotaksurat99).

zip file phpdig-1.8.4.zip 268.50 kb (direct link)

2004-07-14 : Minor Release :
Fix chunk encoding transfer issue with GET requests (thanks to Nad).
Correct typo in defined variable (thanks to davenewt).
Improve limit to directory option so it is consistent across options.
Allow links per depth to be set on a site by site basis.
Various edits to files.

zip file phpdig-1.8.3.zip 248.16 kb (direct link)

2004-07-12 : Minor Release :
Magic quotes issue fixed when magic_quotes_runtime is on (thanks to majestique).
Authentication method based on cookies fixed (thanks to pki, RobM, manfred).
Variable edits for when register_globals is off (thanks to RobM).
Option to show hosts with dirs added to search menu.
Backwards order of search terms fixed.
Limit spider to specific directory.

zip file phpdig-1.8.2.zip 247.09 kb (direct link)

2004-07-06 : Minor Release :
Click tracking now available (thanks to alivin70 and JÿGius³).
Cron job text file management (thanks to alivin70 and JÿGius³).
Search has 'did you mean X instead' fuzzy (thanks to Rolandks).
GET request modification to pass cookies (thanks to fredh).
Reading of robots.txt file updated (thanks to Carl Mikkelsen).
PPT support using external binaries (thanks to Carl Mikkelsen).
Limit spider to max of Y number of links per depth per site.
Different authentication method based on cookies.
Multiple session IDs and var names removable.
Now reads base href tags for indexing.
Some extra characters allowed in URLs.
Plurality of some phrases updated.
RSS feeds by search available.
Search by site or directory.
Can remove '-' index pages.
Support for TIS-620 added.
Different keyword storage.
Various edits to files.
Some bug fixes.

zip file phpdig-1.8.1.zip 245.61 kb (direct link)

2004-01-19 : Minor release :
The "and operator - exact phrase - or operator" replaces "words begin - exact words - any words part" options.
Security vulnerability in config.php file fixed (thanks to fraMe).
Support for iso-8859-7 and windows-1251 added (thanks to sv2bbi, others).
Characters '._~@#$:&%/;,=- now allowed in indexing and searches.
CSS modified in all templates and style.css file.
Various edits to several functions and/or files.
UPDATE TO VERSION 1.8.0 RECOMMENDED!

zip file phpdig-1.8.0.zip 219.79 kb (direct link)

2003-12-03 : Minor release :
Escaping added to path and file if necessary (thanks to ullone).
Highlight fixed when keyword is followed by period (thanks to mark).
Regex relaxed to allow for more characters (thanks to RedThypon).
Max number of results per site changed to allow all results in limit to searches.
Search depth of level zero enabled for index.
Option to bypass renice command added.

zip file phpdig-1.6.5.zip patched* 217.30 kb (direct link)

*For all versions prior to 1.6.5 (and non-patched version 1.6.5) add this patch.

2003-11-17 : Minor release :
Display fix in result message (thanks to 123av).
Regex applied to path and title (thanks to manfred).
Option to bypass is_executable added (thanks to manfred).
Option to specify temp filename length added (thanks to manfred).
Empty temp files no longer in temp directory (thanks to manfred).
Extension options and external binary process modified.
Option to set max number of results per site added.
Exact match word highlighting fixed again.

zip file phpdig-1.6.4.zip 208.08 kb (direct link)

2003-11-09 : Minor release :
End of line marker fixed and added to config file (thanks to Rolandks).
Search box size and maxlength options added to config file (thanks to Rolandks).
Snippet display length option added to config file (thanks to plodz).
Missing l_time column added to logs table (thanks to Iltud, others).
The PHP strip_tags replaced with regular expression (thanks to Rolandks, manute).
The PHP mysql_create_db replaced with mysql_query (thanks to rayvd).
The PHPDIG_INCLUDE_COMMENT excluded from index (thanks to Iltud).
Extension options for external binaries added to config file.
Exact match word highlighting fixed.

zip file phpdig-1.6.3.zip 207.12 kb (direct link)

2003-04-06 : Minor release :
Add support of others charsets than 8859-1, encoding 8859-2 added (Jan Kincl).
PhpDig handles meta http-equiv cookie.
Function phpdigTestUrl fixed.
Css classes for classic mode fixed.
Bug on noindex and nofollow fixed (Michael Chapman).
Small API doc added.
Error on database creation script on some versions of MySql fixed.
zip file phpdig-1.6.2.zip 202.54 kb (direct link)

2003-03-15 : Minor release :
Experimental handle of cookies added.
Experimental removing of Session ids.
Better handling of javascript window.open.
Handle default indexes as option.
Considers '+' as permitted character in Urls.
Add average search time in logs.
All MySql connection parameters are now constants.
Update in install script fixed.
zip file phpdig-1.6.1.zip 179.48 kb

2003-03-09 : Major release :
PhpDig could now index PDF, MS-Word and MS-Excel files using external binaries.
Locking system : An host is locked from concurrent indexings.
Localization of all remaining hard-coded messages complete (Eric Chauvin).
Optimized queries and template parsing.
Admin interface and template "PhpDig" xhtml compliancy added (Eric Chauvin).
Install web interface could update exising databases.
Parts of html pages could be excluded from indexing with special formatted comments.
Handling of mysql connections improved.
Statistics on searchs are collected to know what the visitors want first in the website.
New ranking system added, lowering ranking of pages with a lot of same words.
More explanations of how phpdig works added in documentation.
zip file phpdig-1.6.zip 171.01 kb

2003-03-01 : New features, buxfixes :
Text snipets now match search mode (start/any/exact).
Results extracts are more customizable.
spider can read a file containing urls' list to explore.
Delete more than one host at once from index is possible.
New design for admin interface.
Resume and force indexing fixed.
Templates parsing fixed.
Cleanup scripts fixed.
zip file phpdig-1.4.8.zip 137.84 kb

2003-02-26 : New features :
MySql tables can be prefixed by an user-defined string.
Spidering an entire domain is now possible.
Better handling of redirections.
Doc spelling corrections (John Zastrow)
Updated german locale file (Matthias Strohmaier)
New Norwegian locale file (Martin Kristiansen)
New Czech locale file (Dan Barta)
Remaining E_ALL errors fixed (i tried to hunt all of them...)
zip file phpdig_1_4_7.zip 130.03 kb

2003-02-22 : New features :
PhpDig works with register_globals = off and/or Error_reporting = E_ALL
Restore starting indexing by other path than /
Using only <?php ?> tags now
An option makes search function returning an array
All functions renamed and prefixed by "phpdig"
Using two specific CSS classes for results links and highlighting
Some code improvement where made
If an error message occurs while indexing, please download the .zip again. The first one was corrupted (between 00:30:00 GMT and 11:30:00 GMT, 02/22)
zip file phpdig_1_4_6.zip 123.77 kb

2003-02-18 : PHP Issue :
Patch to correct content retrieval due to php bug.
See Bug #22008 for more explanations.
zip file phpdig_1_4_5c.zip 121.83 kb

2003-02-17 : BugFix :
Broken indexation of hosts bound to another port than 80 repaired.
zip file phpdig_1_4_5b.zip 121.82 kb

2003-02-16 : New Features, Bugfixes :
Note : Upgrade of database is needed, use the update_db_to_1_4_5.sql file.
Search is now a function, making integration easier. (template could be only a part of a page.)
Highlight fixed.
Using a CSS instead "style.php" file.
Configuration directives are now constants, except for arrays.
Exclude a path at robot side is possible now.
zip file phpdig_1_4_5.zip 121.27 kb

2003-02-09 : Corrected version :
PhpDig works with PHP 4.3.0 (still register_globals=on).
Spidering whith shell command (php-cli) fixed.
Templates fixed.
zip file phpdig_1_4_4c.zip 117.98 kb

2001-12-03 : Corrected version :
Fixed doubles inserted in the sites table.
zip file phpdig_1_4_4b.zip 116.34 kb

2001-12-02 : New features, bugfixes :
PhpDig can now spider a site binded to another port than 80.
PhpDig can also spider a password protected site (please read the documentation warning).
Ehanced directory view in admin mode.
Islandic (!) special characters are now supported.
Working on a E_ALL error_reporting level fixed.
Bad Last-Modified HTTP header parsing fixed.
zip file phpdig_1_4_4.zip 112.25 kb

2001-11-27 : Minor update and bugfixes :
Improved templates system
Field added in keywords table optimize search queries
Some queries causing error fixed
Code part causing php core dump fixed
Not updated textual content fixed
Update of branch/files fixed
zip file phpdig_1_4_3.zip 109.45 kb

2001-11-24 : Minor update :
Complete english documentation added.
Best robots.txt file parsing : The wildcard * is now supported, and files can be specified (with complete path).
The special character "ß" is included in indexing, some german words were not reconized. Thanks Christof Fritz for bug report.
zip file phpdig_1_4_2.zip 107.42 kb

2001-11-18 : Documentation updates :
Full english version added.
zip file docs_2001-11-18.zip 34780 bytes

2001-11-11 : Minor bugfixes :
Both two patches included
Complete french documentation added (Need help on english translation)
Simple http authentification added
zip file phpdig_1_4_1.zip 89741 bytes

2001-11-11 : New patch for 1.4 version :
A bug in relative links parsing fixed.
A bug in the test_url() function fixed.
Thanks to Florian Perrichot for the bug report
zip file patch_1_4-2.zip 12440 bytes

2001-11-07 : Last minute patch for 1.4 version :
Correction of last bugs : replace index.php at root dir, spider.php in admin/, config.php in includes/ and black.htm in templates/ weight.gif is a new file.
zip file patch_1_4.zip 10246 bytes

2001-11-06 : Major release :
Both spidering and indexing are proceeded in the same time.
Much less charge on indexed servers with a cache system.
The results page show now extracts of the doccuments with the search keys occurences.
The admin, libs and configuration scripts are now in separate directories, allowing protect it by some .htaccess files.
The results page is highly customizable by a simple template system (samples provided).
Ehanced CGI mode for total automatic updates with a cron task.
Great thanks to Florian Perrichot for cache and templates system.
Portugese locale file provided by Carlos Serrão.
zip file phpdig_1_4.zip 67716 bytes

2001-06-04 : Minor bugfix :
Bug which causes PhpDig send an http request on each link it finds in pages regardless it already make it fixed.
zip file phpdig_1_0_4.zip 62656 bytes

2001-05-28 : Minor update :
Italian locale file provided by Mirko Maischberger.
zip file phpdig_1_0_3.zip 62575 bytes

2001-05-27 : Minor update :
Http and cgi versions of indexing merged.
Lot of more comments in source code.
zip file phpdig_1_0_2.zip 58551 bytes

2001-05-22 : Bug fixes :
Missing field fixed in init_db.sql.
Excluding words in search queries fixed.
Quotes and double quotes in search form fixed.
zip file phpdig_1_0_1.zip 60737 bytes

2001-05-19 : Stable version :
Spanish locale file provided by Geffrey Velásquez.
Bug fixed in parsing of "alt" attributes in img tags.
"description" metatag is included in search results page.
zip file phpdig_1_0.zip 60691 bytes

2001-05-14 : Major bugfixes and Update :
Fixed bug which inserts doubles in database.
Fixed bugged queries in update_cgi script.
Fixed bug which cause phpdig fails in detect description and keywords metatatags.
Fixed bug in html entities parsing.
Fixed bug in reconizing some words in html_to_plain_text() function.
Last-modified header is supported now. Don't forget to update your database with the update_db_0_99.sql script !
Metatag 'Revisit after' is supported now.
Sub-directories in robots.txt file are reconized.
Delete an entire site from database is supported now.
Last minute bug : in the indexing scripts, comment the debug lines $upddate = 0; (lines 78, and 102 in the cgi script)
zip file phpdig_0_99.zip 60214 bytes

2001-05-10 : Update :
German locale file provided by Gregor Mucha.
German stop-words added by the same person.
External domains names in Hrefs are indexed (i.e. www.gnu.org) an can be retrieved by search queries.
Some classic files added : COPYING, README and LICENSE.
zip file phpdig_0_98b.zip 58187 bytes

2001-05-08 : Update :
robots.txt file and META ROBOTS are reconized. See The Web Robots Page to obtain more informations.
Increase speed in indexing text files.
Files without extension are indexed now.
Indexes and primary key in the database are a bit different. Check the init_db.sql file to see changes.
zip file phpdig_0_97b.zip 48729 bytes

2001-05-06 : Update :
Some files corrected by Brien Louque : documentation_en.html, search.php, en-language.php
Greek locale file provided by Sofoklis Magoulas.
An auto-update script was added. You must have access to the crontab and to an executable cgi of php in order to use it.
Expire time for pages are used by indexing scripts.
zip file phpdig_0_96b.zip 43779 bytes

2001-05-05 : Update :
PhpDig is now avaible in both english and french.
Localized search forms are provided with archive.
zip file phpdig_0_95b.zip 38275 bytes

2001-05-03 : Update :
English doc was added to the archive.
I changed the search algorithm. Less SQL, more php.
Localization in some languages in progress.
You can now exclude search keys.
The occurence is based on a product, not more on a sum.
Search form and results page are provided in english.
zip file phpdig_0_93b.zip 39285 bytes

2001-05-02 Minor bugfixes :
Results page now keeps filters.
news: links are not more followed.
Some SQL queries are optimized.
SQL_BIG_SELECT is set to 1 for search queries.
No more IE user_agent string send ;-).
zip file phpdig_0_92b.zip 34203 octets

2001-05-01 Minor buxfixes :
Long texts bug which freezes PhpDig is fixed.
zip file phpdig_0_91b.zip 31436 octets

2001-04-30 Initial release :
zip file phpdig_0_9b.zip 31180 octets


Powered by: vBulletin Version 3.0.7
Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.