Not sure if this is being done yet, but it would be extremely useful to spider a site based on a full url path instead of just a domain name.
PHPDig currently does this, but what it does not do is store the search results based on the starting url. It instead stores them based on domain name.
This is a useful feature when you have a site in multiple languages. Example:
http://www.phpdig.net/en/index.php
http://www.phpdig.net/fr/index.php
http://www.phpdig.net/es/index.php
Those are entry points into the website that set a session language variable. All concurrent pages in the site are then rendered in the proper language.
The only way to support this in phpdig at the moment is to have 3 different installations which is a pain to maintain and adds unnecessary complexity/bloat to the sites code base.
I'm a linux guy personally, but what the best related example that I can think of is Microsoft's Indexing server concept of a catalog. It supports multiple catalogs with each catalog having a starting url. You can then write a search form that queries the catalog for results.
I hope this is somewhat clear, if not please let me know and I'll try to explain further
Did I mention that PHPDig rocks? Excellent work thus far!