|
02-26-2004, 02:26 AM | #1 |
Green Mole
Join Date: Feb 2004
Location: Sao Paulo - Brazil
Posts: 8
|
Working with Domains
Hi:
I am working with PhpDig and it is working fine. I am doing some tests in order to start to PhpDig some sites. I was testing a site but would like to PhpDig only some subdirectories. The site is this: http://dollar.biz.uiowa.edu/~fil/Erico/ I am using depth 20 to go as much as possible inside the site. The problem is: the http://dollar.biz.uiowa.edu/~fil/Erico/ subdirectory has some pages linked to http://dollar.biz.uiowa.edu and PhpDig start to Dig in this link. I think that if I am PhpDiging in http://dollar.biz.uiowa.edu/~fil/Erico/ this should be the domain and so anything found linked different from this domain 'http://dollar.biz.uiowa.edu/~fil/Erico/ ' should not be included in the database. Is that possible? Am I doing something wrong? Any help will pe very appreciated. Regards, Wagner Bazarin www.bazarin.com |
02-28-2004, 04:28 PM | #2 |
Head Mole
Join Date: May 2003
Posts: 2,539
|
Hi. You might try setting up a robots.txt file, or after crawling, exclude and/or delete certain directories from the admin panel.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension. |
Thread Tools | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Banned Domains | JLutterklas | How-to Forum | 0 | 09-05-2006 11:38 AM |
Blocking domains | richwilson | How-to Forum | 0 | 03-29-2006 07:02 AM |
Same Hosting but 2 differents domains | MoonC | Troubleshooting | 1 | 07-23-2005 12:28 PM |
Newbie on Domains: Yes or No Answer Please :) | new2dev | How-to Forum | 1 | 03-02-2005 12:24 AM |
Choosy about domains? | druesome | Troubleshooting | 7 | 04-30-2004 01:39 AM |