View Full Version : Prevent search in some folders

08-06-2004, 04:13 PM
dear all
we can Prevent search in some folders with phpdig search (for example" never search and indexing (images , _???, admin \ ) folders ?????????

08-06-2004, 06:53 PM
If you're on a Linux server, just set your robots.txt to exclude those folders. Another option is to click the red "x" to exclude those directories from the index after spidering, which also deletes them from the index.

Hope this helps. :)

08-06-2004, 11:15 PM
what about solaris server same thing only set robots.txt exclude

08-07-2004, 12:07 PM
Hi. Just stick a robots.txt file in web root. PhpDig will look for the file at http://www.domain.com/robots.txt but if you already crawled directories that you want to exclude, then you can delete and/or exclude them from the PhpDig admin panel. With version 1.8.2+ you can also limit the index to a specific directory. If you have PhpDig crawl http://www.domain.com/dir/ (with ending slash) then it should only crawl content that resides in that directory.

08-07-2004, 03:38 PM
Sorry but i can't inderstand i must creat robots.txt file or what .
if i must creat it how and i would like see an example
thanks ..
any way i did some modification on the search engine now i have advance search section (search by (Meta Tags "Keyword,title,author,description") and by date (week-6 month or from to date)
i can post this new modification or not ?

08-07-2004, 03:55 PM
Here (http://www.robotstxt.org/wc/exclusion.html) is a site I usually point out to people who want to know something about using a robots.txt file on their site. Hope it helps. :)

As to the modification you have, I'm sure Charter wouldn't mind if you posted it. There is no guarantee it would end up in the next release, but perhaps someone may find it useful.

08-07-2004, 11:16 PM
Thank you