PDA

View Full Version : Prevent search in some folders


wessam
08-06-2004, 04:13 PM
dear all
we can Prevent search in some folders with phpdig search (for example" never search and indexing (images , _???, admin \ ) folders ?????????

vinyl-junkie
08-06-2004, 06:53 PM
If you're on a Linux server, just set your robots.txt to exclude those folders. Another option is to click the red "x" to exclude those directories from the index after spidering, which also deletes them from the index.

Hope this helps. :)

wessam
08-06-2004, 11:15 PM
what about solaris server same thing only set robots.txt exclude

Charter
08-07-2004, 12:07 PM
Hi. Just stick a robots.txt file in web root. PhpDig will look for the file at http://www.domain.com/robots.txt but if you already crawled directories that you want to exclude, then you can delete and/or exclude them from the PhpDig admin panel. With version 1.8.2+ you can also limit the index to a specific directory. If you have PhpDig crawl http://www.domain.com/dir/ (with ending slash) then it should only crawl content that resides in that directory.

wessam
08-07-2004, 03:38 PM
Sorry but i can't inderstand i must creat robots.txt file or what .
if i must creat it how and i would like see an example
thanks ..
any way i did some modification on the search engine now i have advance search section (search by (Meta Tags "Keyword,title,author,description") and by date (week-6 month or from to date)
i can post this new modification or not ?
thanks

vinyl-junkie
08-07-2004, 03:55 PM
Here (http://www.robotstxt.org/wc/exclusion.html) is a site I usually point out to people who want to know something about using a robots.txt file on their site. Hope it helps. :)

As to the modification you have, I'm sure Charter wouldn't mind if you posted it. There is no guarantee it would end up in the next release, but perhaps someone may find it useful.

wessam
08-07-2004, 11:16 PM
Thank you