View Single Post
Old 05-14-2004, 04:21 AM   #4
Siava
Green Mole
 
Join Date: May 2004
Location: Russia (Saint-Petersbrg)
Posts: 16
Pulsar-san
No, topic about LOCAL forum installed on my computer in local network

vinyl-junkie
I know, but I need to indexing some files in the forum folder (viewtopic.php, viewforum.php and somthing...).

My robots.txt exclude folders and files that I don't want indexing:

User-agent: *
Disallow: /forum/admin
... other folders ...(I'm skiping other folders)
Disallow: /forum/posting.php
... other files ....

With disallow folders all ok, but this "other files" is not disallow and there are indexing Why??

During spidering I see:

Exclude paths :
- forum/admin
........................(I'm skiping other folders)
- forum/posting\.php
.....................(and other php files with .\)

What a posting.\php ??? Why with .\ ??
Siava is offline   Reply With Quote