alivin70
12-18-2003, 08:49 AM
I tried to index a subdirectory of a website, for example
ww.mysite.com/mydir/
but the spider goes to other dirs if it find the link. For example it can find the link to
ww.mysite.com
and starts to index the whole site.
Do you think it's possible to avoid that?
ww.mysite.com/mydir/
but the spider goes to other dirs if it find the link. For example it can find the link to
ww.mysite.com
and starts to index the whole site.
Do you think it's possible to avoid that?