View Single Post
Old 06-16-2005, 02:43 PM   #6
Vertikal
Former Member
 
Join Date: Jun 2005
Posts: 6
New behaviour

Charter,

I don't know whether this will help, but during my manual indexing of a lot of single pages on the site in question, something different happened.

The site is usually referred to as http://globalflyfisher.com/ with no www. But on one page there was a reference to some subpage with the http://www.globalflyfisher.com/ domain. And lo and behold if the crawler didn't start crawling all links it met on every single page... and it's still crawling as I write this.

Again I cannot see any reason for this, but I thought it might hint where to look for the explanation. Might it be because the www. site was "new and untouched"?

Martin
Vertikal is offline   Reply With Quote