|
Same Problems
Im sure that its something quite simple.
I have been able to successfully spider certain sites and currently show the following stats.
Last Run : May 19, 2004
Pages : 5025 Entries
Index : 1397195 Entries
Keywords : 230416 Entries
Temporary : 110440 Entries
However, I still cannot seem to spider our own site.
A qualified subdomain.mydamain.com will work? I have changed the robots.txt and the .htaccess file and still stumped.
__________________
http://www.thewebnewsroom.com
|