Why Can't I dig this site????

[lighthouse]

SUrely an easy answer to this... but I can't find it!!!

I'm trying to crawl a website built entirely in asp, but it wont spider past the fron page.

Please help!

Thanks
Gary

[lighthouse]

http://www.architecturecentre.net


digging results in:

SITE : http://www.architecturecentre.net/
Exclude paths :
- @NONE@
1:http://www.architecturecentre.net/
(time : 00:00:06)
No link in temporary table

--------------------------------------------------------------------------------

links found : 1
http://www.architecturecentre.net/
Optimizing tables...
Indexing complete !



There's loads of links on there!!!!

ANy ideas?

[Charter]

Hi lighthouse, and welcome to PhpDig.net!

If you uncomment the following line in the robot_functions.php file, you'll see that the site/server sends back a "HTTP/1.1 403 Forbidden" error.

//print $answer."<br>\n";

[lighthouse]

Hi Charter!!!

Thanks for replying....
So basically this site itself won't let me spider it?...

What would the site admin of this site have to do to allow me access?

Thanks

[Charter]

>> So basically this site itself won't let me spider it?

Hi, yep that's it. The 403 means forbidden, not allowed, go away.

>> What would the site admin of this site have to do to allow me access?

Really, I don't have much of an idea as it's not my server, but my guess is to allow HEAD requests.