Problems Indexing

[obscure]

EDIT: PROBLEM NOW FIXED (Sorry)

So I have installed php dig 1.8.0 fresh on a FreeBSD Apache 1.3 php 4.2.3 build.

I'm having a difficult time getting site 'a' on server 'a' (here (http://www.nextwest.com) ) to index properly from server 'a'.

However, our old site site 'b' on server 'b' (here (http://www.desert-solutions.com) ) works fine when indexed from server 'a'.

I have

Hosts : 1 Entries
Pages : 12 Entries
Index : 1045 Entries
Keywords : 701 Entries
Temporary table : 0 Entries

when I dig from server 'a' to site 'b' but site 'a' just hangs when indexed from server 'a'.

All of my permissions on

[PHPDIG_DIR]/text_content
[PHPDIG_DIR]/include
[PHPDIG_DIR]/admin/temp

are correct, as are my configuration parameters.
There is no robots.txt file within site 'a' or site 'b'

I've read through many posts here where people start a dig and it hangs for 60 seconds, then it displays this:

SITE : http://www.yoursite.com/
Exclude paths :
- @NONE@
No link in temporary table

--------------------------------------------------------------------------------

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !



---

This is exactly what happens to me; but just for site 'a'.
In the posts in 'troubleshooting' I have read in regards to similar issues, people upgraded from 1.6.5 and everthing worked, or they had a robots file, or they didn't have permissions set right, or site_url in the sites table contained localhost.

My site_url is http://www.mysite.com/. I also tried the IP address (it hanged too).

I've tried everything I could find in these similar posts and nothing worked.

I went as far as to disable CONTENT_TEXT and toy around with different configuration parameters (e.g.- dig level, turning off PHPDIG_INDEX*, submitting individual pages)

What's even more weird is that site 'b' has javascript redirects and our new one doesn't. It should be simple as pie for phpDig to index site 'a' from server 'a'.

(solution below

-- obscure

[obscure]

Fixed it. (lol)

As soon as I posted this I figured out the problem. (murphy's law) I figure I'll leave it up for anyone who might run into a similar one.

We are behind a Coyotepoint Systems load balancer and that means every webserver in the network has a private IP. When you dig to the same box in the private network that you're digging from it's basically sending what is called a double NAT.

This means it's asking for the public IP but the load balancer is saying it's the same thing and so it's hanging.

Anyhow, to correct this is simple: Just add the www.yoursite.com to /etc/hosts for the private IP and prefix.