PhpDig.net - View Single Post - Shell Spidering Quits After Indexing A Few Pages

Wayne McBryde · 10-03-2004, 07:06 AM

I don’t know if this will help but:

I have had the spidering stop when testing from my test server but never from my production server. The 2 servers are almost identical except the test server is in my house on a DSL line and the production server is co-located about 100 feet from where level 3 comes into Charlotte, NC. The test server has at most 2 websites that have VERY little traffic and no e-mail running. The production server has over 100 websites with lots of e-mail and traffic (But the server load is light). It looks to me like my problem is related to the slower internet connection, not the server. I would expect a server that is overloaded (or at least has a heavy load) could have the same problem.

10-03-2004, 07:06 AM	#8
Wayne McBryde Orange Mole Join Date: Oct 2003 Location: NC, USA Posts: 34	I don’t know if this will help but: I have had the spidering stop when testing from my test server but never from my production server. The 2 servers are almost identical except the test server is in my house on a DSL line and the production server is co-located about 100 feet from where level 3 comes into Charlotte, NC. The test server has at most 2 websites that have VERY little traffic and no e-mail running. The production server has over 100 websites with lots of e-mail and traffic (But the server load is light). It looks to me like my problem is related to the slower internet connection, not the server. I would expect a server that is overloaded (or at least has a heavy load) could have the same problem. __________________ Wayne Mcbryde http://LakeNormansWeb.com We search all of Lake Norman!