PDA

View Full Version : Doesn't index at all!


Ally
02-04-2005, 07:18 AM
I installed PhpDig on our company's intranet site. Installation was successful but when i try to index site (with depth 10 and links 10, for example) i'm getting Spidering in progress... [Stop spider]
Optimizing tables...
Indexing complete ! and that's all! It doesn't index and doesn't give any diagnosis!! Site's url looks like subdomain.ourdomain.tld:7780.
What can be the reason?
Thanks in advance.

clueless
02-04-2005, 11:40 AM
Hi Ally,

I have had that problem as well I tried it again by adjusting the search depth and the links per that fixed it for me. Give it a try I.e set depth to 1 and links per to 10

jmitchell
02-04-2005, 12:08 PM
how many times did you try this?

I'm not sure about why it would do this on the first time, but on subsiquent tries this might come up because you recently tried spidering it.

JMitchell

Ally
02-05-2005, 04:43 PM
I tried it several times with different search depth and the links per but it didn't even appeared in my access/error logs! Does it mean that it couldn't connect to the server?
Thanks.

jmitchell
02-05-2005, 07:38 PM
try deleting the site from the site list in phpdig, and then try to reindex it.

Charter
02-07-2005, 12:31 AM
One of the first requests PhpDig makes is for a robots.txt file. If you don't even see this in your server access logs, then maybe the intranet is blocking PhpDig, or maybe you didn't set the correct permissions?

Ally
02-07-2005, 06:57 AM
The site doesn't have robots.txt at all. Even if a add one with allow / - nothing changes. :(
The site is accessible from the Internet. How can it be blocked from spiders? I'm desperate! I even doesn't know what do i have to ask my admin about...
Help me, please!

Charter
02-07-2005, 09:37 AM
Do you see any requests in the server access log from PhpDig? If not, then ask your Intranet admin, if s/he is blocking PhpDig from accessing the Internet, and thus your site.

Ally
02-07-2005, 02:36 PM
Thanks for your advice, i'll ask him about it.