PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 11-14-2005, 06:58 AM   #1
gim
Green Mole
 
Join Date: May 2005
Location: swizterland
Posts: 5
some problems with spider

Hello,
First the english in not my mother language (it's french ..)
Second I have installed phpdig 1.8.7 and all the other stuff for phpdig in my site.
The site is local and I try to index another site in another server and ... wonderful.. the spider is work fine.
BUT ... when i try to index my first site where phpdig is installed I receive no answer (0 index....).
I have already configurate phpdig with PHPDIG_IN_DOMAIN at true and SPIDER_MAX_LIMIT to 200 and LIMIT_TO_DIRECTORY to false in the file
.../phpdig-1.8.7/includes/config.php

The site where phpdig is ok is intranet.siesa.ch
The site where phpdig is install is intranet2.siesa.ch
and the site where phpdig isnt ok is intranet2.seisa.ch

Do you have any solution ??
Thank for your answer and your time.

Gim
gim is offline   Reply With Quote
Old 11-15-2005, 11:58 PM   #2
gim
Green Mole
 
Join Date: May 2005
Location: swizterland
Posts: 5
Hello,
Ok I find a part of problem with the answer about /etc/hosts and localhost.
now I receive another message
SITE : http://intranet2.siesa.ch/
Chemins exclus :
- @NONE@
1:http://intranet2.siesa.ch/
(temps : 00:00:05)

Pas de liens dans la table temporaire
liens trouvés : 1
http://intranet2.siesa.ch/
Optimizing tables...
Indexation terminée !


???????????
It's seem strange because I have the same site on other server and I dig him with no problems ???
When I dig the reference site in the other server I receive this 'normal' answer


Do you have any suggestions ??

SITE : http://intranet.siesa.ch/
Chemins exclus :
- @NONE@
1:http://intranet.siesa.ch/
(temps : 00:00:10)
+ + + +
niveau 1...
2:http://intranet.siesa.ch/pages/docum...ast_update.php
(temps : 00:00:20)
+ + + + + +
3:http://intranet.siesa.ch/pages/news/news.php
(temps : 00:00:26)
+
4:http://intranet.siesa.ch/pages/agenda/agenda.php
(temps : 00:00:31)
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
5:http://intranet.siesa.ch/include/banniere.php
(temps : 00:00:36)
+ + + + + + + +
niveau 2...
6:http://intranet.siesa.ch/pages/agend...nom=10-11-2005
(temps : 00:00:47)
+ + + +
7:http://intranet.siesa.ch/pages/agend...&nom=9-11-2005
(temps : 00:00:52)
+ + + +
8:http://intranet.siesa.ch/pages/agend...&nom=6-11-2005
(temps : 00:00:58)
+ + + +
9:http://intranet.siesa.ch/pages/agend...&nom=7-11-2005
(temps : 00:01:03)
+ + + +
10:http://intranet.siesa.ch/pages/agend...&nom=8-11-2005
(temps : 00:01:08)
+ + + +
11:http://intranet.siesa.ch/pages/agend...&nom=4-11-2005
(temps : 00:01:14)
+ + + +
12:http://intranet.siesa.ch/pages/agend...&nom=5-11-2005
(temps : 00:01:19)
+ + + +
13:http://intranet.siesa.ch/pages/agend...&nom=3-11-2005
(temps : 00:01:24)
+ + + +
14:http://intranet.siesa.ch/pages/agend...&nom=2-11-2005
(temps : 00:01:30)
+ + + +
15:http://intranet.siesa.ch/pages/agend...&nom=1-11-2005
(temps : 00:01:35)
+ + + +
16:http://intranet.siesa.ch/pages/agend...nom=16-11-2006
(temps : 00:01:41)
+ + + + + + + + + + + + + + + + + + + + + +
17:http://intranet.siesa.ch/pages/agend...nom=16-11-2004
(temps : 00:01:46)
+ + + + + + + + + + + + + + + + + + + + + +
18:http://intranet.siesa.ch/pages/agend...nom=16-12-2005
(temps : 00:01:52)
+ + + + + + + + + + + + + + + + + + + + +
19:http://intranet.siesa.ch/pages/agend...nom=16-10-2005
(temps : 00:01:57)
+ + + + + + + + + + + + + + + + + + + + +
20:http://intranet.siesa.ch/pages/agenda/
(temps : 00:02:03)
+ + + + + +
21:http://intranet.siesa.ch/pages/news/
(temps : 00:02:08)
+ + + + +
22:http://intranet.siesa.ch/files/co_tel_netBusiness.pdf
(temps : 00:02:13)

23:http://intranet.siesa.ch/files/ns_sie_05-439.pdf
(temps : 00:02:18)

24:http://intranet.siesa.ch/files/ns_sie_05-440.pdf
(temps : 00:02:23)

25:http://intranet.siesa.ch/files/pl_sie_horaires.pdf
(temps : 00:02:28)

26:http://intranet.siesa.ch/pages/documents/documents.php
(temps : 00:02:33)
+ + + + + + + + + +
27:http://intranet.siesa.ch/pages/documents/
(temps : 00:02:38)
+ + + + +
28:http://intranet.siesa.ch/pages/agend...nom=11-11-2005
(temps : 00:02:44)

29:http://intranet.siesa.ch/pages/agend...nom=12-11-2005
(temps : 00:02:49)

30:http://intranet.siesa.ch/pages/agend...nom=13-11-2005
(temps : 00:02:54)

31:http://intranet.siesa.ch/pages/agend...nom=14-11-2005
(temps : 00:03:00)

32:http://intranet.siesa.ch/pages/agend...nom=15-11-2005
(temps : 00:03:05)

33:http://intranet.siesa.ch/pages/agend...nom=16-11-2005
(temps : 00:03:10)

34:http://intranet.siesa.ch/pages/agend...nom=17-11-2005
(temps : 00:03:16)

35:http://intranet.siesa.ch/pages/agend...nom=18-11-2005
(temps : 00:03:21)

36:http://intranet.siesa.ch/pages/agend...nom=19-11-2005
(temps : 00:03:26)

37:http://intranet.siesa.ch/pages/agend...nom=20-11-2005
(temps : 00:03:32)

38:http://intranet.siesa.ch/pages/agend...nom=21-11-2005
(temps : 00:03:37)

39:http://intranet.siesa.ch/pages/agend...nom=22-11-2005
(temps : 00:03:43)

40:http://intranet.siesa.ch/pages/agend...nom=23-11-2005
(temps : 00:03:48)

41:http://intranet.siesa.ch/pages/agend...nom=24-11-2005
(temps : 00:03:53)

42:http://intranet.siesa.ch/pages/agend...nom=25-11-2005
(temps : 00:03:59)

43:http://intranet.siesa.ch/pages/agend...nom=26-11-2005
(temps : 00:04:04)

44:http://intranet.siesa.ch/pages/agend...nom=27-11-2005
(temps : 00:04:09)

45:http://intranet.siesa.ch/pages/agend...nom=28-11-2005
(temps : 00:04:15)

46:http://intranet.siesa.ch/pages/agend...nom=29-11-2005
(temps : 00:04:20)

47:http://intranet.siesa.ch/pages/agend...nom=30-11-2005
(temps : 00:04:26)

48:http://intranet.siesa.ch/pages/agenda/agenda.php?deb=2
(temps : 00:04:31)
+ +
49:http://intranet.siesa.ch/include/
(temps : 00:04:36)
+ + + +
50:http://intranet.siesa.ch/pages/modules/modules.php
(temps : 00:04:41)

51:http://intranet.siesa.ch/pages/modul...hp?page=piquet
(temps : 00:04:46)

52:http://intranet.siesa.ch/pages/modul...ge=catastrophe
(temps : 00:04:51)

53:http://intranet.siesa.ch/pages/update/update.php
(temps : 00:04:57)
+ + + + + + +
54:http://intranet.siesa.ch/phpdig-1.8.7/search.php
(temps : 00:05:02)
+ + + + + + + + + + + + +
HTTP/1.1 403 Forbidden - http://www.phpdig.net/robots.txt
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.

HTTP/1.1 403 Forbidden - http://www.phpdig.net//
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.

HTTP/1.1 403 Forbidden - http://www.phpdig.net/
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.
Ok for http://www.phpdig.net/ (site_id:13)
+
55:http://intranet.siesa.ch/pages/smqe/index.php
(temps : 00:05:09)
+ +
56:http://intranet.siesa.ch/journal.php
(temps : 00:05:14)

niveau 3...
57:http://intranet.siesa.ch/pages/agend...nom=10-10-2005
(temps : 00:05:24)
+ + +
58:http://intranet.siesa.ch/pages/agend...nom=10-12-2005
(temps : 00:05:30)
+ + +
59:http://intranet.siesa.ch/pages/agend...nom=10-11-2004
(temps : 00:05:35)
+
60:http://intranet.siesa.ch/pages/agend...nom=10-11-2006
(temps : 00:05:41)
+
61:http://intranet.siesa.ch/pages/agend...&nom=9-10-2005
(temps : 00:05:46)
+ + +
62:http://intranet.siesa.ch/pages/agend...&nom=9-12-2005
(temps : 00:05:51)
+ + +
63:http://intranet.siesa.ch/pages/agend...&nom=9-11-2004
(temps : 00:05:57)
+
64:http://intranet.siesa.ch/pages/agend...&nom=9-11-2006
(temps : 00:06:02)
+
65:http://intranet.siesa.ch/pages/agend...&nom=6-10-2005
(temps : 00:06:08)
+ + +
66:http://intranet.siesa.ch/pages/agend...&nom=6-12-2005
(temps : 00:06:13)
+ + +
67:http://intranet.siesa.ch/pages/agend...&nom=6-11-2004
(temps : 00:06:18)
+
68:http://intranet.siesa.ch/pages/agend...&nom=6-11-2006
(temps : 00:06:24)
+
69:http://intranet.siesa.ch/pages/agend...&nom=7-10-2005
(temps : 00:06:29)
+ + +
70:http://intranet.siesa.ch/pages/agend...&nom=7-12-2005
(temps : 00:06:35)
+ + +
71:http://intranet.siesa.ch/pages/agend...&nom=7-11-2004
(temps : 00:06:40)
+
72:http://intranet.siesa.ch/pages/agend...&nom=7-11-2006
(temps : 00:06:45)
+
73:http://intranet.siesa.ch/pages/agend...&nom=8-10-2005
(temps : 00:06:51)
+ + +
74:http://intranet.siesa.ch/pages/agend...&nom=8-12-2005
(temps : 00:06:56)
+ + +
75:http://intranet.siesa.ch/pages/agend...&nom=8-11-2004
(temps : 00:07:02)
+
76:http://intranet.siesa.ch/pages/agend...&nom=8-11-2006
(temps : 00:07:07)
+
77:http://intranet.siesa.ch/pages/agend...&nom=4-10-2005
(temps : 00:07:12)
+ + +
78:http://intranet.siesa.ch/pages/agend...&nom=4-12-2005
(temps : 00:07:18)
+ + +
79:http://intranet.siesa.ch/pages/agend...&nom=4-11-2004
(temps : 00:07:23)
+
80:http://intranet.siesa.ch/pages/agend...&nom=4-11-2006
(temps : 00:07:29)
+
81:http://intranet.siesa.ch/pages/agend...&nom=5-10-2005
(temps : 00:07:34)
+ + +
gim is offline   Reply With Quote
Old 11-18-2005, 02:14 AM   #3
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Is it still like this:
Code:
The site where phpdig is ok is intranet.siesa.ch
The site where phpdig is install is intranet2.siesa.ch
and the site where phpdig isnt ok is intranet2.seisa.ch
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 11-25-2005, 05:20 AM   #4
gim
Green Mole
 
Join Date: May 2005
Location: swizterland
Posts: 5
Yes exactly
now when I change the /etc/hosts file I have only that result
SITE : http://intranet2.siesa.ch/
Chemins exclus :
- @NONE@
1:http://intranet2.siesa.ch/
(temps : 00:00:06)

Pas de liens dans la table temporaire
liens trouvés : 1
http://intranet2.siesa.ch/
Optimizing tables...
Indexation terminée !

the first index page is a index.php page ?
?????
gim is offline   Reply With Quote
Old 11-25-2005, 06:31 AM   #5
gim
Green Mole
 
Join Date: May 2005
Location: swizterland
Posts: 5
HI
I found the solution, on the index.php we have frames and phpdig don't find any link.
We have add a div with the link not on a frame with visibility: hidden and phpdig find all the necessary link.

Thank for all peoples in the forum for the articles.
gim is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Help Please: Problems to spider... just one site 1001studio Troubleshooting 1 06-28-2004 04:41 PM


All times are GMT -8. The time now is 01:09 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.