PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > External Binaries

Reply
 
Thread Tools
Old 09-08-2004, 05:56 AM   #1
dell_10
Green Mole
 
Join Date: Sep 2004
Posts: 13
Question Spider site with links

Hi..
i need help ?????????????????????
1) I have 2 web sites and phpdig run on solaris machine when we indexing Server A (Windows 2000 Advances Server ) evry thing OK but whem im try index B (Windows 2003 Server) On this Server im only have folder includes MS word and PDF Files and only on HTML page this page include links to this documents and PDF Files ,,, i'm Try to index this page http://serverB/Folder/
got only fisrt page indexed and No Links Found

Im posting my COnfig File and wish any one can help me

2) i need to know if there is and hosting companies support phpdig so i can put phpdig there thanks
Attached Files
File Type: txt config.txt (18.6 KB, 30 views)

Last edited by dell_10; 09-08-2004 at 06:00 AM.
dell_10 is offline   Reply With Quote
Old 09-10-2004, 10:17 PM   #2
dell_10
Green Mole
 
Join Date: Sep 2004
Posts: 13
Whta ???????????
Any one can help me or what ?????
dell_10 is offline   Reply With Quote
Old 09-11-2004, 04:33 AM   #3
dell_10
Green Mole
 
Join Date: Sep 2004
Posts: 13
i need help ?????????????????????

Hi..
i need help ?????????????????????
1) I have 2 web sites and phpdig run on solaris machine when we indexing Server A (Windows 2000 Advances Server ) evry thing OK but whem we try index Server B (Windows 2003 Server) [ On this Server we only have folder includes MS word and PDF Files and only on HTML page this page include links to this documents and PDF Files ,,, We Try to index this page http://serverB/Folder/page.htm
got only fisrt page indexed and No Links Found
Im posting our Config File and wish any one can help me

2) i need to know if there is any hosting company support phpdig so i can put phpdig there thanks
Attached Files
File Type: txt config.txt (18.6 KB, 16 views)
dell_10 is offline   Reply With Quote
Old 09-11-2004, 06:01 AM   #4
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
1) http://www.phpdig.net/forum/showthread.php?t=799

2) PhpDig can run on a lot of, but not all, remote servers
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 09-11-2004, 06:49 AM   #5
dell_10
Green Mole
 
Join Date: Sep 2004
Posts: 13
Dear ...
i don't have any problem with using external binaries my problem is when im trying to index another server (.NEt Server) include folder contain sub folders this sub folder contain files (word, xls, pdf ) and the main folder contain html page with links to this files when im start index this server and put the path to this html page i think phpdig will spider and follow links frm this page and index filse .. i try it on another server (2000 Advance server) and it's work but i can't make it done on this .NET Server the result only indexing of the page without any links
and when im try to index any file bu put the full path of the file phpdig can index it
dell_10 is offline   Reply With Quote
Old 09-11-2004, 07:35 AM   #6
asanad
Green Mole
 
Join Date: Sep 2004
Posts: 11
Question I have the same problem

Dear all:

I am a new commer to this forum, I really like PHPDig very much.
Still I have a problem in one of my servers, whenever I try to crawl that specific Server, PHPDig just crawls the first page then it stops going deeper in the levels. It simply gives me "No temporary table links" or something like that.
For example, I have a document called "doc1.txt" its under the folder "test" in "Server A". When I tell PHPDig to index "http://ServerA/test/doc1.txt" ot indexes the file. Now, i need to index all the documents, so I created a file called "index.html", this file caontains a link to all the documents under the folder "test" , for example, "doc1.txt, doc2.txt,doc3.htm,doc3.pdf, doc4.doc". PHPDig for some reason indexes only the "index.html", it does not go and crawl the links within the page.

Could you please give me a hint, why this is happening.
many thanks for your help and support.
asanad is offline   Reply With Quote
Old 09-11-2004, 07:50 AM   #7
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
If PhpDig can index the HTML page but won't follow the links and index the DOC, XLS, and PDF files, then that seems to indicate an external binary problem on the one server.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 09-11-2004, 08:05 AM   #8
asanad
Green Mole
 
Join Date: Sep 2004
Posts: 11
Question I forgot to mention this point

Thank you for your quick reponse,

I forgot to mention to you that PHPDig is residing on a Unix server, it is able to completely index many other servers sucessfully. This case is only for a specific server.

So, all the binaries and conversions work on the PHPDig Unix Server, it is just the issue that one of the destination servers does not crawl properly.

Regards
asanad is offline   Reply With Quote
Old 09-11-2004, 08:23 AM   #9
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Okay, that changes my response.

Some thoughts: check the destination server request and error logs, and also check the MIME type mappings on the destination server.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 09-12-2004, 03:56 PM   #10
dell_10
Green Mole
 
Join Date: Sep 2004
Posts: 13
Lightbulb Only Crawl Some Files.

Hi
My problem is:
i have 3 servers: Unix(Phpdig Server),2 windows Servers (A,B)
PhpDig crawl Server A completley But it's only crawl & index 7 Pages on Server B, This server contain more than 500 pages.
** No Problems with conversion tools like catdoc ...
Any idea please
dell_10 is offline   Reply With Quote
Old 09-20-2004, 06:42 AM   #11
asanad
Green Mole
 
Join Date: Sep 2004
Posts: 11
I have checked the request and MIME type, it seems normal , does PHPDig have certain settings for the mime type on the server ?
asanad is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
spider ignores links Maarten Wijnen Troubleshooting 2 03-17-2005 02:23 PM
Spider External links to a depth of 1 (1.8.3) kenazo How-to Forum 0 10-20-2004 06:28 AM
specific links not spider flanders How-to Forum 1 10-06-2004 11:12 PM
no spider my file links lolodev Troubleshooting 21 07-16-2004 06:31 PM
cannot spider own site - others yes web newsroom Troubleshooting 1 05-23-2004 03:32 PM


All times are GMT -8. The time now is 10:47 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.