PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 03-23-2004, 10:28 AM   #1
comko
Green Mole
 
Join Date: Mar 2004
Posts: 2
Post index intershop-sites?

Hi folk,

i'm successfull installed phpdig on my linux server (LAMP) - great work, guys!!! thanks for it.
no probs with the most sites, i will search for.
but, only one won't: www.junfermann.de is the site, generated with INTERSHOP, and that's my problem:
all pages where contained, are WITHOUT any "pages". it's true, e.g. one page is : http://www.junfermann.de/cgi-bin/junfermann.storefront/DE/Catalog/1011/
Q: how can i index this site?

thanks a lot for help:
Ingo
__________________
--
my name is dump, core dump
comko is offline   Reply With Quote
Old 03-24-2004, 03:15 PM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. First, download the ZIP file in this thread and replace robot_functions.php with the one in the ZIP file.

Next, in the new robot_functions.php file, search for "<frame" (without the quotes) and on this line add in a [[:blank:]]* so that:
PHP Code:
while (blah blah *content=['\"][0-9]+;url blah blah) { 
becomes the following:
PHP Code:
while (blah blah *content=['\"][0-9]+;[[:blank:]]*url blah blah) { 
Last, the server for that site does not return a content-type for certain pages. You would need to force the content-type for these pages so, in the new robot_functions.php file, search for:
PHP Code:
if (!eregi('[a-z0-9]+',$answer)) { 
and right before that line add:
PHP Code:
// THIS CODE IS ONLY FOR WHEN CONTENT-TYPE IS NOT RETURNED
// IT IS NOT FOR GENERAL INCLUSION IN THE CORE PHPDIG CODE
elseif (!eregi("Content-Type: *([a-z]+)/([a-z.-]+)",$answer,$regs)) {
   
$status 'HTML'// no content-type so set to html

Remember to remove any "word" wrapping in the above code.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 03-24-2004, 11:33 PM   #3
comko
Green Mole
 
Join Date: Mar 2004
Posts: 2
Yikes, it works!!
thanks for geat work!

Ingo
__________________
--
my name is dump, core dump
comko is offline   Reply With Quote
Old 03-30-2004, 06:09 AM   #4
malieut
Green Mole
 
Join Date: Mar 2004
Posts: 1
hello Charter,
I followed your suggestion,but got this screen as the following when i click the dig this button.

------------------------------
Spidering in progress...

------------------------------
Nothing happened else, the spidering is end.
malieut is offline   Reply With Quote
Old 03-30-2004, 08:22 AM   #5
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Recheck the mods and make sure to refresh the admin/index.php page before indexing. Only apply the last bit of code if a content type is not returned, which is generally not the case.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Trying to index some dynamic sites guillaume Troubleshooting 2 08-08-2007 05:40 AM
PHPDig won't index most sites and only go down one level on all confusion Troubleshooting 1 10-14-2005 10:32 AM
I just want to index main sites afesh How-to Forum 1 08-26-2005 08:45 PM
"I don't want to index your sites!!!" - said PHPDig #ASH How-to Forum 1 04-06-2005 01:57 PM
Some sites won't index jalerta Troubleshooting 23 05-19-2004 12:35 PM


All times are GMT -8. The time now is 06:06 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.