Anbody else can't get it to spider Miva Merchant files?

[dreamingdigital]

I can't get it to spider a Miva Merchant Web site. It will, however, spider an individual page if I give it the exact URL. Example: http://www.violinshop.ca/miva/merchant.mvc?Screen=CTGY&Category_Code=_HOME but it will not follow the links from that page and contiinue. It works fine on PHP and HTML sites. Just wondering if anybody had any luck spidering a Miva site?

Colin

[Dave A]

Hi most of what I can see that may cause that to happen is that it looks like the site may have java applets instead of html links.
Most webspiders can't follow Java links..
A heap of the site seems to be Java based so that may be the problem..

[dreamingdigital]

Well it's Miva Merchant. So I don't know if that's Java or not but it's dynamically DB and Miva engine generated Web pages.

PHP and ASP generated pages work with PHPDIG but not Miva I guess. I thought Miva was XML but who knows.

CP

[Dave A]

I tell you what?
I will try and index the site from my end in a few minutes and let you know what happens, maybe this will shed some light on it.

[dreamingdigital]

You're the best! :)

I can get it to index individual pages if I explicitly tell it what page to index. But I can't get it to follow any links and index the site. The root index page has links to pretty much all the root categories so just setting http://www.violinshop.ca/ as the site to index "should" be enough. ;) Good luck. You rock.

CP

[Dave A]

Okay having played with it my self I can see what might be a problem?
The source code shows the menus at the top of the screen in odd colours, I have to move the nouse over them to change the colour from balck to white and at first they are odd symbols until it sorts itself out.

The source for that reads as follows:
<TABLE WIDTH=636 BORDER=0 CELLPADDING=0 CELLSPACING=0>
<TR>
<TD>
<A HREF="http://www.violinshop.ca/miva/merchant.mvc?Screen=CTGY&Category_Code=_HOME"
ONMOUSEOVER="DD_images_to_normal(); changeImages('home', 'images/layout/home-over.gif'); dd_show_selects(); MM_showHideLayers('hider1','','hide', 'hider2','','hide', 'divs_instruments','','hide', 'divs_instruments_beginners','','hide', 'divs_strings','','hide', 'divs_strings_violin','','hide', 'divs_strings_viola','','hide', 'divs_strings_cello','','hide', 'divs_strings_bass','','hide', 'divs_cases','','hide', 'divs_access','','hide', 'divs_access_stands','','hide', 'divs_access_endpins','','hide', 'divs_music_cds','','hide', 'divs_gifts','','hide', 'divs_music','','hide', 'divs_music_violin','','hide', 'divs_music_cello','','hide', 'divs_books','','hide', 'divs_specials','','hide', 'divs_repair','','hide', 'divs_rental','','hide', 'divs_faqs','','hide', 'divs_contact','','hide');"
ONMOUSEOUT="changeImages('home', 'images/layout/home.gif');">
<IMG NAME="home" SRC="images/layout/home.gif" WIDTH=56 HEIGHT=21 BORDER=0></A></TD>
<TD>
<A HREF="http://www.violinshop.ca/miva/merchant.mvc?Screen=CTGY&Category_Code=GAME"
ONMOUSEOVER="DD_images_to_normal(); changeImages('games', 'images/layout/games-over.gif'); dd_show_selects(); MM_showHideLayers('hider1','','hide', 'hider2','','hide', 'divs_instruments','','hide', 'divs_instruments_beginners','','hide', 'divs_strings','','hide', 'divs_strings_violin','','hide', 'divs_strings_viola','','hide', 'divs_strings_cello','','hide', 'divs_strings_bass','','hide', 'divs_cases','','hide', 'divs_access','','hide', 'divs_access_stands','','hide', 'divs_access_endpins','','hide', 'divs_music_cds','','hide', 'divs_gifts','','hide', 'divs_music','','hide', 'divs_music_violin','','hide', 'divs_music_cello','','hide', 'divs_books','','hide', 'divs_specials','','hide', 'divs_repair','','hide', 'divs_rental','','hide', 'divs_faqs','','hide', 'divs_contact','','hide');"
ONMOUSEOUT="changeImages('games', 'images/layout/games.gif');">

<IMG NAME="games" SRC="images/layout/games.gif" WIDTH=52 HEIGHT=21 BORDER=0></A></TD>
<TD>
<A HREF="http://www.violinshop.ca/miva/merchant.mvc?Screen=CTGY&Category_Code=_REPAIR"
ONMOUSEOVER="DD_images_to_normal(); changeImages('repair', 'images/layout/repair-over.gif'); dd_hide_selects(); MM_showHideLayers('hider1','','show', 'hider2','','show', 'divs_instruments','','hide', 'divs_instruments_beginners','','hide', 'divs_strings','','hide', 'divs_strings_violin','','hide', 'divs_strings_viola','','hide', 'divs_strings_cello','','hide', 'divs_strings_bass','','hide', 'divs_cases','','hide', 'divs_access','','hide', 'divs_access_stands','','hide', 'divs_access_endpins','','hide', 'divs_music_cds','','hide', 'divs_gifts','','hide', 'divs_music','','hide', 'divs_music_violin','','hide', 'divs_music_cello','','hide', 'divs_books','','hide', 'divs_specials','','hide', 'divs_repair','','show', 'divs_rental','','hide', 'divs_faqs','','hide', 'divs_contact','','hide');"
ONMOUSEOUT="changeImages('repair', 'images/layout/repair.gif');">
<IMG NAME="repair" SRC="images/layout/repair.gif" WIDTH=49 HEIGHT=21 BORDER=0></A></TD>

That appears to show the wrong graphics and the menu bars don't drop down cleanly to show the links, so this is where the webspider is falling over.

The index has continued but it was very very slow, in the end it found and indexed

the URI : http://www.violinshop.ca/ (8 pages)
Root
/miva
in the root area it found one file 1 pages
In the miva directory it found 7 pages
7 pages
-merchant.mvc?Screen=CTGY&Category_Code=_BOWS
-merchant.mvc?Screen=CTGY&Category_Code=_CASES
-merchant.mvc?Screen=CTGY&Category_Code=_CONSIGNMENT
-merchant.mvc?Screen=CTGY&Category_Code=_MUSIC
-merchant.mvc?Screen=CTGY&Category_Code=_RENTAL
-merchant.mvc?Screen=CTGY&Category_Code=_REPAIR
-merchant.mvc?Screen=CTGY&Category_Code=_STRINGS


So it will get to it in the end but it did take about twenty minutes (The spider is still working)

When you start to index it set the depth to eight and eight links per level and it should take a while but it appears to get there in the end.

I will leave the links inot your website running live, that way if anyone searches for any of the content then they will give you a visit.

Try it set at eight and eight and wait a while it will do it in the end..


Heaps of regards
Dave Andrews
From the Linknz search engine.