|
Hi. It seems that it takes PhpDig about ten minutes, depending on machines, traffic, etcetera, to process through the Keyword QuickFind links. Below is output from crawling http://www.posterbreak.com/art/ at a search depth of one. PhpDig hits the Keyword QuickFind links first and after it gets through those it hits the other links. Try setting a search depth of one and let the spider run for a while. What do you get?
SITE : http://www.posterbreak.com/
Exclude paths :
- _private/
- cgi-bin/
- images/
- search/
- searchsite/
- templates/
1:http://www.posterbreak.com/art/
(time : 00:00:13)
+ + + + + + + + + + + + + + +
level 1...
2:http://www.posterbreak.com/art/c5964-museum-landscapes.shtml
(time : 00:11:05)
3:http://www.posterbreak.com/art/c101318251-museum-religious-art.shtml
(time : 00:11:17)
4:http://www.posterbreak.com/art/c6461-museum-still-life.shtml
(time : 00:11:28)
5:http://www.posterbreak.com/art/c101310996-museum-tours.shtml
(time : 00:11:41)
6:http://www.posterbreak.com/art/c101319580-special-mediums.shtml
(time : 00:11:52)
7:http://www.posterbreak.com/privacy.shtml
(time : 00:11:59)
8:http://www.posterbreak.com/art/c10135861-art-by-nationality.shtml
(time : 00:12:10)
9:http://www.posterbreak.com/art/c101319277-four-centuries.shtml
(time : 00:12:27)
10:http://www.posterbreak.com/art/c101319312-museum-floral.shtml
(time : 00:12:38)
11:http://www.posterbreak.com/art/c101319275-museum-figurative.shtml
(time : 00:12:49)
12:http://www.posterbreak.com/art/c101319282-museum-artists.shtml
(time : 00:12:59)
13:http://www.posterbreak.com/art/c101319594-museum-abstract.shtml
(time : 00:13:10)
14:http://www.posterbreak.com/art/c101319274-art-movements.shtml
(time : 00:13:22)
15:http://www.posterbreak.com/contact.shtml
(time : 00:13:33)
16:http://www.posterbreak.com/
(time : 00:13:39)
No link in temporary table
--------------------------------------------------------------------------------
links found : 16
http://www.posterbreak.com/art/
http://www.posterbreak.com/art/c5964-museum-landscapes.shtml
http://www.posterbreak.com/art/c101318251-museum-religious-art.shtml
http://www.posterbreak.com/art/c6461-museum-still-life.shtml
http://www.posterbreak.com/art/c101310996-museum-tours.shtml
http://www.posterbreak.com/art/c101319580-special-mediums.shtml
http://www.posterbreak.com/privacy.shtml
http://www.posterbreak.com/art/c10135861-art-by-nationality.shtml
http://www.posterbreak.com/art/c101319277-four-centuries.shtml
http://www.posterbreak.com/art/c101319312-museum-floral.shtml
http://www.posterbreak.com/art/c101319275-museum-figurative.shtml
http://www.posterbreak.com/art/c101319282-museum-artists.shtml
http://www.posterbreak.com/art/c101319594-museum-abstract.shtml
http://www.posterbreak.com/art/c101319274-art-movements.shtml
http://www.posterbreak.com/contact.shtml
http://www.posterbreak.com/
Optimizing tables...
Indexing complete !
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
|