PhpDig.net

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)
-   -   only indexes the first page... (http://www.phpdig.net/forum/showthread.php?t=792)

majestique 04-08-2004 03:12 PM

only indexes the first page...
 
when i run "dig this!" it only indexes the index.html page.. it doesn't even go to the links w/in my index page.. anyone know why this is happening? thanks a lot! =)

vinyl-junkie 04-08-2004 04:56 PM

What search depth did you select? If you left it at the default, that's probably the problem. Try changing the search depth to 5 and see what happens.

Hope this helps. :) BTW, welcome to the forums, majestique. We're glad to have you here.

majestique 04-08-2004 05:48 PM

thanks for the welcome :) this script is the best search script .. and i've definitely looked ALL over.. anyway.. i tried setting it to 4 and 5.. and it would still do give me only 1 page.. so what i did was enter in a sub folder within the site.. and that worked beautifully.. the weird thing was.. it indexed my wmv files!? what the heck.. anyway i just went ahead and deleted that.. and entered in the videos folder in my dissallow in robots.txt..

also when i clicked on dig this.. the page seemingly halted.. but it isn't because when i close it.. and enter in the admin area.. i notice my #s going up as I refresh.. shouldn't the temporary table be 0? it's currently stuck at
"Temporary table : 79 Entries "

here's what my database status looks like:
Hosts : 1 Entries
Pages : 36 Entries
Index : 4217 Entries
Keywords : 145076 Entries
Temporary table : 79 Entries

vinyl-junkie 04-08-2004 06:06 PM

The temporary table should be empty when you have finished with the spidering process. Sometimes the process hangs for no good reason, and I have yet to figure out why. I've had that happen a couple of times, but when I started over, it worked like a charm. Go figure.

You might try emptying the phpDig tables and starting over. Before you re-index though, make sure the value for LIMIT_DAYS is set to zero so you'll be able to do that.

If you still have problems, let us know. We'll be glad to help. :)

majestique 04-08-2004 07:02 PM

thanks for the fast reply Pat =) I did exactly what u said.. here's what i got.. strange problem huh? i've setup this search engine before.. no problem.. odd..

--------------------------------------------------------------------------------
SITE : http://www.mysite.com/
Exclude paths :
- video
1:http://www.mysite.com/
(time : 00:00:06)
No link in temporary table

--------------------------------------------------------------------------------

links found : 1
http://www.mysite.com/
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

vinyl-junkie 04-08-2004 07:18 PM

Hmmm. Sounds like you have something in your robots.txt file that is blocking everything. Either that or did you use:
Code:

<!-- phpdigExclude -->
<!-- phpdigInclude -->

comments to exclude any links? That's another thing that could potentially trip you up.

majestique 04-08-2004 07:22 PM

yeah u were exactly right =) i fixed the robot.. and i'm back to that old problem again

majestique 04-08-2004 07:29 PM

i think i got it now.. i did everything from scratch... now it seems to work =) thanks Pat!!!

vinyl-junkie 04-08-2004 07:34 PM

Kewl! Glad you got it working! :D


All times are GMT -8. The time now is 08:51 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.