PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 10-13-2003, 06:23 PM   #16
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Quote:
Was recently indexed
Did it index the first time? Are you trying to reindex?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 10-13-2003, 06:30 PM   #17
vvvvv
Green Mole
 
Join Date: Oct 2003
Posts: 6
>Did it index the first time?

Nope. This is all I ever got and I've tried it on several different URLs.

SITE : http://www.mysite.com/
Exclude paths :
- @NONE@
No link in temporary table
vvvvv is offline   Reply With Quote
Old 10-13-2003, 06:42 PM   #18
mike221
Green Mole
 
Join Date: Oct 2003
Posts: 3
I had the same problem but it went away after performing the mods published Here .

My sever (Where phpdig is) : Apache/1.3.28 (Unix) mod_auth_passthrough/1.8 mod_gzip/1.3.26.1a mod_log_bytes/1.2 mod_bwlimited/1.0 PHP/4.3.3 FrontPage/5.0.2.2634 mod_ssl/2.8.15 OpenSSL/0.9.7a on Linux.

I still have problems indexing a couple of servers running Netscape out of 265 servers with all kind of configurations.

Good Luck

Last edited by mike221; 10-13-2003 at 06:48 PM.
mike221 is offline   Reply With Quote
Old 10-13-2003, 06:58 PM   #19
vvvvv
Green Mole
 
Join Date: Oct 2003
Posts: 6
OK I'll give that a spin. Thanks for the suggestion mike221

Looks like a late night cup of coffee for me.
vvvvv is offline   Reply With Quote
Old 10-13-2003, 07:33 PM   #20
vvvvv
Green Mole
 
Join Date: Oct 2003
Posts: 6
OK I did the mods but still the same.

Any other ideas? Again much appreciate the help.
vvvvv is offline   Reply With Quote
Old 10-14-2003, 08:32 AM   #21
rayvd
Green Mole
 
Join Date: Oct 2003
Location: Mesa, AZ
Posts: 15
You've probably already checked this... there's no robots.txt file on your server preventing the crawling is there?
rayvd is offline   Reply With Quote
Old 10-17-2003, 04:40 AM   #22
Tanasja
Green Mole
 
Join Date: Oct 2003
Location: Amsterdam
Posts: 9
Unhappy

Hi,

Think I have the same problem. The first time the indexing went fine. Then I changed some filenames. When reindexing, the old filenames were taken and the new ones skipped. Also when I index directly indexed the new filename the index couldn't find it.

I read the posts on this item and tried the following things:
- delete en reindex site (several ways)
- delete en reinstalling database
- empty dir text_content (not keepalive) and dir admin/temp (which stayed empty when reindexing)
- change config: LIMIT_DAYS=1 and PHPDIG_DEFAULT_INDEX=false
- run spider.php from a browser

I also see suggestions like:
- lynx from command line
- adjusting the routing table on the machine with the webserver

I don't understand these suggestions. Can anybody exlpain them? Or are there other options left? Maybe useful information: I host my sides at a provider.

Anybody can help?
Greetings from Amsterdam,
Tanasja
Tanasja is offline   Reply With Quote
Old 10-17-2003, 04:52 PM   #23
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi vvvvv. Maybe this is a JavaScript issue? Does setting PHPDIG_DEFAULT_INDEX to false have any effect?

Hi Tanasja. Just to be sure, when you say "the old filenames were taken and the new ones skipped" are the new links in the files you are trying to index?
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 05-19-2004, 01:35 PM   #24
web newsroom
Green Mole
 
Join Date: May 2004
Posts: 4
Same Problems

Im sure that its something quite simple.

I have been able to successfully spider certain sites and currently show the following stats.

Last Run : May 19, 2004
Pages : 5025 Entries
Index : 1397195 Entries
Keywords : 230416 Entries
Temporary : 110440 Entries


However, I still cannot seem to spider our own site.

A qualified subdomain.mydamain.com will work? I have changed the robots.txt and the .htaccess file and still stumped.
__________________
http://www.thewebnewsroom.com
web newsroom is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Trying to index some dynamic sites guillaume Troubleshooting 2 08-08-2007 06:40 AM
PHPDig won't index most sites and only go down one level on all confusion Troubleshooting 1 10-14-2005 11:32 AM
I just want to index main sites afesh How-to Forum 1 08-26-2005 09:45 PM
"I don't want to index your sites!!!" - said PHPDig #ASH How-to Forum 1 04-06-2005 02:57 PM
index intershop-sites? comko Troubleshooting 4 03-30-2004 09:22 AM


All times are GMT -8. The time now is 08:35 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.