PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 04-30-2004, 06:01 PM   #1
tryangle
Green Mole
 
Join Date: Jan 2004
Location: Penns Woods
Posts: 17
Question Nothing indexed on some sites

Hi,

Sometimes I get output like this after I try to index a site:

SITE : http://www.domain.tld/
Exclude paths :
- @NONE@
1:http://www.domain.tld/
(time : 00:00:47)
No link in temporary table

I checked the source of the site's index page and found this:

<meta name="robots" content="index,follow">

How come I don't get anything though?

TIA for any info on this...

Last edited by tryangle; 04-30-2004 at 06:06 PM.
tryangle is offline   Reply With Quote
Old 04-30-2004, 07:27 PM   #2
vinyl-junkie
Purple Mole
 
Join Date: Jan 2004
Posts: 694
What search depth did you choose? If you left the default at zero, phpDig will only spider the root.
vinyl-junkie is offline   Reply With Quote
Old 05-02-2004, 10:22 AM   #3
tryangle
Green Mole
 
Join Date: Jan 2004
Location: Penns Woods
Posts: 17
Search Depth

Hi vinyl-junkie,

Thanks for your reply.

I've tried a search depth of 1 or 2. But it didn't matter, I got the same results... even though they are not one page sites... and they've got lots of internal and external links. What baffles me is that the information on the home page is not even getting indexed, even though there is this line in their source:

<meta name="robots" content="index,follow">

Fortunately I've only gotten this on just a few sites, but I'm wondering what would prevent a site from getting spidered... when it appears to be inviting it.
tryangle is offline   Reply With Quote
Old 05-02-2004, 01:31 PM   #4
vinyl-junkie
Purple Mole
 
Join Date: Jan 2004
Posts: 694
Perhaps this thread might be of some help.
vinyl-junkie is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
List how many Sites have been indexed? Dan Mod Requests 1 11-17-2006 07:00 AM
Certain sites, pages and pdfs are not indexed 1.8.9 RC1 [Workaround included] obottek Bug Tracker 0 08-24-2006 06:30 AM
show sites indexed richwilson How-to Forum 2 04-06-2006 05:31 PM
Meta Robots = NoIndex, or already indexed : No content indexed jerrywin5 How-to Forum 2 04-06-2005 03:50 PM
Many sites indexed - how to restrict search results to just one or two ? ciaran@clissman How-to Forum 2 05-27-2004 02:11 AM


All times are GMT -8. The time now is 02:25 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.