PDA

View Full Version : digging subdomains


b2l_grefix
05-06-2004, 10:46 PM
Hi,

is it possibile to set up phpdig to also scan subdomains?

Just to clarify a bit:
When I start digging http://somewhere.someplace.com, phpdig doesn't recognize http://here.somewhere.someplace.com as a part of the site.

Any way to make this possible?

(I've tried to search this forum but couldn't find the answer, so my apoligies if this kind of thead allready exists)

bloodjelly
05-07-2004, 01:18 AM
If you have a link to "http://here.somewhere.someplace.com" on "http://somewhere.someplace.com" phpDig will find it.

b2l_grefix
05-10-2004, 01:52 AM
Well, it sure doesn't seem to.

I tried it severall times, but phpdig doesn't seem to recognize it as a part of a larger domain

vinyl-junkie
05-10-2004, 05:38 AM
Did phpdig only index the root? If so, you need to specify a search depth other than zero. Or maybe it's a completely different problem.

Is the top level domain publicly accessible? If so, post the URL you're trying to index. That might give us a better clue as to what's going on.

b2l_grefix
05-10-2004, 08:21 AM
Well, it's certainly tnot the search depth (I always set it on 2 levels)

The main site is: http://grefix.buildtolearn.com

it has an index page with 2 links
1. http://digitalart.grefix.buildtolearn.com
2 http://concinite.grefix.buildtolearn.com

vinyl-junkie
05-10-2004, 12:30 PM
I'll be the first to admit that I don't really know how indexing subdomains, works. However, I was able to get
http://digitalart.grefix.buildtolearn.com/
to index about 35 pages or so.

Maybe if you index those subdomains individually, you'll end up with what you want. Just a thought.

b2l_grefix
05-10-2004, 02:34 PM
well, that' would be an option if I would only dig my own sites. But since I spider several sites wich aren't mine, I don't know if subdomains are present or not, and manually checking them would be an enormous task :)

Guess I'll gonna have to walk through the script myself to adjust it a bit