PhpDig.net
Page 1 of 2 1 2 >
Show 40 post(s) from this thread on one page

PhpDig.net (http://www.phpdig.net/forum/index.php)
-   Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)
-   -   How do i index my test server. (http://www.phpdig.net/forum/showthread.php?t=741)

silverfish 03-29-2004 11:12 AM
How do i index my test server.
 
I want to index my localhost, which is just my ip right now, as it is a test server.

Ever time i try to index my IP, one level in.. It jsut indexes forever.. saying LOCKED over and over.. it never stops, and it isnt putting anything into the database.

If anyone has successfully indexed a folder in their localhost, please let me know how to do it!

Silver

Charter 03-29-2004 12:37 PM
Hi. Perhaps something in this thread might help.

silverfish 03-29-2004 12:39 PM
Hmm...

No... See im behind a firewall, but so is my server...So im just indexing locally...

It just says locked, but nothing is being written to the db, yet it loads and loads..

Charter 03-29-2004 12:50 PM
Hi. Is anything getting into the tempspider table?

silverfish 03-29-2004 12:51 PM
Ok..
i changed the 'USE_IS_EXECUTABLE_COMMAND' to 0 and it seemed to work in indexing..

But it wont spider.. it just indexed the first page, thats it..

Charter 03-29-2004 01:09 PM
Hi. What search depth are you using?

silverfish 03-29-2004 01:10 PM
It doesnt seem to matter...

I go from 1 all the way to 20 and it still only returns the first page.

:(

Charter 03-29-2004 01:18 PM
Hi. Perhaps try the suggestions in this thread, and if there is a base href tag, then try the code in this post.

silverfish 03-29-2004 01:37 PM
Still only indexes the first page.

:(

Charter 03-29-2004 01:47 PM
Hi. Try uncommenting //print $answer."<br>\n"; in robot_functions.php and see what you get on index.

silverfish 03-29-2004 01:50 PM
This is what i get when I spider:


Spidering in progress...

--------------------------------------------------------------------------------
SITE : http://webdev/
Exclude paths :
- @NONE@
1:http://webdev/mambo/
(time : 00:00:08)
No link in temporary table

--------------------------------------------------------------------------------

links found : 1
http://webdev/mambo/
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

silverfish 03-29-2004 01:54 PM
If i can get this to work, I can post to the whole mambo community about this engine.. :)

Charter 03-29-2004 02:01 PM
Hi. Doesn't Mambo already have PhpDig integrated? I thought it did, but maybe I'm just confused.

Anyway, what did you get when you uncommented //print $answer."<br>\n"; in the robot_functions.php file?

silverfish 03-29-2004 02:04 PM
Opps...

here is what I got:

HTTP/1.1 404 Not Found
HTTP/1.1 200 OK
Date: Mon, 29 Mar 2004 23:03:34 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=3e3025a85c2bba6ea08f8512c863dc2c; expires=Tue, 30-Mar-2004 11:03:34 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Mon, 29 Mar 2004 23:03:34 GMT
Content-Type: text/html

HTTP/1.1 200 OK
Date: Mon, 29 Mar 2004 23:03:35 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=1ede113f1f6232445c38b9a20237c08f; expires=Tue, 30-Mar-2004 11:03:35 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Mon, 29 Mar 2004 23:03:35 GMT
Content-Type: text/html

HTTP/1.1 404 Not Found

--------------------------------------------------------------------------------
SITE : http://webdev/
Exclude paths :
- @NONE@
HTTP/1.1 200 OK
Date: Mon, 29 Mar 2004 23:03:40 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=1ffb0f6dac1a34f53e5f5b7da5899104; expires=Tue, 30-Mar-2004 11:03:40 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Mon, 29 Mar 2004 23:03:40 GMT
Content-Type: text/html

1:http://webdev/mambo/
(time : 00:00:08)
No link in temporary table

--------------------------------------------------------------------------------

links found : 1
http://webdev/mambo/
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

Charter 03-29-2004 02:23 PM
Hi. What happens when you add one of the following to your hosts file?

127.0.0.1 localhost
127.0.0.1 webdev/
127.0.0.1 webdev/mambo/

# or maybe this:

127.0.0.1 localhost
127.0.0.1 webdev.com/
127.0.0.1 webdev.com/mambo/

# or maybe this:

127.0.0.1 localhost
127.0.0.1 www.webdev.com/
127.0.0.1 www.webdev.com/mambo/

silverfish 03-29-2004 02:36 PM
I am using apache with windows.

silverfish 03-29-2004 02:49 PM
I cant index my other site outside this server.. but i cannot index my localhost site past the first page..

:(

Arghhh!!

Mike

silverfish 03-29-2004 03:04 PM
Hmm i edited the hosts file, didnt seem to do much....

:(

silverfish 03-29-2004 03:08 PM
Hmm i edited the hosts file, didnt seem to do much....

:(
Spidering in progress...
HTTP/1.1 404 Not Found
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 00:07:23 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=6f5949a767040f2205d08dd10ef22138; expires=Tue, 30-Mar-2004 12:07:23 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 00:07:23 GMT
Content-Type: text/html

HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 00:07:23 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=7884431717962635e6edaf2e0de86858; expires=Tue, 30-Mar-2004 12:07:24 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 00:07:24 GMT
Content-Type: text/html

HTTP/1.1 404 Not Found

--------------------------------------------------------------------------------
SITE : http://localhost/
Exclude paths :
- @NONE@
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 00:07:29 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=c6d9f45c7fee53937c186d09fe4aee4f; expires=Tue, 30-Mar-2004 12:07:29 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 00:07:29 GMT
Content-Type: text/html

1:http://localhost/mambo/
(time : 00:00:08)
No link in temporary table

--------------------------------------------------------------------------------

links found : 1
http://localhost/mambo/
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

Charter 03-29-2004 03:21 PM
Hi. Is there an index page in the mambo directory, or are you trying to index the directory content?

silverfish 03-29-2004 03:23 PM
The content is created dynamically.. but the spider should spider the links.

Charter 03-29-2004 08:15 PM
Hi. It looks like PhpDig isn't recognizing any links from the page, but I'm not sure why. :( What happens if you enter the site from a different location? What gets indexed then?

Charter 03-30-2004 06:12 AM
Hi. Here's another thing to try.

In spider.php find:
PHP Code:
if (isset($lien['newhost'])) { 
and afterwards place:
PHP Code:
echo "New Url: http://".$lien['newhost'].$lien['path'].$lien['file']."<br>";
echo "Old Url: ".$url."<br><br>"; 
When you index, what do you get?

silverfish 03-30-2004 07:47 AM
This is what I get:

Spidering in progress...
HTTP/1.1 404 Not Found
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 16:42:54 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=2c5587f668574f2f226b5414477a485a; expires=Wed, 31-Mar-2004 04:42:54 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 16:42:54 GMT
Content-Type: text/html

HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 16:42:55 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=8fa61cdab071b4d8ae778fb946bab4aa; expires=Wed, 31-Mar-2004 04:42:55 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 16:42:55 GMT
Content-Type: text/html

HTTP/1.1 404 Not Found

--------------------------------------------------------------------------------
SITE : http://webdev/
Exclude paths :
- @NONE@
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 16:43:01 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=58dd11d18fa05c61810a8b2e3a909c69; expires=Wed, 31-Mar-2004 04:43:01 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 16:43:01 GMT
Content-Type: text/html

1:http://webdev/mambo/
(time : 00:00:08)
No link in temporary table

--------------------------------------------------------------------------------

links found : 1
http://webdev/mambo/
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

silverfish 03-30-2004 08:23 AM
Hmm it found 9 links.. not suire why only nine...


Spidering in progress...
HTTP/1.1 404 Not Found
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:32 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=a87906bd18548ebbec948831c7b706c8; expires=Wed, 31-Mar-2004 05:08:32 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 17:08:32 GMT
Content-Type: text/html

HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:33 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=cd87162a1bfaa95bf6605be2f3fa3710; expires=Wed, 31-Mar-2004 05:08:33 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 17:08:33 GMT
Content-Type: text/html

HTTP/1.1 404 Not Found

--------------------------------------------------------------------------------
SITE : http://webdev/
Exclude paths :
- @NONE@
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:38 GMT
Server: Apache/1.3.29 (Win32)
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=1fc9a1d3dba9d95f30b4712b2de765fb; expires=Wed, 31-Mar-2004 05:08:38 GMT; path=/
Set-Cookie: mosvisitor=1
Last-Modified: Tue, 30 Mar 2004 17:08:38 GMT
Content-Type: text/html

1:http://webdev/mambo/
(time : 00:00:08)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:40 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:40 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:40 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=304011bcdee116fbd36707af98a36ed9; expires=Wed, 31-Mar-2004 05:08:40 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:40 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=5d335223ad6d46ec0c4d11d0ca6bc08c; expires=Wed, 31-Mar-2004 05:08:40 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:40 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=844ff72a35830e71c4a898576ce94115; expires=Wed, 31-Mar-2004 05:08:40 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:40 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=4b39d7aa40ed6b36f57183ac9a0c9fc5; expires=Wed, 31-Mar-2004 05:08:40 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:40 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=0a672831a7d09ca2ed4fac184fc86b3c; expires=Wed, 31-Mar-2004 05:08:40 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ New Url: http://www.200interchange.com/updates/update_02_04.html
Old Url: http://webdev/

New Url: http://www.200interchange.com/updates/update_02_04.html
Old Url: http://webdev/

New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


level 1...
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:50 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=b38bcc92d2841d104ff4a4776996cf8f; expires=Wed, 31-Mar-2004 05:08:50 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

2:http://webdev/mambo/?option=content&...&id=1&Itemid=2
(time : 00:00:20)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:52 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:52 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:52 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=2b1e654f67940b3c193c7495dddbb495; expires=Wed, 31-Mar-2004 05:08:52 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:52 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=dd57a856984a8e9cec3ec69a66571390; expires=Wed, 31-Mar-2004 05:08:52 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:52 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=269c8d06d9a366ed545d64fe6c76ba7f; expires=Wed, 31-Mar-2004 05:08:52 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

+ New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:57 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=7740cc0f4f0e4c3a25571037d7998fcd; expires=Wed, 31-Mar-2004 05:08:57 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

Duplicate of an existing document
3:http://webdev/mambo/?option=content&...&id=3&Itemid=2
(time : 00:00:26)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:59 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:08:59 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:04 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=fe19bc9dc3907b2069956fcc69a42974; expires=Wed, 31-Mar-2004 05:09:04 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

4:http://webdev/mambo/?option=content&...w&id=4&Itemid=
(time : 00:00:33)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:05 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:05 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:10 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=47b90cb0be7fa8f6d576c9c36839f0af; expires=Wed, 31-Mar-2004 05:09:10 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

Duplicate of an existing document
5:http://webdev/mambo/?option=content&...w&id=5&Itemid=
(time : 00:00:39)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:11 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:11 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:16 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=6bf30fcd99967c95c10e2d02d4460b67; expires=Wed, 31-Mar-2004 05:09:16 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

Duplicate of an existing document
6:http://webdev/mambo/?option=content&...w&id=8&Itemid=
(time : 00:00:45)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:17 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:17 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


level 2...
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:27 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=edad5cb9e902a30b0b0f6cef927309b9; expires=Wed, 31-Mar-2004 05:09:27 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

Duplicate of an existing document
7:http://webdev/mambo/?option=content&...w&id=7&Itemid=
(time : 00:00:56)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:28 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:28 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:33 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=f48dc83fa28d8c6ad50afe12fd5965d2; expires=Wed, 31-Mar-2004 05:09:33 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

8:http://webdev/mambo/?option=com_regi...k=lostPassword
(time : 00:01:02)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:35 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:35 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:40 GMT
Server: Apache/1.3.29 (Win32)
X-Powered-By: PHP/4.3.4
Set-Cookie: sessioncookie=bf329c036413e5e4f1c8d10f8515abc2; expires=Wed, 31-Mar-2004 05:09:40 GMT; path=/
Set-Cookie: mosvisitor=1
Content-Type: text/html

9:http://webdev/mambo/?option=com_regi...&task=register
(time : 00:01:08)
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:41 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Thu, 04 Mar 2004 19:57:09 GMT
ETag: "0-1c12-40478a15"
Accept-Ranges: bytes
Content-Length: 7186
Content-Type: text/css
HTTP/1.1 200 OK
Date: Tue, 30 Mar 2004 17:09:41 GMT
Server: Apache/1.3.29 (Win32)
Last-Modified: Wed, 03 Mar 2004 22:41:40 GMT
ETag: "0-3ff5-40465f24"
Accept-Ranges: bytes
Content-Length: 16373
Content-Type: text/css
New Url: http://www.mamboserver.com/mambo/
Old Url: http://webdev/


No link in temporary table

--------------------------------------------------------------------------------

links found : 9
http://webdev/mambo/
http://webdev/mambo/?option=content&...&id=1&Itemid=2
http://webdev/mambo/?option=content&...&id=3&Itemid=2
http://webdev/mambo/?option=content&...w&id=4&Itemid=
http://webdev/mambo/?option=content&...w&id=5&Itemid=
http://webdev/mambo/?option=content&...w&id=8&Itemid=
http://webdev/mambo/?option=content&...w&id=7&Itemid=
http://webdev/mambo/?option=com_regi...k=lostPassword
http://webdev/mambo/?option=com_regi...&task=register
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

Charter 03-30-2004 08:44 AM
Hi. What did you do to get it to work?

silverfish 03-30-2004 08:50 AM
I didnt do anything... it just worked..

Problem is, there are a lot more than just 9 links on that site.. any reason as to why it wouldnt index the whole thing?

Mike

Charter 03-30-2004 09:00 AM
Hi. Did you happen to restart Apache, and then it worked?

PhpDig is written to crawl links within one domain. If you want to crawl across domains, then check out this thread.

silverfish 03-30-2004 09:42 AM
Hmmm

It really isnt a domain, its a directory off my localhost...

Charter 03-30-2004 09:52 AM
Hi. What changes did you make to your hosts file?

silverfish 03-30-2004 10:21 AM
Man, I just dont understand...

It desnt dig links.. like in text.. my article titles.. etc... i hate this!

SITE : http://webdev/
Exclude paths :
- administrator/
- classes/
- components/
- editor/
- images/
- includes/
- language/
- mambots/
- media/
- modules/
- templates/
- uploadfiles/
1:http://webdev/
(time : 00:00:06)
No link in temporary table

--------------------------------------------------------------------------------

links found : 1
http://webdev/
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

silverfish 03-30-2004 11:16 AM
ok...

IM perplexed.. I mean. I have no menu, just an article on my front page.. and it WILL NOT DIG THIS ARTICLE!!!

WHY???

ARghhh!!!

silverfish 03-30-2004 11:21 AM
This is my current result.. WHy is it seeing them as duplicates?

Spidering in progress...

--------------------------------------------------------------------------------
SITE : http://webdev/
Exclude paths :
- administrator/
- classes/
- components/
- editor/
- images/
- includes/
- language/
- mambots/
- media/
- modules/
- templates/
- uploadfiles/
1:http://webdev/
(time : 00:00:06)
+
level 1...
2:http://webdev/?option=content&task=view&id=8&Itemid=
(time : 00:00:16)
+ + + + +
level 2...
Duplicate of an existing document
3:http://webdev/?option=content&task=view&id=7&Itemid=
(time : 00:00:27)

Duplicate of an existing document
4:http://webdev/?option=content&task=view&id=5&Itemid=
(time : 00:00:33)

Duplicate of an existing document
5:http://webdev/?option=content&task=view&id=1&Itemid=2
(time : 00:00:38)

Duplicate of an existing document
6:http://webdev/?option=content&task=view&id=3&Itemid=2
(time : 00:00:43)

Duplicate of an existing document
7:http://webdev/?option=content&task=view&id=4&Itemid=
(time : 00:00:49)

No link in temporary table

--------------------------------------------------------------------------------

links found : 7
http://webdev/
http://webdev/?option=content&task=view&id=8&Itemid=
http://webdev/?option=content&task=view&id=7&Itemid=
http://webdev/?option=content&task=view&id=5&Itemid=
http://webdev/?option=content&task=view&id=1&Itemid=2
http://webdev/?option=content&task=view&id=3&Itemid=2
http://webdev/?option=content&task=view&id=4&Itemid=
Optimizing tables...
Indexing complete !
--------------------------------------------------------------------------------
[Back] to admin interface.

Charter 03-30-2004 11:58 AM
Hi. There is a thread here that talks about how PhpDig determines a duplicate page. As I cannot access your web content, I'm not sure that I can offer constructive help.

silverfish 03-30-2004 12:38 PM
OK, when it says it found nine links...

BUT....

Why when i search for something in one of those links is it not there??? What does "found 9 links" mean if you cant go to them through the search?

Mike

Charter 03-30-2004 12:41 PM
Hi. Look in your PhpDig text_content directory, view a file, and try a search on some words that appear in the file.

silverfish 03-30-2004 12:48 PM
Nope cannot see any of the words from the page being referenced in the link on the content directory... :(

Charter 03-30-2004 12:50 PM
Hi. Do you have the following set in the config file?

define('DISPLAY_SNIPPETS',true);

silverfish 03-30-2004 01:12 PM
Yes, it is set to true.

Charter 03-30-2004 01:14 PM
Hi. Are there any TXT files in the PhpDig text_content directory?


All times are GMT -8. The time now is 08:49 PM.
Page 1 of 2 1 2 >
Show 40 post(s) from this thread on one page

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.