PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Mod Submissions

Reply
 
Thread Tools
Old 09-13-2003, 03:19 PM   #1
Iltud
Green Mole
 
Join Date: Sep 2003
Location: Brest - France
Posts: 22
Mod Proxy (English version)

This mod allows phpDid to index sites through proxies.

BUT, there are some constraints :[list=1][*] the database must run at localhost[*] FTP features of phpDig won't work[*] this proxy feature works in exclusive mode : Either you decide to use it, or not. The detection of local sites isn't automatic.[*] perhaps phpDig won't use cookies and authentication on remote sites (to check ?)[/list=1]

The line numbers given here are approximative and based on the version 1.6.2 of phpDig.

In the file /includes/config.php :[list=2][*]Add these 3 lines (anywhere) :
PHP Code:
define('PHPDIG_USE_PROXY',false); // Use proxy-mode ? true or false
define('PHPDIG_PROXY_URL','my.proxy.net'); // Proxy url
define('PHPDIG_PROXY_PORT',3128); // Proxy port 
[/list=2]

In the file /admin/robot_functions.php :
[list=3][*]Replace the line #264 :

PHP Code:
$fp = @fsockopen($host,$port); 
by this section :

PHP Code:
if(PHPDIG_USE_PROXY==true && PHPDIG_PROXY_URL!="")
{
  
$fp = @fsockopen(PHPDIG_PROXY_URL,PHPDIG_PROXY_PORT);  
} else {
  
$fp = @fsockopen($host,$port);  

[*]At lines #285-287, replace :

PHP Code:
$request =
  
"HEAD $path HTTP/1.1\\n"
  
."Host: $host$sport\\n" 
by :

PHP Code:
$request =
  
"HEAD ".(PHPDIG_USE_PROXY==true?$url:$path)." HTTP/1.0\\r\\n"
  
.(PHPDIG_USE_PROXY==true?"":"Host: $host$sport\\r\\n"
[*]At lines #352-353, replace :
PHP Code:
$req1 "HEAD $path HTTP/1.1\\n"
        
."Host: $host$sport\\n" 
by :

PHP Code:
$req1 "HEAD ".(PHPDIG_USE_PROXY==true?$newpath:$path)." HTTP/1.0\\r\\n"
        
.(PHPDIG_USE_PROXY==true?"":"Host: $host$sport\\r\\n"
[*]Delete the line #630 :

PHP Code:
$file_content = @file($uri); 
[*]At lines #638-641, replace :

PHP Code:
if (is_array($file_content)) {
   
fwrite($f_handler,implode('',$file_content));
}
fclose($f_handler); 
by this section :

PHP Code:
if(PHPDIG_USE_PROXY==true && PHPDIG_PROXY_URL!="")
{
  
$request "GET $uri HTTP/1.0\\r\\n"
            
."Accept: */*\\r\\n"
            
."Accept-Charset: iso-8859-1\\r\\n"
            
."Accept-Encoding: identity\\r\\n"
            
."User-Agent: PhpDig/".PHPDIG_VERSION." (PHP; MySql)\\r\\n\\r\\n";
  
$fp fsockopen(PHPDIG_PROXY_URL,PHPDIG_PROXY_PORT);
  if (
$fp) {
    
fputs($fp,$request);
  
    
$c "\\0";
    while (!
feof($fp) && $c != "\\r\\n")
      
$c fgets($fp512); // 512 can be changed
  
    
while (!feof($fp))
    {
      
$answer fgets($fp,255);  // 255 can be changed
      
$answer str_replace("\\r\\n","\\n",$answer);
      
fwrite($f_handler,$answer);
    } 
    
fclose($fp);
    
fclose($f_handler);
  }
}
else
{
  
$file_content = @file($uri);
  if (
is_array($file_content)) {
     
fwrite($f_handler,implode('',$file_content));
  }
  
fclose($f_handler);

[/list=3]

That's all folks :-)
I hope that I didn't forget anything.
Any comments or bugfixes/enhancements are welcome.

Last edited by Iltud; 09-20-2003 at 12:55 PM.
Iltud is offline   Reply With Quote
Old 09-20-2003, 12:56 PM   #2
Iltud
Green Mole
 
Join Date: Sep 2003
Location: Brest - France
Posts: 22
Correction of cariage returns in HTTP/1.0 requests.

( \n --> \r\n )
Iltud is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to index chinese pages ...and spanish, english, french.. mikiwiki How-to Forum 0 05-12-2006 08:41 AM
Japanese characters on an English page Shdwdrgn Troubleshooting 1 03-15-2005 09:28 AM
User Submitted URLs Mod Version 1 jerrywin5 Mod Submissions 25 08-31-2004 01:41 PM
Mod Proxy (Version française) Iltud Mod Submissions 1 09-20-2003 12:53 PM
Bugfix <!-- phpDigInclude --> (English version) Iltud Bug Tracker 0 09-13-2003 12:53 PM


All times are GMT -8. The time now is 02:52 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.