PhpDig.net - View Single Post - need help: phpdig suddenly reads html-comments!

Charter · 10-13-2003, 03:39 PM

Hi Rolandks. The bug is that strip_tags is more lenient than before, meaning that certain things that used to be stipped are no longer. With preg_replace('/<.*>/sU', '', $text); and eregi_replace("<[^>]*>","",$text); everything between the < and > should be stripped. My personal preference is to use eregi_replace("<[^>]*>","",$text); over preg_replace('/<.*>/sU', '', $text); but I don't want to keep using strip_tags($text); because of problems encountered.

Hi manute. What version of PhpDig are you running? In robot_functions.php, the phpdigCleanHtml function in version 1.6.2 is as follows:

PHP Code:


			
function phpdigCleanHtml($text) {

//htmlentities

global $spec;



//replace blank characters by spaces

$text = ereg_replace("[\\r\\n\\t]+"," ",$text);



//extracts title

if ( eregi("<title *>([^<>]*)</title *>",$text,$regs) ) {

    $title = $regs[1];

}

else {

    $title = "";

}

//delete content of head, script, and style tags

$text = eregi_replace("<head[^<>]*>.*</head>"," ",$text);

$text = eregi_replace("<script[^>]*>.*</script>"," ",$text);

$text = eregi_replace("<style[^>]*>.*</style>"," ",$text);

// clean tags

$text = eregi_replace("(</?[a-z0-9 ]+>)",'\\1 ',$text);



//tries to replace htmlentities by ascii equivalent

foreach ($spec as $entity => $char) {

      $text = eregi_replace ($entity."[;]?",$char,$text);

      $title = eregi_replace ($entity."[;]?",$char,$title);

}

$text = ereg_replace('&#([0-9]+);',chr('\\1').' ',$text);



//replace blank characters by spaces

$text = eregi_replace("--|[{}();\\"]+|</[a-z0-9]+>|[rnt]+",' ',$text);



//f..k <!SOMETHING tags !!

$text = eregi_replace('(<)!([^-])','\\1\\2',$text);



//replace any group of blank characters by an unique space

$text = ereg_replace("[[:blank:]]+"," ",eregi_replace("<[^>]*>","",$text));



$retour['content'] = $text;

$retour['title'] = $title;

return $retour;

}

and in config.php, the $spec array in version 1.6.2 is as follows:

PHP Code:


			
//----------HTML ENTITIES

$spec = array( "&amp" => "&",

               "&agrave" => "Ã*",

               "&egrave" => "Ã¨",

               "&ugrave" => "Ã¹",

               "&oacute;" => "Ã³",

               "&eacute" => "Ã©",

               "&icirc" => "Ã®",

               "&ocirc" => "Ã´",

               "&ucirc" => "Ã»",

               "&ecirc" => "Ãª",

               "&ccedil" => "Ã§",

               "&#156" => "oe",

               "&gt" => " ",

               "&lt" => " ",

               "&deg" => " ",

               "&apos" => "'",

               "&quot" => " ",

               "&acirc" => "Ã¢",

               "&iuml" => "Ã¯",

               "&euml" => "Ã«",

               "&auml" => "Ã¤",

               "&ouml" => "Ã¶",

               "&uuml" => "Ã¼",

               "&nbsp" => " ",

               "&szlig" => "ÃŸ",

               "&iacute" => "Ã*",

               "&reg" => " ",

               "&copy" => " ",

               "&aacute" => "Ã¡",

               "&Aacute" => "Ã",

               "&eth" => "Ã°",

               "&ETH" => "Ã",

               "&Eacute" => "Ã‰",

               "&Iacute" => "Ã",

               "&Oacute" => "Ã“",

               "&uacute" => "Ãº",

               "&Uacute" => "Ãš",

               "&THORN" => "Ãž",

               "&thorn" => "Ã¾",

               "&Ouml" => "Ã–",

               "&aelig" => "Ã¦",

               "&AELIG" => "Ã†",

               "&aring" => "Ã¥",

               "&Aring" => "Ã…",

               "&oslash" => "Ã¸",

               "&Oslash" => "Ã˜"

               );

10-13-2003, 03:39 PM	#16
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi Rolandks. The bug is that strip_tags is more lenient than before, meaning that certain things that used to be stipped are no longer. With preg_replace('/<.>/sU', '', $text); and eregi_replace("<[^>]>","",$text); everything between the < and > should be stripped. My personal preference is to use eregi_replace("<[^>]>","",$text); over preg_replace('/<.>/sU', '', $text); but I don't want to keep using strip_tags($text); because of problems encountered. Hi manute. What version of PhpDig are you running? In robot_functions.php, the phpdigCleanHtml function in version 1.6.2 is as follows: PHP Code: function phpdigCleanHtml($text) { //htmlentities global $spec; //replace blank characters by spaces $text = ereg_replace("[\\r\\n\\t]+"," ",$text); //extracts title if ( eregi("<title >([^<>])</title >",$text,$regs) ) { $title = $regs[1]; } else { $title = ""; } //delete content of head, script, and style tags $text = eregi_replace("<head[^<>]>.</head>"," ",$text); $text = eregi_replace("<script[^>]>.</script>"," ",$text); $text = eregi_replace("<style[^>]>.</style>"," ",$text); // clean tags $text = eregi_replace("(</?[a-z0-9 ]+>)",'\\1 ',$text); //tries to replace htmlentities by ascii equivalent foreach ($spec as $entity => $char) { $text = eregi_replace ($entity."[;]?",$char,$text); $title = eregi_replace ($entity."[;]?",$char,$title); } $text = ereg_replace('&#([0-9]+);',chr('\\1').' ',$text); //replace blank characters by spaces $text = eregi_replace("--\|[{}();\\"]+\|</[a-z0-9]+>\|[rnt]+",' ',$text); //f..k <!SOMETHING tags !! $text = eregi_replace('(<)!([^-])','\\1\\2',$text); //replace any group of blank characters by an unique space $text = ereg_replace("[[:blank:]]+"," ",eregi_replace("<[^>]>","",$text)); $retour['content'] = $text; $retour['title'] = $title; return $retour; } and in config.php, the $spec array in version 1.6.2 is as follows: PHP Code: //----------HTML ENTITIES $spec = array( "&amp" => "&", "&agrave" => "Ã", "&egrave" => "Ã¨", "&ugrave" => "Ã¹", "ó" => "Ã³", "&eacute" => "Ã©", "&icirc" => "Ã®", "&ocirc" => "Ã´", "&ucirc" => "Ã»", "&ecirc" => "Ãª", "&ccedil" => "Ã§", "&#156" => "oe", "&gt" => " ", "&lt" => " ", "&deg" => " ", "&apos" => "'", "&quot" => " ", "&acirc" => "Ã¢", "&iuml" => "Ã¯", "&euml" => "Ã«", "&auml" => "Ã¤", "&ouml" => "Ã¶", "&uuml" => "Ã¼", "&nbsp" => " ", "&szlig" => "ÃŸ", "&iacute" => "Ã", "&reg" => " ", "&copy" => " ", "&aacute" => "Ã¡", "&Aacute" => "Ã", "&eth" => "Ã°", "&ETH" => "Ã", "&Eacute" => "Ã‰", "&Iacute" => "Ã", "&Oacute" => "Ã“", "&uacute" => "Ãº", "&Uacute" => "Ãš", "&THORN" => "Ãž", "&thorn" => "Ã¾", "&Ouml" => "Ã–", "&aelig" => "Ã¦", "&AELIG" => "Ã†", "&aring" => "Ã¥", "&Aring" => "Ã…", "&oslash" => "Ã¸", "&Oslash" => "Ã˜" ); __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.