Jump to content

Problem with Charset

agustin

By agustin
in Using WHMCS

 Share

Recommended Posts

agustin Junior Member

agustin

Posted
Posted

Hello Everyone,

 

I open this thread to ask for your help: I use WHMCS and import emails to create a ticket using POP3. The problem is that when the email I import has a character like this: á,é, í, ó, ú or ñ, WHMCS shows this: �

 

My WHMCS System Charset is utf-8 and I import emails from Google Apps (the same than Gmail, but for business).

 

Could you help me with this?

Thanks!

 

Regards,

0
Link to comment
Share on other sites
  • 2 months later...
Eduardo G. Senior Member

Eduardo G.

Posted
Posted

Can do nothing about php as import script is coded, as you know.

From firefox, if I change the charset view from utf8 to iso8859, I can read it perfectly, even if the admin header contains:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

 

So, the presentation is utf-8 and the database storage is utf-8. The problem must be inside the import script

0
Link to comment
Share on other sites
HerrZ Senior Member

HerrZ

Posted
Posted

mmh right. there was a params problem with umlauts in my plesk module, too. the whmcs "gettheparamsorsomething" function does not handle it correct. you have always to mysql select by self.

 

but its still encrpyted the importer script (where is the script in backendnavigation?) ...

0
Link to comment
Share on other sites
Eduardo G. Senior Member

Eduardo G.

Posted
Posted

I've made a simple php script to read from tickets table and just shows imported messages.

It uses mb_detect_encoding() to detect possible utf8 content, and when found, it uses utf8_encode() to force utf8 and shows message again with ferfect characters

Would you like to try it out? :-) This is the first step for on-the-fly charset fix

 

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 
<html xmlns="http://www.w3.org/1999/xhtml"> 
<head> 
<meta http-equiv="content-type" content="text/html; charset=utf-8" /> 
<title>Encode testing</title> 
</head> 
<body> 

<?php
// Encode testing

mysql_connect('localhost', 'root', '');
mysql_select_db('whmcs');

// Read first 5 tickets
$sql = "SELECT id, tid, title, message FROM tbltickets LIMIT 5";
$res = mysql_query($sql);

// For each ticket
while(list($id, $tid, $title, $message) = mysql_fetch_array($res)) {
// Shows generic info for this ticket
echo "[$id] ";
echo "<strong>$title</strong><br />\n";
echo "<p><strong>Before</strong><br />$message</p>\n";

// Check if encoding is utf8
$enc = mb_detect_encoding($message, 'auto');
if($enc=="UTF-8") {
	// If so, force charset
	$message = utf8_encode($message);
	// and print fixed message
	echo "<p><strong>After</strong><br />$message</p>\n";
}
echo "<br />\n";
}

?>

</body>
</html>

0
Link to comment
Share on other sites
Eduardo G. Senior Member

Eduardo G.

Posted
Posted

And this could be a hook

 

<?php

function convert_utf8($vars) {

$ticket_id = $vars['id'];
$message = $vars['message'];

$is_utf8 = mb_detect_encoding($message, 'UTF-8, ISO-8859-1');

if($is_utf8=="ISO-8859-1") {
	$message = utf8_encode($message);
		$upd = "UPDATE tbltickets SET message='$message' WHERE id='$ticket_id'";
		$res = mysql_query($upd);
}

add_hook("TicketOpen", 1, "convert_utf8");

?>

0
Link to comment
Share on other sites
HerrZ Senior Member

HerrZ

Posted
Posted (edited)

thx, but i can't confirm that ISO-8859-1 emails get in correct with this hook

Hook:

function init_TicketOpen($vars)
{
$ticket_id = $vars['id'];
   $message = $vars['message'];

   $is_utf8 = mb_detect_encoding($message, 'UTF-8, ISO-8859-1');
   if ($is_utf8 == "ISO-8859-1") {
       $message = utf8_encode($message);
       $res = mysql_query("UPDATE tbltickets SET message='$message' WHERE id='$ticket_id' LIMIT 1"); 
}	
}
add_hook("TicketOpen",1,"init_TicketOpen","");

 

i think the hook is not called.

trying

 $message = "huhuhuhuhu " . utf8_encode($message);

has no effect.

Edited by HerrZ
0
Link to comment
Share on other sites
ditto Senior Member

ditto

Posted
Posted
What can we do now? Change the WHMCS charset to ISO-8859-1? What may be affected by this change?

 

My WHMCS installation is using iso-8859-1, and I also have problems with displaying characters correct. All email sent from Gmail accounts with æ, ø or å is not displayed correct. I am using email piping/forwarders.

 

So it seems there is problems both for us that use iso-8859-1, and for those using utf-8

0
Link to comment
Share on other sites
HerrZ Senior Member

HerrZ

Posted
Posted

i think the hook TicketOpen is not called in piping progress.

 

anyway the correct function params for a hook are:

 

$ticket_id = $vars['ticketid'];
$subject = $vars['subject'];
$message = $vars['message'];

 

not like above.

0
Link to comment
Share on other sites
sparky Senior Member

sparky

Posted
Posted

This should convert the accents to normal charactors from any form posted in whmcs or any file that includes dbfunctions.php (which is pretty well all of them)

 

Create a file called remove_accents.php in your includes/hooks directory and insert this code into the file.

<?php
function removeaccents($string) {
   $UTF8_UPPER_ACCENTS = array(
     'À' => 'A', 'Ô' => 'O', 'ÄŽ' => 'D', 'Ḟ' => 'F', 'Ë' => 'E', 'Å ' => 'S', 'Æ ' => 'O',
     'Ă' => 'A', 'Ř' => 'R', 'Ț' => 'T', 'Ň' => 'N', 'Ā' => 'A', 'Ķ' => 'K',
     'Ŝ' => 'S', 'Ỳ' => 'Y', 'Ņ' => 'N', 'Ĺ' => 'L', 'Ħ' => 'H', 'Ṗ' => 'P', 'Ó' => 'O',
     'Ú' => 'U', 'Ě' => 'E', 'É' => 'E', 'Ç' => 'C', 'Ẁ' => 'W', 'Ċ' => 'C', 'Õ' => 'O',
     'Ṡ' => 'S', 'Ø' => 'O', 'Ģ' => 'G', 'Ŧ' => 'T', 'Ș' => 'S', 'Ė' => 'E', 'Ĉ' => 'C',
     'Ś' => 'S', 'Î' => 'I', 'Ű' => 'U', 'Ć' => 'C', 'Ę' => 'E', 'Ŵ' => 'W', 'Ṫ' => 'T',
     'Ū' => 'U', 'Č' => 'C', 'Ö' => 'Oe', 'È' => 'E', 'Ŷ' => 'Y', 'Ą' => 'A', 'Ł' => 'L',
     'Ų' => 'U', 'Ů' => 'U', 'Ş' => 'S', 'Ğ' => 'G', 'Ļ' => 'L', 'Ƒ' => 'F', 'Ž' => 'Z',
     'Ẃ' => 'W', 'Ḃ' => 'B', 'Å' => 'A', 'Ì' => 'I', 'Ï' => 'I', 'Ḋ' => 'D', 'Ť' => 'T',
     'Ŗ' => 'R', 'Ä' => 'Ae', 'Í' => 'I', 'Ŕ' => 'R', 'Ê' => 'E', 'Ü' => 'Ue', 'Ò' => 'O',
     'Ē' => 'E', 'Ñ' => 'N', 'Ń' => 'N', 'Ĥ' => 'H', 'Ĝ' => 'G', 'Đ' => 'D', 'Ĵ' => 'J',
     'Ÿ' => 'Y', 'Ũ' => 'U', 'Ŭ' => 'U', 'Ư' => 'U', 'Ţ' => 'T', 'Ý' => 'Y', 'Ő' => 'O',
     'Â' => 'A', 'Ľ' => 'L', 'Ẅ' => 'W', 'Å»' => 'Z', 'Ī' => 'I', 'Ã' => 'A', 'Ä ' => 'G',
     'Ṁ' => 'M', 'Ō' => 'O', 'Ĩ' => 'I', 'Ù' => 'U', 'Į' => 'I', 'Ź' => 'Z', 'Á' => 'A',
     'Û' => 'U', 'Þ' => 'Th', 'Ð' => 'Dh', 'Æ' => 'Ae', 'Ĕ' => 'E',
     );

   $UTF8_LOWER_ACCENTS = array(
     'à ' => 'a', 'ô' => 'o', 'ď' => 'd', 'ḟ' => 'f', 'ë' => 'e', 'š' => 's', 'ơ' => 'o',
     'ß' => 'ss', 'ă' => 'a', 'ř' => 'r', 'ț' => 't', 'ň' => 'n', 'ā' => 'a', 'ķ' => 'k',
     'ŝ' => 's', 'ỳ' => 'y', 'ņ' => 'n', 'ĺ' => 'l', 'ħ' => 'h', 'ṗ' => 'p', 'ó' => 'o',
     'ú' => 'u', 'ě' => 'e', 'é' => 'e', 'ç' => 'c', 'ẁ' => 'w', 'ċ' => 'c', 'õ' => 'o',
     'ṡ' => 's', 'ø' => 'o', 'ģ' => 'g', 'ŧ' => 't', 'ș' => 's', 'ė' => 'e', 'ĉ' => 'c',
     'ś' => 's', 'î' => 'i', 'ű' => 'u', 'ć' => 'c', 'ę' => 'e', 'ŵ' => 'w', 'ṫ' => 't',
     'ū' => 'u', 'č' => 'c', 'ö' => 'oe', 'è' => 'e', 'ŷ' => 'y', 'ą' => 'a', 'ł' => 'l',
     'ų' => 'u', 'ů' => 'u', 'ş' => 's', 'ğ' => 'g', 'ļ' => 'l', 'ƒ' => 'f', 'ž' => 'z',
     'ẃ' => 'w', 'ḃ' => 'b', 'å' => 'a', 'ì' => 'i', 'ï' => 'i', 'ḋ' => 'd', 'ť' => 't',
     'Å—' => 'r', 'ä' => 'ae', 'Ã*' => 'i', 'Å•' => 'r', 'ê' => 'e', 'ü' => 'ue', 'ò' => 'o',
     'ē' => 'e', 'ñ' => 'n', 'ń' => 'n', 'ĥ' => 'h', 'ĝ' => 'g', 'đ' => 'd', 'ĵ' => 'j',
     'ÿ' => 'y', 'Å©' => 'u', 'Å*' => 'u', 'Æ°' => 'u', 'Å£' => 't', 'ý' => 'y', 'Å‘' => 'o',
     'â' => 'a', 'ľ' => 'l', 'ẅ' => 'w', 'ż' => 'z', 'ī' => 'i', 'ã' => 'a', 'ġ' => 'g',
     'ṁ' => 'm', 'ō' => 'o', 'ĩ' => 'i', 'ù' => 'u', 'į' => 'i', 'ź' => 'z', 'á' => 'a',
     'û' => 'u', 'þ' => 'th', 'ð' => 'dh', 'æ' => 'ae', 'µ' => 'u', 'ĕ' => 'e', 
     );
   $accents = array_merge($UTF8_UPPER_ACCENTS, $UTF8_LOWER_ACCENTS);

   $convertedstring = array();
   if (is_array($string)){
       foreach ($string AS $key=>$value) {
           if (is_array($value)) {
               $convertedstring[$key] = removeaccents($value);
           } else {
               $encoding = mb_detect_encoding($value, 'auto');
               if($encoding != "UTF-8") $value = utf8_encode($string);
               $convertedstring[$key] = strtr($value,$accents);
           }
       }
   } else {
       $encoding = mb_detect_encoding($string, 'auto');
       if($encoding != "UTF-8") $string = utf8_encode($string);
       $convertedstring = strtr($string,$accents);
   }
   return utf8_encode($convertedstring);
}
foreach ($_REQUEST as $key=>$value) {
   $_REQUEST[$key] = removeaccents($value);
}
unset($key,$value);
foreach ($_POST as $key=>$value) {
   $_POST[$key] = removeaccents($value);
}
unset($key,$value);
foreach ($_GET as $key=>$value) {
   $_GET[$key] = removeaccents($value);
}
unset($key,$value);
foreach ($_REQUEST AS $key=>$value) {
   $$key = $value;
}
unset($key,$value);
?>

0
Link to comment
Share on other sites
Eduardo G. Senior Member

Eduardo G.

Posted
Posted

That would work, but that's not the way. Different letters make different words, and you shouldn't modify customer's email, letter by letter. It's not a good practice and it's not an elegant solution.

I won't change España into Espana, won't chage "tú" (your) into "tu" (yours)

There is a bug and should be fixed ASAP, in a professional way.

0
Link to comment
Share on other sites
sparky Senior Member

sparky

Posted
Posted

It was meant to be for a charactor replacement... see post #1

At least if the charactor is converted to something that is partly readable instead of ��������

Maybe you can tell me what the above says?

 

What your asking for know is a complete dictionary replacement.

You could always try google translate.

0
Link to comment
Share on other sites
Eduardo G. Senior Member

Eduardo G.

Posted
Posted

Hi, sparky

 

I'm not asking for a replacement, but for a charset conversion, a very common feature used in email readers.

Post #1 is pointing a bug and a fix is needed. You can try to replace every string, but it's not an adecuated solution, as you may be changing some strings that shouldn't be translated, like passwords or any other field in any custom module.

 

You cannot replace every letter in every module, that's not the problem. The problem is a charset conversion, not a character replacement. There is a bug, and should be fixed by whmcs team.

 

I want to use whmcs in spanish, with spanish letters, spanish words and multilingual support.

This can not be done as long as import scripts don't care about charsets.

 

Relax ;-)

0
Link to comment
Share on other sites
HerrZ Senior Member

HerrZ

Posted
Posted
This should convert the accents to normal charactors from any form posted in whmcs or any file that includes dbfunctions.php (which is pretty well all of them)

 

Create a file called remove_accents.php in your includes/hooks directory and insert this code into the file.

<?php
function removeaccents($string) {
   $UTF8_UPPER_ACCENTS = array(
     'À' => 'A', 'Ô' => 'O', 'ÄŽ' => 'D', 'Ḟ' => 'F', 'Ë' => 'E', 'Å ' => 'S', 'Æ ' => 'O',
     'Ă' => 'A', 'Ř' => 'R', 'Ț' => 'T', 'Ň' => 'N', 'Ā' => 'A', 'Ķ' => 'K',
     'Ŝ' => 'S', 'Ỳ' => 'Y', 'Ņ' => 'N', 'Ĺ' => 'L', 'Ħ' => 'H', 'Ṗ' => 'P', 'Ó' => 'O',
     'Ú' => 'U', 'Ě' => 'E', 'É' => 'E', 'Ç' => 'C', 'Ẁ' => 'W', 'Ċ' => 'C', 'Õ' => 'O',
     'Ṡ' => 'S', 'Ø' => 'O', 'Ģ' => 'G', 'Ŧ' => 'T', 'Ș' => 'S', 'Ė' => 'E', 'Ĉ' => 'C',
     'Ś' => 'S', 'Î' => 'I', 'Ű' => 'U', 'Ć' => 'C', 'Ę' => 'E', 'Ŵ' => 'W', 'Ṫ' => 'T',
     'Ū' => 'U', 'Č' => 'C', 'Ö' => 'Oe', 'È' => 'E', 'Ŷ' => 'Y', 'Ą' => 'A', 'Ł' => 'L',
     'Ų' => 'U', 'Ů' => 'U', 'Ş' => 'S', 'Ğ' => 'G', 'Ļ' => 'L', 'Ƒ' => 'F', 'Ž' => 'Z',
     'Ẃ' => 'W', 'Ḃ' => 'B', 'Å' => 'A', 'Ì' => 'I', 'Ï' => 'I', 'Ḋ' => 'D', 'Ť' => 'T',
     'Ŗ' => 'R', 'Ä' => 'Ae', 'Í' => 'I', 'Ŕ' => 'R', 'Ê' => 'E', 'Ü' => 'Ue', 'Ò' => 'O',
     'Ē' => 'E', 'Ñ' => 'N', 'Ń' => 'N', 'Ĥ' => 'H', 'Ĝ' => 'G', 'Đ' => 'D', 'Ĵ' => 'J',
     'Ÿ' => 'Y', 'Ũ' => 'U', 'Ŭ' => 'U', 'Ư' => 'U', 'Ţ' => 'T', 'Ý' => 'Y', 'Ő' => 'O',
     'Â' => 'A', 'Ľ' => 'L', 'Ẅ' => 'W', 'Å»' => 'Z', 'Ī' => 'I', 'Ã' => 'A', 'Ä ' => 'G',
     'Ṁ' => 'M', 'Ō' => 'O', 'Ĩ' => 'I', 'Ù' => 'U', 'Į' => 'I', 'Ź' => 'Z', 'Á' => 'A',
     'Û' => 'U', 'Þ' => 'Th', 'Ð' => 'Dh', 'Æ' => 'Ae', 'Ĕ' => 'E',
     );

   $UTF8_LOWER_ACCENTS = array(
     'à ' => 'a', 'ô' => 'o', 'ď' => 'd', 'ḟ' => 'f', 'ë' => 'e', 'š' => 's', 'ơ' => 'o',
     'ß' => 'ss', 'ă' => 'a', 'ř' => 'r', 'ț' => 't', 'ň' => 'n', 'ā' => 'a', 'ķ' => 'k',
     'ŝ' => 's', 'ỳ' => 'y', 'ņ' => 'n', 'ĺ' => 'l', 'ħ' => 'h', 'ṗ' => 'p', 'ó' => 'o',
     'ú' => 'u', 'ě' => 'e', 'é' => 'e', 'ç' => 'c', 'ẁ' => 'w', 'ċ' => 'c', 'õ' => 'o',
     'ṡ' => 's', 'ø' => 'o', 'ģ' => 'g', 'ŧ' => 't', 'ș' => 's', 'ė' => 'e', 'ĉ' => 'c',
     'ś' => 's', 'î' => 'i', 'ű' => 'u', 'ć' => 'c', 'ę' => 'e', 'ŵ' => 'w', 'ṫ' => 't',
     'ū' => 'u', 'č' => 'c', 'ö' => 'oe', 'è' => 'e', 'ŷ' => 'y', 'ą' => 'a', 'ł' => 'l',
     'ų' => 'u', 'ů' => 'u', 'ş' => 's', 'ğ' => 'g', 'ļ' => 'l', 'ƒ' => 'f', 'ž' => 'z',
     'ẃ' => 'w', 'ḃ' => 'b', 'å' => 'a', 'ì' => 'i', 'ï' => 'i', 'ḋ' => 'd', 'ť' => 't',
     'Å—' => 'r', 'ä' => 'ae', 'Ã*' => 'i', 'Å•' => 'r', 'ê' => 'e', 'ü' => 'ue', 'ò' => 'o',
     'ē' => 'e', 'ñ' => 'n', 'ń' => 'n', 'ĥ' => 'h', 'ĝ' => 'g', 'đ' => 'd', 'ĵ' => 'j',
     'ÿ' => 'y', 'Å©' => 'u', 'Å*' => 'u', 'Æ°' => 'u', 'Å£' => 't', 'ý' => 'y', 'Å‘' => 'o',
     'â' => 'a', 'ľ' => 'l', 'ẅ' => 'w', 'ż' => 'z', 'ī' => 'i', 'ã' => 'a', 'ġ' => 'g',
     'ṁ' => 'm', 'ō' => 'o', 'ĩ' => 'i', 'ù' => 'u', 'į' => 'i', 'ź' => 'z', 'á' => 'a',
     'û' => 'u', 'þ' => 'th', 'ð' => 'dh', 'æ' => 'ae', 'µ' => 'u', 'ĕ' => 'e', 
     );
   $accents = array_merge($UTF8_UPPER_ACCENTS, $UTF8_LOWER_ACCENTS);

   $convertedstring = array();
   if (is_array($string)){
       foreach ($string AS $key=>$value) {
           if (is_array($value)) {
               $convertedstring[$key] = removeaccents($value);
           } else {
               $encoding = mb_detect_encoding($value, 'auto');
               if($encoding != "UTF-8") $value = utf8_encode($string);
               $convertedstring[$key] = strtr($value,$accents);
           }
       }
   } else {
       $encoding = mb_detect_encoding($string, 'auto');
       if($encoding != "UTF-8") $string = utf8_encode($string);
       $convertedstring = strtr($string,$accents);
   }
   return utf8_encode($convertedstring);
}
foreach ($_REQUEST as $key=>$value) {
   $_REQUEST[$key] = removeaccents($value);
}
unset($key,$value);
foreach ($_POST as $key=>$value) {
   $_POST[$key] = removeaccents($value);
}
unset($key,$value);
foreach ($_GET as $key=>$value) {
   $_GET[$key] = removeaccents($value);
}
unset($key,$value);
foreach ($_REQUEST AS $key=>$value) {
   $$key = $value;
}
unset($key,$value);
?>

 

uuhhh,

on each pagecall!

 

foreach $_REQUEST

and

$_POST (actually $_POST is in $_REQUEST ;)

 

this is an very unperformant solution.

0
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use & Guidelines and understand your posts will initially be pre-moderated

  I accept