French accent firstname

[c20xh2]

VERSION: 2.14-626a
BUILD: 170825-1708
Cluster Install

Hi,

I'm having trouble loading leads/lists containing french accent (é, à, è, etc..). An example:

Éric become Éric in the DB.

Should I change the collation of the fields in the DB ? Right now fields are set to utf8_unicode_ci.

Thank you.

[mflorell]

There are often several different settings in both MySQL/MariaDB and Apache for localization that you will need to set for UTF-8 encoding to work with special characters like that. Make sure none of them are set to any of the "latin" charsets

[c20xh2]

Yes I'm trying to find the right one,

If I edit a lead directly in the DB and change the broken character to a "É", the name show correctly in Vicidial.

This make me think that I would need to make some change in the Vicidial lead loading script. I'm afraid I will have trouble with future update if I do that. Is this my only solution ?

[mflorell]

I think it's probably an Apache or PHP configuration issue. I had worked on this a year ago for a client and that was the cause, although since the client found the fix for it, I can't say I know for sure what changed setting fixed it for them.

[c20xh2]

Found a solution for the admin_modify_lead (was writing empty characters instead of french accents in db):

If you look at admin_modify_lead.php line 268:

Code: Select all

$first_name = preg_replace('/[^- \'\+\_\.0-9a-zA-Z]/','',$first_name);
    $first_name = preg_replace('/\+/',' ',$first_name);

The script is stripping the french accents, commenting the line out fixed the issue for me.

I'm still working on finding a solution for the list_loader (admin_listloader_fourth_gen.php).

I tryed using :

Code: Select all

mysqli_query($conn,"SET NAMES 'utf8'");

And

Code: Select all

$link->set_charset("utf8");

In "dbconnect_mysqli.php" but not luck so far, if I print $link->character_set_name(), the answer is allready utf8

I might be wrong but I think the php and apache config are allright since I can add "Éric" first name using "admin_modify_lead.php". I will keep working on this and update when I found the solution.

If anyone have some advice

[EDIT]

Found out that if I convert my file to CSV and make sure the file is saved with UTF8 encoding, everything work fine for the list loader !

[mflorell]

Thanks for posting your solution!

[blackbird2306]

First of all:
Did you set in system settings: Use Non-Latin: 1 (default is 0) ?
Because in admin_modify_lead.php the line 268 will be true (and executed) only if "non-latin" is 0:

if ($non_latin < 1)
{
...
$first_name = preg_replace('/[^- \'\+\_\.0-9a-zA-Z]/','',$first_name);
$first_name = preg_replace('/\+/',' ',$first_name);
...
}

And next there was a problem in this script with non-latin "/srv/www/htdocs/vicidial/sheet2tab.pl" I found 2 years ago, which was fixed in 2016.
Please check, if these lines are the same in your file:

use Spreadsheet::Read;
use Spreadsheet::XLSX;
use File::Basename;
---IMPORTANT LINE-----> use open ':std', ':encoding(UTF-8)';