Email Handling - Unknown encoding

[StephenV]

Hi

I'm testing out the email function in Vicidial as a client is interested in doing an email campaign.
I am getting stuck at a point where I believe Perl is complaining about how the email is encoded, but I don't know what to do to fix it.

When I run AST_inbound_email_parser.pl, it fetches most of the mail, but seem to leave some behind that it cannot decode. This is what I get from debug:

Code: Select all

./AST_inbound_email_parser.pl --debugX
Attempting to connect to POP3 server (x.x.x.x)

Email message is multipart.  Need to select the best format type and parse it.
Part content disposition is
 Part content type is Content-Type: text/plain; charset="us-ascii".
Part content size is 182
First acceptable content-type match is text/plain.  Stripping headers to get text...
Part content disposition is
 Part content type is Content-Type: text/html; charset="us-ascii".
Part content size is 2157
Unknown encoding 'Type: text/html' at ./AST_inbound_email_parser.pl line 729

I edited the file to allow it to fetch mail in plain text, so line numbers might not correspond to the original.

I followed the Email handling doc and installed all the requirements in cpan successfully. What can I do so it fetches these emails as well?

My setup:
Vicidial: 2.6b0.5
Asterisk: 1.4.41
SUSE Linux Enterprise Server 11 SP1 x86_64

[StephenV]

Some more information. The part where it fails is:

Code: Select all

 # Do a clean-and-covert on the message, to decode base64 messages and also to UTF8 decode quoted printable
 text, in order to pick up special characters
                                if ($content_transfer_encoding eq "base64") {
                                        $message=decode_base64($message);
                                        if ($charset ne "") {
                                                # decode to Perl's internal format
                                                $message=decode($charset, $message);
                                                #encodetoUTF-8
                                                $message=encode('utf-8', $message);
                                        }
                                } elsif ($content_transfer_encoding eq "quoted-printable") {
                                        $message=decode_qp($message);

                                        if ($charset ne "") {
                                                # decode to Perl's internal format
                                                $message=decode($charset, $message);
                                                #encodetoUTF-8
                                                $message=encode('utf-8', $message);
                                        }
                                }



At line:
$message=decode($charset, $message);

Any help or suggestions will be appreciated.

[jjohnson78]

Try editing this line:

$sub_content_type=~/charset\=(KOI8\-R|ISO\-8859\-[0-9]+|windows\-12[0-9]+|utf\-8)/i;

To this:

$sub_content_type=~/charset\=\"?(KOI8\-R|ISO\-8859\-[0-9]+|windows\-12[0-9]+|utf\-8|us\-ascii)\"?/i;

It appears twice in the script, once for IMAP and once for POP3. This line is for pattern-matching the charset, and us-ascii isn't one of the charset patterns it's looking for and it probably should be - the change also accounts for there possibly being quotes around the charset as in your output above in your first post (charset="us-ascii"). Hopefully it's that simple.

[StephenV]

Thanks jjohnson
Although still failing, it looks like I'm getting somewhere:

Code: Select all

Email message is multipart.  Need to select the best format type and parse it.
Part content disposition is
 Part content type is Content-Type: text/plain; charset="us-ascii".
Part content size is 183
First acceptable content-type match is text/plain.  Stripping headers to get text...
Part content disposition is
 Part content type is Content-Type: text/html; charset="us-ascii".
Part content size is 2158
Unknown encoding '"us-ascii"' at ./AST_inbound_email_parser.pl line 729

[jjohnson78]

Oh, derp. Right underneath that coding I mentioned earlier, there's this set of lines:

$charset=$&;
$charset=substr($charset, 8 );
$content_type.="; charset=$charset";

Add one quick replace statement to trim the quotes out:
$charset=$&;
$charset=substr($charset, 8 );
$charset=~s/\"//gi;
$content_type.="; charset=$charset";

THAT should do it, I think. I tested the decode() statement by forcing the charset to be "us-ascii" and it didn't spit out any errors.

[StephenV]

Great, thanks! Looks like it works quite nicely now.

So, my next question, should these additions/changes not be added into newer builds of Vicidial? I'm sure I am not the only person who managed to send us-ascii emails.

[jjohnson78]

I will be sending a revised version with the above changes to be put into SVN for future releases, so if you upgrade in the future these modifications should be in there and you won't have to re-patch.

[mflorell]

This patch has been applied to svn/trunk as of 2013-06-07 02:00:00

[StephenV]

Thanks jjohnson & Matt