recording_log | Errors

[kieran333]

Vicibox 8.1.2 - ISO
VERSION: 2.14-697a
BUILD: 190121-2019
SVN: 3063
DB: 1562
V: 2.14b0.5

- - - - -

Hi all,

I have just spotted an issue with a couple of our servers (multiple versions) however the example below is from our most recent install (server details above).

Initially I spotted this when viewing (admin_modify_lead.php) but running the following statement flags up the issue.

Code: Select all


   +--------------+-----------+---------------+------------------------------------------+---------+----------+------------------+
   | recording_id | extension | length_in_sec | filename                                 | lead_id | user     | vicidial_id      |
   +--------------+-----------+---------------+------------------------------------------+---------+----------+------------------+
   |        10457 | 8309      |             7 | 1000_XXXXX25685_XXXXXXX_20190515-165933  |    6796 | XXXXXXX  | 1557935944.25815 |
   |        10607 | 8309      |            77 | 1000_XXXXX25685_XXXXXXX_20190516-090548  |    6796 | XXXXXXX  | 1557993928.423   |
   |        12541 | 8309      |           161 | 1000_XXXXX97209_XXXXXX_20190516-134418   |    6796 | XXXXXXX  | 1558010808.15564 |
   |        12559 | 8309      |             1 | 1000_XXXXX25685_XXXXXX_20190516-134658   |    6796 | XXXXXXX  | 1558010808.15564 |
   |        16866 | 8309      |            75 | 1000_XXXXX78543_XXXXXXX_20190517-160757  |    6796 | XXXXXXX  | 1558105725.25193 |
   |        17987 | 8309      |            51 | 1000_XXXXX08329_XXXXXXXX_20190520-105231 |    6796 | XXXXXXXX | 1558345975.8424  |
   |        18566 | 8309      |            49 | 1000_XXXXX27947_XXXXXXXX_20190520-122237 |    6796 | XXXXXXXX | 1558351378.13155 |
   +--------------+-----------+---------------+------------------------------------------+---------+----------+------------------+



Searching on lead_id there are different recordings attached of different clients / lead_id's under the same record / lead_id in the recording_log table.

At first I thought it may be alternative numbers for the lead, however this is not the case.

This is quite a concern as the recordings or statuses / dispositions are not being recorded correctly.

Is this something that you have come across before?

[williamconley]

do your agents have the ability to "search for a lead" on inbound?

[kieran333]

Hi William,

Here are the campaign settings: Agent Lead Search: DISABLED

Here is a screenshot from the admin interface from another lead.



This will also give us an indication of where the issues are.

Code: Select all

mysql asterisk -e "SELECT phone_number, lead_id, location FROM recording_log JOIN vicidial_list USING (lead_id) WHERE location NOT LIKE concat('%',phone_number,'%');"

[kieran333]

I have just updated our development box to the latest and was able to replicate this with 2 examples in less than 10 autodial calls handled 1/1 Ratio.

Routing Extensions tried: 8368 & 8369

VERSION: 2.14-708a
BUILD: 190414-0924
AST: 13.21.1-vici
SVN: 3098

- - - - - - - -
Lead_ID: 21
Phone_Number: REDACTED 4062
- - - - - - - -
Recording_Log
Lead_ID: 21
Phone_Number: REDACTED 02924
- - - - - - - -
Screenshot
https://33three.eu/dump/HdjEksldD.png

[williamconley]

I have just updated our development box to the latest and was able to replicate this with 2 examples in less than 10 autodial calls handled 1/1 Ratio.

Routing Extensions tried: 8368 & 8369

VERSION: 2.14-708a
BUILD: 190414-0924
AST: 13.21.1-vici
SVN: 3098

- - - - - - - -
Lead_ID: 21
Phone_Number: REDACTED 4062
- - - - - - - -
Recording_Log
Lead_ID: 21
Phone_Number: REDACTED 02924
- - - - - - - -
Screenshot
https://33three.eu/dump/HdjEksldD.png

during this replication, was the DB fresh and stock, or is the DB in all these examples imported from your system? (ie: is your DB the common infection? lol)

[kieran333]

Hi William, DB is stock... only changes were adding carrier, creating a basic autodial campaign, uploading leads and creating a user.

[williamconley]

Understood. So the question, then, is what specific changes did you make to the campaigns/users/DIDs, etc. to cause this. I will note that by default recording isn't "on". I will also note that none of our clients have ever had this complaint. So ... since I have to say that you seem to know what you're doing, I then have to ask: What is it that this smart guy who knows what he's doing *IS* doing that others do not?

It's got to be in there somewhere unless a whole bunch of other users start complaining of it shortly.

[kieran333]

Hi William,

Thanks for your quick response. I have just set up another fresh install from the latest ISO.
Here are the exact steps I took to replicate the issue.
- - -

> os-install
> yast lan (change IP)
> reboot
> zypper up -y
> reboot
> vicibox-install (default settings)
> reboot
- - - - -

> change pw & elevate 6666 user settings
> change GMT time - 0.00
> change Voicemail Zone - UK
> change inidications.conf to UK
> change (Enhanced Disconnect Logging: 1)
> add outbound carrier / SIP Trunk

> add campaign
Method: RATIO
Auto Dial Level: 1
Routing: 8368
Recording: ALLFORCE
Filename: CAMPAIGN_CUSTPHONE_AGENT_FULLDATE
Recording Delay: 1
Dial Timeout: 30
Dial Prefix: X
Caller ID: Set

> add list, upload leads, active list
> add campaign status: TEST
> add user group (Agents)
> add agent
> change hopper level to 50
> log in, place test calls
> run mysql to flag errors

> 22 calls handled until issue popped up


-- UPDATE

One thing I have noticed with the erroneous records is....

end_time, end_epoch, vicidial_id are set as NULL inside recording_log

This brings up all the issues...

Code: Select all

mysql asterisk -e "select * from recording_log where vicidial_id  IS NULL;"

[williamconley]

If these leads have each other's recordings visible there's a challenge, of course. The question is whether they share the lead id on the recording_log or if the lead_modify page is showing recordings for other leads.

While we don't have any recent UK installs, and you didn't show any changes to the List entry, I am not sure what to tell you. I find it hard to believe that UK signalling would cause intermixed recordings.

However: You now have the opportunity to check the logs for these "intermixed" calls to see where the flaw lies.

Also: "run mysql to flag errors", I was under the impression the problem was visible in the lead_modify page, but you didn't specify if that was true here or if you used the mysql method and did not confirm?

[kieran333]

If these leads have each other's recordings visible there's a challenge, of course. The question is whether they share the lead id on the recording_log or if the lead_modify page is showing recordings for other leads.

While we don't have any recent UK installs, and you didn't show any changes to the List entry, I am not sure what to tell you. I find it hard to believe that UK signalling would cause intermixed recordings.

However: You now have the opportunity to check the logs for these "intermixed" calls to see where the flaw lies.

Also: "run mysql to flag errors", I was under the impression the problem was visible in the lead_modify page, but you didn't specify if that was true here or if you used the mysql method and did not confirm?

Hi William,

Yes it's visible inside the lead_modify page, it's where I first spotted different numbers...

Running this inside MySQL flags them up easier.

lead_id returns 0 and vicidial_id = NULL

Code: Select all

mysql asterisk -e "Select * from recording_log where vicidial_id  IS NULL;"

[williamconley]

These wouldn't all be on lead_id ... Zero? Would they?

[kieran333]

These wouldn't all be on lead_id ... Zero? Would they?

No, if I search for the phone number which is in the filename or location I can find the lead whilst searching for a lead and can confirm that they have a correct lead_id attached.

Currently going to test with Routing Initiated Recording: Y - to see if this helps.

[williamconley]

but the lead id of the recording. is that "0"?

[kieran333]

but the lead id of the recording. is that "0"?

Apologies.... yes it is.

[williamconley]

Then it would seem likely that one of your extension contexts does not have "hangup" in it or one of your carriers may not have the agi script before the dial line OR perhaps you're changing the caller ID name. Somehow the Vicidial scripting system is losing track of these calls and the recordings then don't get updated.

This flaw is often only visible when "0" appears in a random table instead of the Vicidial Lead ID that should be there. We have had to chase this down several times and it's always a very technical mistake, but rarely one in Vicidial (usually it's a client-introduced error).

Also: Did you use the "express" install? Or standard "vicibox-install"?

[kieran333]

Hi William,

Just a quick positive update.

Since changing - Routing Initiated Recording to Y I have been unable to replicate the issue manually (test system) or on 2 live working systems (4 hours so far without a trace of the error).

FYI: I was able to replicate using vicibox-install & vicibox-express.

I can confirm that there isn't a "hangup" missing in any of the contexts, I am pedantic when it comes to dialplans

I will continue to monitor our live systems and test servers, but fingers crossed this is our fix!

Thanks for your assistance with this, much appreciated.

[williamconley]

Excellent postback. I hope you keep chasing it down for fun, but I can definitely understand why you'd call it quits at this point if you don't need to fix it now that it's not broken for you.

[kieran333]

Hi Will,

We appear to have narrowed it down to the manager_send.php file.

Please see image below, this is what is visible in developer console when the error appears. lead_ID: is blank including uniqueid

Just to rule out any hardware issues, this can be replicated on dedicated server & virtual. (Agent workstation is a 2018 Apple Mac, but also happens on Windows 10 Laptop).



Next step for us.... take a look at manager_send.php and apply a fix.

Do you suggest that I open this issue on the bug tracker?

Best regards
Kieran

[williamconley]

If you find the actual bug to fix, definitely post it. And a link to it in this post.