Problems after installing Sangoma Lyra

[runderwoodcr]

Hello Community,

Let me try to explain the issue that I'm experiencing, I've installed Sangoma Lyra on a Windows Server(IBM x Series with 2 Intel Xeon with 4 Cores each and 8GB of RAM) after testing the environment it seemed that everything was working fine, so I moved it to production and here is where the issues started.
We are recording all the calls we made for QA and training, those recordings has been working fine till I moved the cpd on the loop, now what its going on is that the recordings are getting messed up, for some reason when a call comes into the agent and the agent dispo the call after the talk the call get hanged up and he gets another call as normal, but the recordings are getting mixed up, so the first call comes the call its recorded now the agent is hanging the call and dispo the call, the next call comes and its recorded as well too but on the previous recording for the previous call its also including the new call recording, so I get two calls two call recordings but one of them contains the recording of two calls, now this is the odd part, this doesn't happen with all the calls just with a few ones, also this wasn't happening before, the setup has been running for 5 months with no issues until I included the Sangoma Lyra in the loop, an I forgot the first recording file which includes the two calls recorded appears in the Vici reports with length of 0 Seconds, everything else its working fine at least for now, if someone has experienced this in the past I will really appreciate the advice.

Regards,

Ricardo

[williamconley]

First: Test that you can remove Lyra and fix the problem. After all, I've known many technicians who chased a problem they thought was the result of one change only to find out it was the result of another simultaneous (or recent) change or even a completely unrelated bug that just happened to occur during the same time period. So ... please check that to be sure you have identified the culprit.

Next: If Lyra uses the Sangoma CPD settings and signalling, you may need to be sure that it uses them the same way Netborder does. There may be differences that the Vicidial Group has not yet had the opportunity to integrate. For this, see if you can check the logs (carrier logs, whatever logs, heck create new Lyra logs if you need to!) that will show the "ultimate" disconnect code that was used for each. This may tell you if it is just a single Lyra signal that is out of whack and in need of updating (based on version of Lyra/CPD signalling).

Happy Hunting!

[runderwoodcr]

Hello William,

Thanks for your reply, will do that, will post more info when available, as per the the problem am 100% sure its related with the addition of Lyra, that has been the only change performed in the last 5 months to the setup, but nevertheless will look for any undocumented change.

[williamconley]

Believe me: You must bypass lyra and verify cessation of the problem. Don't be sure .. "Check!". Doesn't cost any money ...

Also, another post yesterday indicated that the "listen" screen was related to a similar "combined recordings" issue. FYI.

viewtopic.php?f=4&t=10879&p=95766#p95766

[runderwoodcr]

Hello William,

You were right, after I checked the logs and asked the QA team to look at the calls from the previous month we found that we have been having this issue for a couple of month now, so I can say that probably the Sangoma Lyra its not the issue. I looked at the post about the listen screen, however mine doesn't seem to be stuck its actually pretty busy.
I found something that maybe its causing this problem or maybe not, but it could be related, I've seem that several agents are using the same session, meaning the same conf room on the same server which shouldn't be, its a cluster so the conf room could be repeated but in different server not on the same, let me give you an extract from the vicidial_session_data:

1362084468_106519879572 121 PRM 10.0.5.46 8600057 1065 2013-02-28 14:47:48 ||2013-02-28 14:47:48|NEW|N|10.0.5.46||Originate|S1302281447488600057|Channel: SIP/1065|Context: default|Exten: 8600057|Priority: 1|Callerid: S1302281447488600057|||||

1362166601_107211797230 154 PRM 10.0.5.46 8600057 1072 2013-03-01 13:36:41 ||2013-03-01 13:36:41|NEW|N|10.0.5.46||Originate|S1303011336418600057|Channel: SIP/1072|Context: default|Exten: 8600057|Priority: 1|Callerid: S1303011336418600057|||||

This is the current load on the Servers:
Dialer 1:
top - 14:53:24 up 19 days, 18:32, 2 users, load average: 0.12, 0.14, 0.14
Tasks: 235 total, 2 running, 232 sleeping, 0 stopped, 1 zombie
Cpu(s): 0.3%us, 0.1%sy, 0.0%ni, 99.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 12014M total, 9042M used, 2972M free, 282M buffers
Swap: 3075M total, 0M used, 3075M free, 7981M cached

Dialer 2:
top - 14:53:38 up 19 days, 18:36, 2 users, load average: 0.09, 0.11, 0.07
Tasks: 212 total, 1 running, 211 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 12014M total, 10056M used, 1958M free, 270M buffers
Swap: 3075M total, 0M used, 3075M free, 8897M cached

Web Gui:

top - 14:53:55 up 19 days, 18:49, 2 users, load average: 0.00, 0.01, 0.05
Tasks: 175 total, 1 running, 174 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.6%us, 0.1%sy, 0.0%ni, 99.1%id, 0.1%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 4057380k total, 1422536k used, 2634844k free, 236396k buffers
Swap: 2097148k total, 0k used, 2097148k free, 734428k cached

[williamconley]

you'll need to check your astguiclient logs to find the process that made the decision to put that agent in the same room as the other agent. either it acquired faulty data, placed there previously in error by another process, or it "broke" even though it had the correct information. you'll need to have agi logging on for both servers to catch this sort of problem, and learn what happens in each of the log files to track it down.

and since you're clustered, it would not be a bad idea to verify that your servers are truly time-synced as in they have the same time of day (exactly) on all the servers especially at the moment something like this happens. not saying it's related, just that time sync is important and easily checked (and removed from the running as a problem).

[runderwoodcr]

Hello William.

Thanks for the reply, didn't mentioned on the previous post, but actually the time sync was the first thing I looked at, I do have an ntp server on the network and all my servers sync against it, and the Vici Servers are in correct sync not even 1 sec of diff between them, as per the logs debugging, do you happen to know which log file is, or what to look for, as per I've been searching on the listen and action_full but can't seem to find the entries about the agent login, or probably am not looking for the right information, any hint or advise its highly appreciated.

[williamconley]

I'd love to, but you're going to have to decipher them, just like our technicians do. LOL. there is no manual or legend or schema definition at present.

[runderwoodcr]

Hello,

Still debugging this problem, after all agents logs out is it normal to still see data on the table vicidial_session_data, as per I can see the data still there, I mean I can see all the information about the agents sessions but they are not connected anymore and the Vici Real Time report doesn't show then online, also from the live_channels table there isn't any live channel

[williamconley]

Probably. I've never looked. But if you want to test against "should it look this way", set up a sample server in a virtual system "stock" with no changes in it at all. Then you'll have something to compare to.