You have been kicked from the conference

[Acidshock]

Ok having a weird one on a rather large system. Agents randomly with no noticeable pattern are getting "you have been kicked from this conference" messages and then the call drops. When looking up the call entries I see normal clearing, etc. When I look up the lead the call recording will show no time in the seconds field. I am sort of drawing a blank where to simply find the error at this point. All my logs look fine.

Code: Select all

listen.2015-07-02-Destination: SIP/CarrierA-00000029
listen.2015-07-02-CallerIDNum: XXXXXX0306
listen.2015-07-02-CallerIDName: M7020708180050168290
listen.2015-07-02-ConnectedLineNum: XXXXXX0306
listen.2015-07-02-ConnectedLineName: M7020708180050168290
listen.2015-07-02-UniqueID: 1435846099.131
listen.2015-07-02-DestUniqueID: 1435846099.133
listen.2015-07-02:Dialstring: CarrierA/1XXXXXX5326
listen.2015-07-02-
listen.2015-07-02-Event: Newstate
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/8600055@default-00000026;2
listen.2015-07-02-ChannelState: 5
listen.2015-07-02-ChannelStateDesc: Ringing
listen.2015-07-02-CallerIDNum: XXXXXX0306
--
listen.2015-07-02-AccountCode:
listen.2015-07-02-OldAccountCode:
listen.2015-07-02-
listen.2015-07-02-Event: NewCallerid
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;1
listen.2015-07-02-CallerIDNum:
listen.2015-07-02:CallerIDName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-Uniqueid: 1435846099.134
listen.2015-07-02-CID-CallingPres: 0 (Presentation Allowed, Not Screened)
listen.2015-07-02-
listen.2015-07-02-Event: Newstate
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;2
listen.2015-07-02-ChannelState: 6
listen.2015-07-02-ChannelStateDesc: Up
listen.2015-07-02-CallerIDNum:
listen.2015-07-02:CallerIDName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-ConnectedLineNum:
listen.2015-07-02:ConnectedLineName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-Uniqueid: 1435846099.135
listen.2015-07-02-
listen.2015-07-02-Event: Newstate
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;1
listen.2015-07-02-ChannelState: 6
listen.2015-07-02-ChannelStateDesc: Up
listen.2015-07-02-CallerIDNum:
listen.2015-07-02:CallerIDName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-ConnectedLineNum:
listen.2015-07-02:ConnectedLineName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-Uniqueid: 1435846099.134
listen.2015-07-02-
listen.2015-07-02-Event: MonitorStart
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;1
listen.2015-07-02-Uniqueid: 1435846099.134
listen.2015-07-02-
--
listen.2015-07-02-2015-07-02 7:08:16|Event: MeetmeJoin
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;2
listen.2015-07-02-Uniqueid: 1435846099.135
listen.2015-07-02-Meetme: 8600055
listen.2015-07-02-Usernum: 3
listen.2015-07-02-CallerIDnum: <unknown>
listen.2015-07-02:CallerIDname: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-ConnectedLineNum: <unknown>
listen.2015-07-02:ConnectedLineName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-
listen.2015-07-02-|
listen.2015-07-02-2015-07-02 7:08:16|Event: MeetmeMute
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;2
listen.2015-07-02-Uniqueid: 1435846099.135
listen.2015-07-02-Meetme: 8600055
--
listen.2015-07-02-Event: MeetmeLeave
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;2
listen.2015-07-02-Uniqueid: 1435846099.135
listen.2015-07-02-Meetme: 8600055
listen.2015-07-02-Usernum: 3
listen.2015-07-02-CallerIDNum: <unknown>
listen.2015-07-02:CallerIDName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-ConnectedLineNum: <unknown>
listen.2015-07-02:ConnectedLineName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-Duration: 1617
listen.2015-07-02-
listen.2015-07-02-Event: Hangup
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;2
listen.2015-07-02-Uniqueid: 1435846099.135
listen.2015-07-02-CallerIDNum: <unknown>
listen.2015-07-02:CallerIDName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-ConnectedLineNum: <unknown>
listen.2015-07-02:ConnectedLineName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-Cause: 16
listen.2015-07-02-Cause-txt: Normal Clearing
listen.2015-07-02-
listen.2015-07-02-Event: Hangup
listen.2015-07-02-Privilege: call,all
listen.2015-07-02-Channel: Local/58600055@default-00000027;1
listen.2015-07-02-Uniqueid: 1435846099.134
listen.2015-07-02-CallerIDNum: <unknown>
listen.2015-07-02:CallerIDName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-ConnectedLineNum: <unknown>
listen.2015-07-02:ConnectedLineName: 20150702-070818_XXXXXX5326_SALES_3007_50168290
listen.2015-07-02-Cause: 16
listen.2015-07-02-Cause-txt: Normal Clearing
listen.2015-07-02-

Code: Select all

action_send.2015-07-02-2015-07-02 7:08:19|1|88469825|
action_send.2015-07-02-Action: Originate
action_send.2015-07-02:Exten: 91XXXXXX5326
action_send.2015-07-02-Context: default
action_send.2015-07-02-Channel: Local/8600055@default
action_send.2015-07-02-Priority: 1
action_send.2015-07-02-Callerid: "M7020708180050168290" <XXXXXX0306>
action_send.2015-07-02-Timeout: 60000
action_send.2015-07-02-
action_send.2015-07-02-2015-07-02 7:08:19|1|88469826|
action_send.2015-07-02-Action: Originate
action_send.2015-07-02-Channel: Local/58600055@default
action_send.2015-07-02-Context: default
action_send.2015-07-02-Exten: 8309
action_send.2015-07-02-Priority: 1
action_send.2015-07-02:Callerid: 20150702-070818_XXXXXX5326_SALES_3007_50168290
action_send.2015-07-02-
action_send.2015-07-02-2015-07-02 7:08:29|1|88469827|
action_send.2015-07-02-Action: Originate
action_send.2015-07-02-Exten: 83047777777777
action_send.2015-07-02-Context: vicidial-auto
action_send.2015-07-02-Channel: Local/192*168*000*033*78600056@default
action_send.2015-07-02-Priority: 1
--


Code: Select all

----BEGIN NEW COMMAND----
action_process.2015-07-02-CallerID: M7020708180050168290
action_process.2015-07-02-Action: Originate
action_process.2015-07-02:Exten: 91XXXXXX5326
action_process.2015-07-02-Context: default
action_process.2015-07-02-Channel: Local/8600055@default
action_process.2015-07-02-Priority: 1
action_process.2015-07-02-Callerid: "M7020708180050168290" <XXXXXX0306>
action_process.2015-07-02-Timeout: 60000
action_process.2015-07-02-
action_process.2015-07-02-----END NEW COMMAND----
--
action_process.2015-07-02-2015-07-02 7:08:19|----BEGIN NEW COMMAND----
action_process.2015-07-02-CallerID: 20150702-070818_2818
action_process.2015-07-02-Action: Originate
action_process.2015-07-02-Channel: Local/58600055@default
action_process.2015-07-02-Context: default
action_process.2015-07-02-Exten: 8309
action_process.2015-07-02-Priority: 1
action_process.2015-07-02:Callerid: 20150702-070818_XXXXXX5326_SALES_3007_50168290
action_process.2015-07-02-
action_process.2015-07-02-----END NEW COMMAND----
action_process.2015-07-02-|
action_process.2015-07-02-2015-07-02 7:08:19|SQL_QUERY|UPDATE vicidial_manager set status='SENT' where man_id='88469826'||
action_process.2015-07-02-2015-07-02 7:08:29|SQL_QUERY|SELECT man_id,uniqueid,entry_date,status,response,server_ip,channel,action,callerid,cmd_line_b,cmd_line_c,cmd_line_d,cmd_line_e,cmd_line_f,cmd_line_g,cmd_line_h,cmd_line_i,cmd_line_j,cmd_line_k FROM vicidial_manager where server_ip = '192.168.0.33' and status = 'QUEUE' order by man_id limit 1||
action_process.2015-07-02-2015-07-02 7:08:29|----BEGIN NEW COMMAND----
action_process.2015-07-02-CallerID: VH7020708298600056
--
action_process.2015-07-02-----END NEW COMMAND----

Code: Select all

screenlog.0-[Jul  2 07:08:16]     -- Executing [8309@default:3] Wait("Local/58600051@default-00000025;1", "7200") in new stack
screenlog.0-[Jul  2 07:08:17]   == Manager 'sendcron' logged off from 127.0.0.1
screenlog.0-[Jul  2 07:08:17]   == Manager 'sendcron' logged off from 127.0.0.1
screenlog.0-[Jul  2 07:08:17]     -- SIP/CarrierA-00000028 is ringing
screenlog.0-[Jul  2 07:08:19]   == Manager 'sendcron' logged on from 127.0.0.1
screenlog.0-[Jul  2 07:08:19]     -- Executing [8600055@default:1] MeetMe("Local/8600055@default-00000026;2", "8600055,F") in new stack
screenlog.0-[Jul  2 07:08:19]        > Channel Local/8600055@default-00000026;1 was answered.
screenlog.0:[Jul  2 07:08:19]     -- Executing [91XXXXXX5326@default:1] AGI("Local/8600055@default-00000026;1", "agi://127.0.0.1:4577/call_log") in new stack
screenlog.0-[Jul  2 07:08:19]     -- <Local/8600055@default-00000026;1>AGI Script agi://127.0.0.1:4577/call_log completed, returning 0
screenlog.0:[Jul  2 07:08:19]     -- Executing [91XXXXXX5326@default:2] Dial("Local/8600055@default-00000026;1", "SIP/CarrierA/1XXXXXX5326,,tTor") in new stack
screenlog.0-[Jul  2 07:08:19]   == Using SIP RTP CoS mark 5
screenlog.0:[Jul  2 07:08:19]     -- Called SIP/CarrierA/1XXXXXX5326
screenlog.0-[Jul  2 07:08:19]   == Manager 'sendcron' logged on from 127.0.0.1
screenlog.0-[Jul  2 07:08:19]     -- Executing [58600055@default:1] MeetMe("Local/58600055@default-00000027;2", "8600055,Fmq") in new stack
screenlog.0-[Jul  2 07:08:19]        > Channel Local/58600055@default-00000027;1 was answered.
screenlog.0-[Jul  2 07:08:19]     -- Executing [8309@default:1] Answer("Local/58600055@default-00000027;1", "") in new stack
screenlog.0:[Jul  2 07:08:20]     -- Executing [8309@default:2] Monitor("Local/58600055@default-00000027;1", "wav,20150702-070818_XXXXXX5326_SALES_3007_50168290") in new stack
screenlog.0-[Jul  2 07:08:20]     -- Executing [8309@default:3] Wait("Local/58600055@default-00000027;1", "7200") in new stack
screenlog.0-[Jul  2 07:08:20]   == Manager 'sendcron' logged off from 127.0.0.1
screenlog.0-[Jul  2 07:08:20]   == Manager 'sendcron' logged off from 127.0.0.1
screenlog.0-[Jul  2 07:08:21]     -- SIP/CarrierA-00000029 is ringing
screenlog.0-[Jul  2 07:08:29]   == Manager 'sendcron' logged on from 127.0.0.1
screenlog.0-[Jul  2 07:08:29]     -- Executing [192*168*000*033*78600056@default:1] Goto("Local/192*168*000*033*78600056@default-00000028;2", "default,78600056,1") in new stack
screenlog.0-[Jul  2 07:08:29]     -- Goto (default,78600056,1)
--
screenlog.0-[Jul  2 07:35:20]     -- Executing [8600055@default:2] Hangup("Local/8600055@default-00000026;2", "") in new stack
screenlog.0-[Jul  2 07:35:20]   == Spawn extension (default, 8600055, 2) exited non-zero on 'Local/8600055@default-00000026;2'
screenlog.0-[Jul  2 07:35:20]     -- Executing [h@default:1] AGI("Local/8600055@default-00000026;2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----16---------------") in new stack
screenlog.0-[Jul  2 07:35:20]     -- <SIP/3007-00000022>AGI Script agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----16--------------- completed, returning 0
screenlog.0-[Jul  2 07:35:20]     -- <Local/8600055@default-00000026;2>AGI Script agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----16--------------- completed, returning 0


[mflorell]

Well, the version in your signature is over 2 years old, so my first recommendation would be to upgrade.

[williamconley]

Is it possible that the agent's never answered their phones? Or perhaps that Vicidial believes they did not answer? Large system ... clustering could be a challenge, how did you cluster? Or is it a single server? This could also be overload ...

[Acidshock]

Clustered. Just upgraded everything to:
VERSION: 2.12-493a
BUILD: 150703-2105

6 asterisk boxes
1 DB server
1 Archive
1 Web Server

Also did a DB upgrade. Got dual 8 core Xeon processors, 12 SSD's in raid 10, and 128GB of memory for the database server. Seems like the problem has subsided. Got a few other problems though. Going to post another 2 threads though for each problem.