User Status shows Dead frequently

[ayan07]

Hi There ,

My Vicibox Server specification is here :

(Vicibox 8.1.2 from .iso | Version 2.14b0.5 | Asterisk 13.21.1 | Express Installation mode | No Digium/Sangoma Hardware | No Extra Software After Installation | Intel Xeon CPU)

Very frequently I noticed that Agent Status is real time screen showing as "DEAD" but if i go to that particular agents port , Agent Status showing as "IN CALL" though agents are not in call... Its happening with random agent ids and almost with everyone and we had to logoff and re logging again to resolve this... It never happened when I was using Vicibox-8 .... After started using Vicibox-8.1.2 this error started coming in ... Please Advise ...

Thanks in Advance...

Roy

[williamconley]

but if i go to that particular agents port

What does this mean?

[ayan07]

Suppose if i see agent id 101 Status in real time screen is showing as "DEAD" but if i go to that 101 agents login port their status showing ready but in Agent Status position showing as "IN CALL" though agents are not in call...

Please advise...

[ayan07]

Problem not yet resolved ...

station status frequently showing as "DEAD" and its showing for random stations ... had to logoff and logging back to ignore this but happening frequently ...

Any expert advise...

Thanks in Advance...

Roy

[roger.milligan]

Hi Roy

How many agents do you have on the Vici and how busy is your server? I've had a similar problem in the past where, due to performance problems (this site has 400+ agents), the RealTime screen was not reading the auto_calls table correctly. I put a whole lot of debugging into the code and can't remember what I finally did. I think that I built in a retry to do the DB lookup a second time before marking the agent as DEAD. Let me know if this sounds like it's on the right track and I'll did out what I did for you.

Roger

[ayan07]

Thanks Roger for your reply..

We are dialing with 10-15 agents max at time in vicibox 8.1.2 ... Previously it was ok while using vicibox 8.. Problem started after upgrading the vicibox versions.

Please Advise and Thanks in Advance...

[dspaan]

Wow this is a very weird problem. Did you check for time sync issues, go to the Reports page and at the bottom where you see your server check if all three times are the same when you press the + symbol.

[ayan07]

Yes Roger its same...

SERVER - DESCRIPTION IP ACT LOAD CHAN AGNT DISK TIME VER
vicibox81 Server vicibox81 192.168.1.240 Y / Y / Y 1290 - 65% 346 13 23% 2018-11-15 11:33:30 3047
PHP Time 2018-11-15 11:33:30
DB Time 2018-11-15 11:33:30

[roger.milligan]

Looked at my code changes and I basically did a double-check. When an agent is suspected of being DEAD in AST_timeonVDADall.php I check again and that eliminated most of the fake DEADs:

Code: Select all

#RJM Reselect Status and Re-Count auto calls
$stmtL="select status from vicidial_live_agents where user='$Auser[$i]';";
...
$stmtR="select * from vicidial_auto_calls where callerid='$Acallerid[$i]' or lead_id=$Alead_id[$i];";
...

It will be FAR easier to sort out your server performance than to change code. Also, take a look in MySQL "show processlist" and see what processes are waiting. Database locking may well be an issue.

Roger

[dspaan]

Is this dedicated hardware or virtualized?

[ayan07]

Please see result of mysql show processlist ...

MariaDB [(none)]> show processlist;
+---------+-------------+-----------+----------+---------+------+--------------------------+------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+---------+-------------+-----------+----------+---------+------+--------------------------+------------------+----------+
| 1 | system user | | NULL | Daemon | NULL | InnoDB purge coordinator | NULL | 0.000 |
| 2 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 3 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 4 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 5 | system user | | NULL | Daemon | NULL | InnoDB shutdown handler | NULL | 0.000 |
| 19 | cron | localhost | asterisk | Sleep | 1 | | NULL | 0.000 |
| 21 | cron | localhost | asterisk | Sleep | 0 | | NULL | 0.000 |
| 23 | cron | localhost | asterisk | Sleep | 1 | | NULL | 0.000 |
| 24 | cron | localhost | asterisk | Sleep | 0 | | NULL | 0.000 |
| 26 | cron | localhost | asterisk | Sleep | 0 | | NULL | 0.000 |
| 27 | cron | localhost | asterisk | Sleep | 0 | | NULL | 0.000 |
| 28 | cron | localhost | asterisk | Sleep | 0 | | NULL | 0.000 |
| 684519 | cron | localhost | asterisk | Sleep | 0 | | NULL | 0.000 |
| 2855303 | root | localhost | NULL | Query | 0 | init | show processlist | 0.000 |
| 2855332 | cron | localhost | asterisk | Sleep | 0 | | NULL | 0.000 |
+---------+-------------+-----------+----------+---------+------+--------------------------+------------------+----------+

[ayan07]

Dedicated Server..

My Vicibox Server specification is here :

(Vicibox 8.1.2 from .iso | Version 2.14b0.5 | Asterisk 13.21.1 | Express Installation mode | No Digium/Sangoma Hardware | No Extra Software After Installation | Intel Xeon CPU)

[ayan07]

Please note : mainly station status showing "DEAD" in real-time screen immediate after disposing the call...

We are running IVR blusting campaign and calls are landing inbound types , after disposing the call extension is showing as "IN CALL" in agent view and "DEAD" in real time screen though agent is in waiting status....

Please Advise ...

[dspaan]

We are running IVR blusting campaign

Do you mean a survey campaign where outbound calls are made and the caller gets an IVR and when a choice is made then is routed to an agent?

Please also check the vicibox bugfix thread: viewtopic.php?f=8&t=38745

[ayan07]

HI dspaan,

yes you are correct and i am using Vicibox 8.1.2 ...

Actually stations going "DEAD" in real-time screen immediate after disposing the call in that stations...

[williamconley]

Actually stations going "DEAD" in real-time screen immediate after disposing the call in that stations...

Please be clear: When someone "dispos" a call, they are no longer on a call and are then either "paused" or "ready", so DEAD shouldn't even be possible. If, however, they "hang up" the customer call and sit on the dispo screen without doing anything, once again the status should be "dispo" not DEAD. Only if the customer hangs up and the agent does not push the "hangup customer" button should it be DEAD.

So when you say after they dispo the call ... I have to wonder if you've actually checked the real status of that agent. Yes, agents WILL sit on their asses and stare at the screen after a prospect hangs up. And yes, that's why DEAD exists, to rat out those agents. Have you confirmed independently that this is not the case?

[ayan07]

Yes I have checked the status of those agents physically and they are disposing the calls correctly ,, they are disposing the call by using using hotkeys and manual disposition options too ..but again its happening after disposing calls ... its not happening in every calls ,, its happening randomly for all stations .. sometimes frequently and sometimes lately..

[williamconley]

To be clear: It's entirely possible that your "dispositioning correctly" statement and "DEAD" calls are an indication everything is working properly. If the agent does not click on "hangup customer" after the customer has hung up because the agent is still entering data, the agent will show "DEAD" until they complete their data entry and the click "hangup customer".

Perhaps you should provide the details button by button, moment by moment for screen display, for a single instance. Also post the Average Server Load (from "uptime" or "htop") during an instance. Also have a look at the mysql command "show processlist" and see if there are any backlogs (queries over 1 second) during your problems.

Does the problem occur at certain times of day? During certain "live agent" counts (ie: when you have the most agents dialing vs only 1 or 2), or can it literally happen with a single agent, too?

[dspaan]

There is also a debug option in system settings that you can enable.

[ayan07]

hi dspaan,

Thanks for your help... Please clarify which debugging option should i reset in system settings ?

Roy

[williamconley]

Admin->Servers->(Choose a server)

On this page look for the word debug. There are several options. Each has a ? button for deeper details.

Also in Admin->System Settings is "Agent Screen Debug Logging". Very handy.