DROP where no drop occured 2.14-756a 200609-

[richgen]

Hi,

I've been using vici for a number of years now and its been fine, but recently its reporting 25% or more drop rate which is nonsense even when set to RATIO 1:1 and with agents available.

We use REMOTE agents, not the vici gui but that has always worked just fine. Single Server; a HPDL360G10, 32M, and load averages are 0.06 0.04 0.05. Asterisk 13.34.0-vici.

The issue we see is that the call is reported as status=DROP, term_reason=ABANDON when in fact an agent took the call.

Turning on ALL the asterisk debug I tracked a 'DROP' and find that:

Code: Select all

11:00:00 Upstream Ring
11:00:06 Upstream ANSWER V0201059590011424779
11:00:06 Agent INVITE -> 8543@
11:00:08 Agent ANSWER/200
11:00:08 Bridge up
11:00:08 RTP Stream start
11:00:13 Agent BYE
11:00:13 RTP Stream end
11:00:13 Bridge Down
11:00:13 Upstream BYE

But for some reason this call is coded as DROP. I even have a call recording, and listening to that it was a live call to an answerphone.

Any help/pointers would be very gratefully received as I'm at a loss right now.

The actual record from vicidial_log

Code: Select all

`lead_id` = 11424779
`list_id` = 974
`campaign_id` = '74'
`call_date` = '2022-10-20 11:00:06'
`start_epoch` = 1666260006
`end_epoch` = 1666260013
`length_in_sec` = 7
`status` = 'DROP'
`phone_code` = ''
`phone_number` = '01782.......'
`user` = 'VDAD'
`comments` = '0.008'
`processed` = 'N'
`user_group` = NULL
`term_reason` = 'ABANDON'
`alt_dial` = 'NONE'
`called_count` = 1
`uniqueid` = '1666259999.2451'

For completeness, here's the campaign setup:

Code: Select all

`campaign_id` = '74'
`campaign_name` = 'testtree'
`active` = 'Y'
`dial_status_a` = ''
`dial_status_b` = ''
`dial_status_c` = ''
`dial_status_d` = ''
`dial_status_e` = ''
`lead_order` = 'DOWN'
`park_ext` = ''
`park_file_name` = ''
`web_form_address` = ''
`allow_closers` = 'N'
`hopper_level` = 200
`auto_dial_level` = '1.1'
`next_agent_call` = 'random'
`local_call_time` = '24hours'
`voicemail_ext` = ''
`dial_timeout` = 20
`dial_prefix` = '9'
`campaign_cid` = '0114.......'
`campaign_vdad_exten` = '8368'
`campaign_rec_exten` = '8309'
`campaign_recording` = 'ALLFORCE'
`campaign_rec_filename` = 'FULLDATE-CUSTPHONE-CALLID-AGENT'
`campaign_script` = ''
`get_call_launch` = 'NONE'
`am_message_exten` = 'vm-goodbye'
`amd_send_to_vmx` = 'N'
`xferconf_a_dtmf` = ''
`xferconf_a_number` = ''
`xferconf_b_dtmf` = ''
`xferconf_b_number` = ''
`alt_number_dialing` = 'N'
`scheduled_callbacks` = 'N'
`lead_filter_id` = 'NONE'
`drop_call_seconds` = 0
`drop_action` = 'AUDIO'
`safe_harbor_exten` = '8307'
`display_dialable_count` = 'Y'
`wrapup_seconds` = 0
`wrapup_message` = 'Wrapup Call'
`closer_campaigns` = NULL
`use_internal_dnc` = 'N'
`allcalls_delay` = 0
`omit_phone_code` = 'N'
`dial_method` = 'RATIO'
`available_only_ratio_tally` = 'Y'
`adaptive_dropped_percentage` = '2.9'
`adaptive_maximum_level` = '4'
`adaptive_latest_server_time` = '1900'
`adaptive_intensity` = '0'
`adaptive_dl_diff_target` = 1
`concurrent_transfers` = 'AUTO'
`auto_alt_dial` = 'NONE'
`auto_alt_dial_statuses` = ' B N NA DC -'
`agent_pause_codes_active` = 'N'
`campaign_description` = 'Home Care'
`campaign_changedate` = '2022-10-20 09:53:02'
`campaign_stats_refresh` = 'N'
`campaign_logindate` = '2022-10-20 11:23:25'
`dial_statuses` = ' NEW -'
`disable_alter_custdata` = 'N'
`no_hopper_leads_logins` = 'N'
`list_order_mix` = 'DISABLED'
`campaign_allow_inbound` = 'N'
`manual_dial_list_id` = 998
`default_xfer_group` = '---NONE---'
`xfer_groups` = ''
`queue_priority` = 50
`drop_inbound_group` = '---NONE---'
`qc_enabled` = 'N'
`qc_statuses` = NULL
`qc_lists` = NULL
`qc_shift_id` = '24HRMIDNIGHT'
`qc_get_record_launch` = 'NONE'
`qc_show_recording` = 'Y'
`qc_web_form_address` = NULL
`qc_script` = NULL
`survey_first_audio_file` = 'US_pol_survey_hello'
`survey_dtmf_digits` = '1238'
`survey_ni_digit` = '8'
`survey_opt_in_audio_file` = 'US_pol_survey_transfer'
`survey_ni_audio_file` = 'US_thanks_no_contact'
`survey_method` = 'AGENT_XFER'
`survey_no_response_action` = 'OPTIN'
`survey_ni_status` = 'NI'
`survey_response_digit_map` = '1-DEMOCRAT|2-REPUBLICAN|3-INDEPENDANT|8-OPTOUT|X-NO RESPONSE|'
`survey_xfer_exten` = '8300'
`survey_camp_record_dir` = '/home/survey'
`disable_alter_custphone` = 'Y'
`display_queue_count` = 'Y'
`manual_dial_filter` = 'NONE'
`agent_clipboard_copy` = 'NONE'
`agent_extended_alt_dial` = 'N'
`use_campaign_dnc` = 'N'
`three_way_call_cid` = 'CAMPAIGN'
`three_way_dial_prefix` = ''
`web_form_target` = 'vdcwebform'
`vtiger_search_category` = 'LEAD'
`vtiger_create_call_record` = 'Y'
`vtiger_create_lead_record` = 'Y'
`vtiger_screen_login` = 'Y'
`cpd_amd_action` = 'DISABLED'
`agent_allow_group_alias` = 'N'
`default_group_alias` = ''
`vtiger_search_dead` = 'ASK'
`vtiger_status_call` = 'N'
`survey_third_digit` = ''
`survey_third_audio_file` = 'US_thanks_no_contact'
`survey_third_status` = 'NI'
`survey_third_exten` = '8300'
`survey_fourth_digit` = ''
`survey_fourth_audio_file` = 'US_thanks_no_contact'
`survey_fourth_status` = 'NI'
`survey_fourth_exten` = '8300'
`drop_lockout_time` = '0'
`quick_transfer_button` = 'N'
`prepopulate_transfer_preset` = 'N'
`drop_rate_group` = 'DISABLED'
`view_calls_in_queue` = 'NONE'
`view_calls_in_queue_launch` = 'MANUAL'
`grab_calls_in_queue` = 'N'
`call_requeue_button` = 'N'
`pause_after_each_call` = 'N'
`no_hopper_dialing` = 'N'
`agent_dial_owner_only` = 'NONE'
`agent_display_dialable_leads` = 'N'
`web_form_address_two` = ''
`waitforsilence_options` = ''
`agent_select_territories` = 'N'
`campaign_calldate` = '2022-10-20 11:23:25'
`crm_popup_login` = 'N'
`crm_login_address` = ''
`timer_action` = 'NONE'
`timer_action_message` = ''
`timer_action_seconds` = 1
`start_call_url` = ''
`dispo_call_url` = ''
`xferconf_c_number` = ''
`xferconf_d_number` = ''
`xferconf_e_number` = ''
`use_custom_cid` = 'N'
`scheduled_callbacks_alert` = 'NONE'
`queuemetrics_callstatus_override` = 'DISABLED'
`extension_appended_cidname` = 'N'
`scheduled_callbacks_count` = 'ALL_ACTIVE'
`manual_dial_override` = 'NONE'
`blind_monitor_warning` = 'DISABLED'
`blind_monitor_message` = 'Someone is blind monitoring your session'
`blind_monitor_filename` = ''
`inbound_queue_no_dial` = 'DISABLED'
`timer_action_destination` = ''
`enable_xfer_presets` = 'DISABLED'
`hide_xfer_number_to_dial` = 'DISABLED'
`manual_dial_prefix` = ''
`customer_3way_hangup_logging` = 'ENABLED'
`customer_3way_hangup_seconds` = 5
`customer_3way_hangup_action` = 'NONE'
`ivr_park_call` = 'DISABLED'
`ivr_park_call_agi` = ''
`manual_preview_dial` = 'PREVIEW_AND_SKIP'
`realtime_agent_time_stats` = 'CALLS_WAIT_CUST_ACW_PAUSE'
`use_auto_hopper` = 'Y'
`auto_hopper_multi` = '1'
`auto_hopper_level` = 9
`auto_trim_hopper` = 'Y'
`api_manual_dial` = 'STANDARD'
`manual_dial_call_time_check` = 'DISABLED'
`display_leads_count` = 'N'
`lead_order_randomize` = 'N'
`lead_order_secondary` = 'LEAD_ASCEND'
`per_call_notes` = 'DISABLED'
`my_callback_option` = 'UNCHECKED'
`agent_lead_search` = 'DISABLED'
`agent_lead_search_method` = 'CAMPLISTS_ALL'
`queuemetrics_phone_environment` = ''
`auto_pause_precall` = 'N'
`auto_pause_precall_code` = 'PRECAL'
`auto_resume_precall` = 'N'
`manual_dial_cid` = 'CAMPAIGN'
`post_phone_time_diff_alert` = 'DISABLED'
`custom_3way_button_transfer` = 'DISABLED'
`available_only_tally_threshold` = 'WAITING_AGENTS'
`available_only_tally_threshold_agents` = 1
`dial_level_threshold` = 'DISABLED'
`dial_level_threshold_agents` = 1
`safe_harbor_audio` = 'buzz'
`safe_harbor_menu_id` = ''
`survey_menu_id` = ''
`callback_days_limit` = 0
`dl_diff_target_method` = 'ADAPT_CALC_ONLY'
`disable_dispo_screen` = 'DISPO_DISABLED'
`disable_dispo_status` = ''
`screen_labels` = '--SYSTEM-SETTINGS--'
`status_display_fields` = 'CALLID'
`na_call_url` = ''
`survey_recording` = 'N'
`pllb_grouping` = 'DISABLED'
`pllb_grouping_limit` = 50
`call_count_limit` = 0
`call_count_target` = 3
`callback_hours_block` = 0
`callback_list_calltime` = 'DISABLED'
`user_group` = '---ALL---'
`hopper_vlc_dup_check` = 'N'
`in_group_dial` = 'DISABLED'
`in_group_dial_select` = 'CAMPAIGN_SELECTED'
`safe_harbor_audio_field` = 'DISABLED'
`pause_after_next_call` = 'DISABLED'
`owner_populate` = 'DISABLED'
`use_other_campaign_dnc` = ''
`allow_emails` = 'N'
`amd_inbound_group` = '---NONE---'
`amd_callmenu` = '---NONE---'
`survey_wait_sec` = 10
`manual_dial_lead_id` = 'N'
`dead_max` = 0
`dead_max_dispo` = 'DCMX'
`dispo_max` = 0
`dispo_max_dispo` = 'DISMX'
`pause_max` = 0
`max_inbound_calls` = 0
`manual_dial_search_checkbox` = 'SELECTED'
`hide_call_log_info` = 'N'
`timer_alt_seconds` = 0
`wrapup_bypass` = 'ENABLED'
`wrapup_after_hotkey` = 'DISABLED'
`callback_active_limit` = 0
`callback_active_limit_override` = 'N'
`allow_chats` = 'N'
`comments_all_tabs` = 'DISABLED'
`comments_dispo_screen` = 'DISABLED'
`comments_callback_screen` = 'DISABLED'
`qc_comment_history` = 'CLICK'
`show_previous_callback` = 'ENABLED'
`clear_script` = 'DISABLED'
`cpd_unknown_action` = 'DISABLED'
`manual_dial_search_filter` = 'NONE'
`web_form_address_three` = ''
`manual_dial_override_field` = 'ENABLED'
`status_display_ingroup` = 'ENABLED'
`customer_gone_seconds` = 30
`agent_display_fields` = ''
`am_message_wildcards` = 'N'
`manual_dial_timeout` = ''
`routing_initiated_recordings` = 'Y'
`manual_dial_hopper_check` = 'N'
`callback_useronly_move_minutes` = 0
`ofcom_uk_drop_calc` = 'N'
`manual_auto_next` = 0
`manual_auto_show` = 'N'
`allow_required_fields` = 'N'
`dead_to_dispo` = 'DISABLED'
`agent_xfer_validation` = 'N'
`ready_max_logout` = 0
`callback_display_days` = 0
`three_way_record_stop` = 'N'
`hangup_xfer_record_start` = 'N'
`scheduled_callbacks_email_alert` = 'N'
`max_inbound_calls_outcome` = 'DEFAULT'
`manual_auto_next_options` = 'DEFAULT'
`agent_screen_time_display` = 'DISABLED'
`next_dial_my_callbacks` = 'DISABLED'
`inbound_no_agents_no_dial_container` = ''
`inbound_no_agents_no_dial_threshold` = 0
`cid_group_id` = '---DISABLED---'
`pause_max_dispo` = 'PAUSMX'
`script_top_dispo` = 'N'
`dead_trigger_seconds` = 0
`dead_trigger_action` = 'DISABLED'
`dead_trigger_repeat` = 'NO'
`dead_trigger_filename` = ''
`dead_trigger_url` = ''
`scheduled_callbacks_force_dial` = 'N'
`scheduled_callbacks_auto_reschedule` = 'DISABLED'
`scheduled_callbacks_timezones_container` = 'DISABLED'
`three_way_volume_buttons` = 'ENABLED'
`callback_dnc` = 'DISABLED'
`manual_dial_validation` = 'N'
`mute_recordings` = 'N'
`auto_active_list_new` = 'DISABLED'
`call_quota_lead_ranking` = 'DISABLED'
`call_quota_process_running` = 0
`call_quota_last_run_date` = NULL
`sip_event_logging` = 'DISABLED'
`campaign_script_two` = ''
`leave_vm_no_dispo` = 'DISABLED'
`leave_vm_message_group_id` = '---NONE---'
`dial_timeout_lead_container` = 'DISABLED'
`amd_type` = 'AMD'
`vmm_daily_limit` = 0
`opensips_cid_name` = ''
`amd_agent_route_options` = 'DISABLED'
`browser_alert_sound` = '---NONE---'
`browser_alert_volume` = 50
`three_way_record_stop_exception` = 'DISABLED'

[jamiemurray]

Drop call seconds needs to be higher than 0, try setting this to 2 and see if the issue persists.

[richgen]

Drop call seconds needs to be higher than 0, try setting this to 2 and see if the issue persists.

I've set the ratio 1:1 campaign to Drop Call Seconds = 2 and will let you know, Thanks for the suggestion!

[richgen]

Setting the "Drop Call Seconds" to 2 has had no effect, still showing 25% drop rate and I can still show that every one of those got to an agents phone

Outbound only, low volume, high availability and ratio 1:1.
When I match CLI's between vici call log and agent coded outcomes in a third party system that's driven on softphone answers I can match every call that vici thinks is a 'DROP' to an agent outcome.
They are almost all short calls of a few seconds coded as answerphones (confirmed by call recordings) but not all, there are a few longer calls of a few minutes. Its not one agent, carrier or campaign.

Any idea's gratefully appreciated?

[jamiemurray]

What SVN version are you on?
Install method?
Any customizations performed on any of the perl scripts in /usr/share/astguiclient or /var/lib/asterisk/agi-bin
Output of screen -ls too

[jamiemurray]

Also could you share the remote agents' settings?

[richgen]

Also could you share the remote agents' settings?

Code: Select all

There are screens on:
   3714.ASTemail   (Detached)
   3711.ASTVDadFILL   (Detached)
   3708.ASTfastlog   (Detached)
   3705.ASTVDadapt   (Detached)
   3702.ASTVDremote   (Detached)
   3699.ASTVDauto   (Detached)
   3696.ASTlisten   (Detached)
   3693.ASTsend   (Detached)
   3690.ASTupdate   (Detached)
   3375.asterisk   (Detached)
   3370.astshell20221021030421   (Detached)
11 Sockets in /run/screens/S-root.

Remote Agents

Code: Select all

`remote_agent_id` = 190
`user_start` = '8545'
`number_of_lines` = 1
`server_ip` = '81.130.73.1'
`conf_exten` = '28545'
`status` = 'INACTIVE'
`campaign_id` = '75'
`closer_campaigns` = ''
`extension_group` = 'NONE'
`extension_group_order` = 'NONE'
`on_hook_agent` = 'N'
`on_hook_ring_time` = 15

Same User

Code: Select all

`user_id` = 218
`user` = '8545'
`pass` = '54588545'
`full_name` = 'Brendan'
`user_level` = 4
`user_group` = 'Agents'
`phone_login` = '28545'
`phone_pass` = 'ur22'
`delete_users` = '0'
`delete_user_groups` = '0'
`delete_lists` = '0'
`delete_campaigns` = '0'
`delete_ingroups` = '0'
`delete_remote_agents` = '0'
`load_leads` = '0'
`campaign_detail` = '0'
`ast_admin_access` = '0'
`ast_delete_phones` = '0'
`delete_scripts` = '0'
`modify_leads` = '0'
`hotkeys_active` = '0'
`change_agent_campaign` = '0'
`agent_choose_ingroups` = '1'
`closer_campaigns` = ''
`scheduled_callbacks` = '1'
`agentonly_callbacks` = '0'
`agentcall_manual` = '0'
`vicidial_recording` = '1'
`vicidial_transfers` = '1'
`delete_filters` = '0'
`alter_agent_interface_options` = '0'
`closer_default_blended` = '0'
`delete_call_times` = '0'
`modify_call_times` = '0'
`modify_users` = '0'
`modify_campaigns` = '0'
`modify_lists` = '0'
`modify_scripts` = '0'
`modify_filters` = '0'
`modify_ingroups` = '0'
`modify_usergroups` = '0'
`modify_remoteagents` = '0'
`modify_servers` = '0'
`view_reports` = '0'
`vicidial_recording_override` = 'DISABLED'
`alter_custdata_override` = 'NOT_ACTIVE'
`qc_enabled` = '0'
`qc_user_level` = 1
`qc_pass` = '0'
`qc_finish` = '0'
`qc_commit` = '0'
`add_timeclock_log` = '0'
`modify_timeclock_log` = '0'
`delete_timeclock_log` = '0'
`alter_custphone_override` = 'NOT_ACTIVE'
`vdc_agent_api_access` = '0'
`modify_inbound_dids` = '0'
`delete_inbound_dids` = '0'
`active` = 'Y'
`alert_enabled` = '0'
`download_lists` = '0'
`agent_shift_enforcement_override` = 'DISABLED'
`manager_shift_enforcement_override` = '0'
`shift_override_flag` = '0'
`export_reports` = '0'
`delete_from_dnc` = '0'
`email` = ''
`user_code` = ''
`territory` = ''
`allow_alerts` = '0'
`agent_choose_territories` = '0'
`custom_one` = ''
`custom_two` = ''
`custom_three` = ''
`custom_four` = ''
`custom_five` = ''
`voicemail_id` = ''
`agent_call_log_view_override` = 'DISABLED'
`callcard_admin` = '0'
`agent_choose_blended` = '1'
`realtime_block_user_info` = '0'
`custom_fields_modify` = '0'
`force_change_password` = 'N'
`agent_lead_search_override` = 'NOT_ACTIVE'
`modify_shifts` = '0'
`modify_phones` = '0'
`modify_carriers` = '0'
`modify_labels` = '0'
`modify_statuses` = '0'
`modify_voicemail` = '0'
`modify_audiostore` = '0'
`modify_moh` = '0'
`modify_tts` = '0'
`preset_contact_search` = 'NOT_ACTIVE'
`modify_contacts` = '0'
`modify_same_user_level` = '1'
`admin_hide_lead_data` = '0'
`admin_hide_phone_data` = '0'
`agentcall_email` = '0'
`modify_email_accounts` = '0'
`failed_login_count` = 0
`last_login_date` = '2001-01-01 00:00:01'
`last_ip` = ''
`pass_hash` = ''
`alter_admin_interface_options` = '1'
`max_inbound_calls` = 0
`modify_custom_dialplans` = '0'
`wrapup_seconds_override` = -1
`modify_languages` = '0'
`selected_language` = 'default English'
`user_choose_language` = '0'
`ignore_group_on_search` = '0'
`api_list_restrict` = '0'
`api_allowed_functions` = ' ALL_FUNCTIONS '
`lead_filter_id` = 'NONE'
`admin_cf_show_hidden` = '0'
`agentcall_chat` = '0'
`user_hide_realtime` = '0'
`access_recordings` = '0'
`modify_colors` = '0'
`user_nickname` = ''
`user_new_lead_limit` = -1
`api_only_user` = '0'
`modify_auto_reports` = '0'
`modify_ip_lists` = '0'
`ignore_ip_list` = '0'
`ready_max_logout` = -1
`export_gdpr_leads` = '0'
`pause_code_approval` = '0'
`max_hopper_calls` = 0
`max_hopper_calls_hour` = 0
`mute_recordings` = 'DISABLED'
`hide_call_log_info` = 'DISABLED'
`next_dial_my_callbacks` = 'NOT_ACTIVE'
`user_admin_redirect_url` = ''
`max_inbound_filter_enabled` = '0'
`max_inbound_filter_statuses` = ''
`max_inbound_filter_ingroups` = ''
`max_inbound_filter_min_sec` = -1
`status_group_id` = ''

Version: 2.14b0.5
SVN Version: 3256
DB Schema Version: 1596
DB Schema Update Date: 2020-06-14 15:08:10

No changes to any scripts or tables, just stock, and has been fine for a year or more, this is a recent problem.

[jamiemurray]

`available_only_tally_threshold` = 'WAITING_AGENTS'
`available_only_tally_threshold_agents` = 1

Try disabling this, it could be causing your campaign to continue dialing even when all agents are occupied.

[richgen]

`available_only_tally_threshold` = 'WAITING_AGENTS'
`available_only_tally_threshold_agents` = 1

Try disabling this, it could be causing your campaign to continue dialing even when all agents are occupied.

Disabled. However, from what I can see, and I'm not a vici expert, the dialler isn't over-dialling, but for some reason a proportion of calls that asterisk delivers to agents are coded as DROP when an agent took the call. I can't see any pattern to it. When I actually compare Asterisk CDR to Agent outcomes I can see we're not dropping a single call and every call made has an outcome (based on linking vici list/call log CLI to agent system CLI) and the agent system is separate from vici and creates a record when their softphone is answered so a perfect out-of-scope recon and we never call the same CLI more than once in the same day.

From the Managers Manual : DROP - Agent Not Available - Call was dropped while customer was waiting for an agent, so why does vici think the call didn't reach the agent?

This is perplexing for sure. On vici's realtime for the campaign, I see calls dialling which is always <= number of agents, and I see the status change to 'QUEUE' then 'INCALL' and sometimes 'DEAD' but in all cases the agent's phone rings and they take the call. I honestly don't know where to look next

[jamiemurray]

There's so much involved in the delivery of a call to an agent, any number of things could be interrupting that process. It could also be that whatever is happening is actually being mis-reported and the call wasn't actually dropped.
Add me on skype if you want me to take a look over it - live:support_71847 - we can arrange a session where I can work with you to trace every single event leading up to the calls getting marked DROP