vicidial.org

I have recently setup a vicidial server that is used for a 24 hour servicedesk for birth care. Agents are logged in on their laptops during the night with an on-hook phone that is a cell phone. Apart from this they also login as remote agent just for one ingroup for emergency calls with the same cell phone extension setup for that remote agent.

We've had some complaints about calls not being sent to agents during the night. I was wondering if there are any cronjobs we need to be aware of during the night that could disrupt production. I've already disabled the nightly reboot job and just noticed the backup job was still scheduled and disabled it too.

Also i've set the ingroup on-hook CID to RA_AGENT_PHONE since i've been told this is the best setting for Asterisk 13.

Any recommendations?
If i want to see which agent extensions were rung during the night which logfile can look at?

Do you have slow query logging enabled in MySQL on that system? If not, you should enable it. If so, then take a look there to see if there are any specific slow queries running at the same time each night, that should help identify what process may be causing issues.

Hi Matt, i have enabled slow query log now so i guess we will see what shows up. Last night i logged in as agent and this morning my screen said 'your session has been disabled'. So something funky is going on. I read a topic that recommended deleting .TMM files but i couldn't find them.

Our sessions were disabled again last night
Unfortunately i did a reboot last night which seems to have disabled the slow query log again because i only see log entries from before the reboot.

How about you post your crontab here for us to look at (crontab -l)

Here it is!

Code: Select all

# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.FiwvnL installed on Wed Apr 24 23:07:33 2019)
# (Cronie version 4.2)
### keepalive script for astguiclient processes
* * * * * /usr/share/astguiclient/ADMIN_keepalive_ALL.pl

### Compress astguiclient log files and remove old ones
25 2 * * * /usr/bin/find /var/log/astguiclient -maxdepth 1 -type f -mtime +1 -print | grep -v \.xz | xargs xz -9 >/dev/null 2>&1
28 0 * * * /usr/bin/find /var/log/astguiclient -maxdepth 1 -type f -mtime +30 -print | xargs rm -f

### fix the vicidial_agent_log once every hour and the full day run at night
33 * * * * /usr/share/astguiclient/AST_cleanup_agent_log.pl
50 0 * * * /usr/share/astguiclient/AST_cleanup_agent_log.pl --last-24hours
## uncomment below if using QueueMetrics
#*/5 * * * * /usr/share/astguiclient/AST_cleanup_agent_log.pl --only-qm-live-call-check

### updater for VICIDIAL hopper
* * * * * /usr/share/astguiclient/AST_VDhopper.pl -q

### adjust the GMT offset for the leads in the vicidial_list table
1 1,7 * * * /usr/share/astguiclient/ADMIN_adjust_GMTnow_on_leads.pl --debug --list-settings

### optimize the database tables within the asterisk database
3 1 * * * /usr/share/astguiclient/AST_DB_optimize.pl

### VICIDIAL agent time log weekly and daily summary report generation
2 0 * * 0 /usr/share/astguiclient/AST_agent_week.pl
22 0 * * * /usr/share/astguiclient/AST_agent_day.pl

### VICIDIAL campaign export scripts (OPTIONAL)
#32 0 * * * /usr/share/astguiclient/AST_VDsales_export.pl
#42 0 * * * /usr/share/astguiclient/AST_sourceID_summary_export.pl

### inventory report optional
#1 7 * * * /usr/share/astguiclient/AST_dialer_inventory_snapshot.pl -q --override-24hours

### roll logs monthly on high-volume dialing systems
#30 1 1 * * /usr/share/astguiclient/ADMIN_archive_log_tables.pl --months=6

### roll call_log and vicidial_log_extended daily on very high-volume dialing systems
#20 1 * * * /usr/share/astguiclient/ADMIN_archive_log_tables.pl --daily

## uncomment below if using Vtiger
#1 1 * * * /usr/share/astguiclient/Vtiger_optimize_all_tables.pl --quiet

# cleanup of the scheduled callback records
25 0 * * * /usr/share/astguiclient/AST_DB_dead_cb_purge.pl --purge-non-cb --quiet
# removal of duplicate scheduled callback records
#35 0 * * * /usr/share/astguiclient/AST_DB_dead_cb_purge.pl --purge-non-cb --remove-dup-cb --quiet

### inbound email parser should only be active on a single server
#* * * * * /usr/share/astguiclient/AST_inbound_email_parser.pl

### flush queue DB table every hour for entries older than 1 hour
11 * * * * /usr/share/astguiclient/AST_flush_DBqueue.pl -q

### remove and rotate old asterisk logs
29 0 * * * /usr/bin/find /var/log/asterisk -maxdepth 3 -type f -mtime +30 -print | xargs rm -f
30 0 * * * /usr/bin/find / -maxdepth 1 -name "screenlog.0*" -mtime +7 -print | xargs rm -f
31 0 * * * /usr/bin/find /tmp -maxdepth 1 -type f -mtime +7 -print | xargs rm -f
32 0 * * * /usr/bin/find /var/log/asterisk -maxdepth 1 -type f -mtime +1 -print | grep -v \.xz | xargs xz >/dev/null 2>&1

### recording mixing/compressing/ftping scripts
#0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl --MIX
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_VDonly.pl
1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3 --HTTPS
#2,5,8,11,14,17,20,23,26,29,32,35,38,41,44,47,50,53,56,59 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3 --run-check
#0 1 * * * /usr/share/astguiclient/AST_CRON_audio_4_ftp2.pl --ftp-server=server.ip --ftp-login=user --ftp-pass=pass --ftp-directory=/ --ftp-persistent --ftp-validate --transfer-limit=100000 --list-limit=100000

### remove old recordings more than 180 days old, and delete originals after 1 day
24 0 * * * /usr/bin/find /var/spool/asterisk/monitorDONE -maxdepth 2 -type f -mtime +180 -delete
24 1 * * * /usr/bin/find /var/spool/asterisk/monitorDONE/ORIG -maxdepth 2 -type f -mtime +1 -delete

### kill Hangup script for Asterisk updaters
* * * * * /usr/share/astguiclient/AST_manager_kill_hung_congested.pl

### updater for voicemail
* * * * * /usr/share/astguiclient/AST_vm_update.pl

### updater for conference validator
* * * * * /usr/share/astguiclient/AST_conf_update.pl

### reset several temporary-info tables in the database
2 1 * * * /usr/share/astguiclient/AST_reset_mysql_vars.pl

### Reboot nightly to manage asterisk issues and memory leaks - uncomment if issues arise
#0 3 * * 0 /sbin/reboot

### remove text to speech file more than 4 days old
#20 0 * * * /usr/bin/find /var/lib/asterisk/sounds/tts/ -maxdepth 2 -type f -mtime +4 -print | xargs rm -f

### Update agent records with the IP of the phone they are logging in on
#*/5 * * * * /usr/share/astguiclient/AST_phone_update.pl --agent-lookup

### Delete voicemail that is older then 60 days
#0 2 * * * /usr/local/bin/vmspool_manager.pl --active --age=60

### ViciBox integrated firewall, by default just load the VoIP Black list and reload it every 4 hours
### You can lock everyone out of your server if you set this wrong, so understand what you are doing!!!
#@reboot /usr/local/bin/VB-firewall.pl --voipbl --noblack --quiet
#0 */6 * * * /usr/local/bin/VB-firewall.pl --voipbl --noblack --flush --quiet

### Process dynamic and white lists for VB Firewall, comment out the black/block list entries above this
### Enable iptables white list rules with : touch /etc/sysconfig/scripts/SuSEfirewall2-viciwhite
### Enable iptables dynamic list rules with : touch /etc/sysconfig/scripts/SuSEfirewall2-vicidynamic
### Enable iptables full control with : touch /etc/sysconfig/scripts/SuSEfirewall2-vicifull
* * * * * /usr/local/bin/VB-firewall.pl --dynamic --white --flush --quiet

### Renew SSL certificate monthly

0 5 1 * * /usr/sbin/certbot-open.sh
1 5 1 * * /usr/bin/certbot certonly > /var/log/certbot.log
3 5 1 * * /usr/sbin/certbot-close.sh
5 5 1 * * /usr/sbin/apache2ctl -k graceful >/dev/null 2>&1

### Viciboard rapportage
*/5 * * * * /usr/bin/curl --silent https://localhost/viciboard/rolling_agent_stats.php > /dev/null 2>&1

### nightly backup
#0 2 * * * /usr/share/astguiclient/ADMIN_backup.pl

Comment out (#) these two and retest:

Code: Select all

### adjust the GMT offset for the leads in the vicidial_list table
1 1,7 * * * /usr/share/astguiclient/ADMIN_adjust_GMTnow_on_leads.pl --debug --list-settings
### reset several temporary-info tables in the database
2 1 * * * /usr/share/astguiclient/AST_reset_mysql_vars.pl  <<most likely because of this


dspaan wrote:Here it is!

Code: Select all

# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.FiwvnL installed on Wed Apr 24 23:07:33 2019)
# (Cronie version 4.2)
### keepalive script for astguiclient processes
* * * * * /usr/share/astguiclient/ADMIN_keepalive_ALL.pl

### Compress astguiclient log files and remove old ones
25 2 * * * /usr/bin/find /var/log/astguiclient -maxdepth 1 -type f -mtime +1 -print | grep -v \.xz | xargs xz -9 >/dev/null 2>&1
28 0 * * * /usr/bin/find /var/log/astguiclient -maxdepth 1 -type f -mtime +30 -print | xargs rm -f

### fix the vicidial_agent_log once every hour and the full day run at night
33 * * * * /usr/share/astguiclient/AST_cleanup_agent_log.pl
50 0 * * * /usr/share/astguiclient/AST_cleanup_agent_log.pl --last-24hours
## uncomment below if using QueueMetrics
#*/5 * * * * /usr/share/astguiclient/AST_cleanup_agent_log.pl --only-qm-live-call-check

### updater for VICIDIAL hopper
* * * * * /usr/share/astguiclient/AST_VDhopper.pl -q

### adjust the GMT offset for the leads in the vicidial_list table
1 1,7 * * * /usr/share/astguiclient/ADMIN_adjust_GMTnow_on_leads.pl --debug --list-settings

### optimize the database tables within the asterisk database
3 1 * * * /usr/share/astguiclient/AST_DB_optimize.pl

### VICIDIAL agent time log weekly and daily summary report generation
2 0 * * 0 /usr/share/astguiclient/AST_agent_week.pl
22 0 * * * /usr/share/astguiclient/AST_agent_day.pl

### VICIDIAL campaign export scripts (OPTIONAL)
#32 0 * * * /usr/share/astguiclient/AST_VDsales_export.pl
#42 0 * * * /usr/share/astguiclient/AST_sourceID_summary_export.pl

### inventory report optional
#1 7 * * * /usr/share/astguiclient/AST_dialer_inventory_snapshot.pl -q --override-24hours

### roll logs monthly on high-volume dialing systems
#30 1 1 * * /usr/share/astguiclient/ADMIN_archive_log_tables.pl --months=6

### roll call_log and vicidial_log_extended daily on very high-volume dialing systems
#20 1 * * * /usr/share/astguiclient/ADMIN_archive_log_tables.pl --daily

## uncomment below if using Vtiger
#1 1 * * * /usr/share/astguiclient/Vtiger_optimize_all_tables.pl --quiet

# cleanup of the scheduled callback records
25 0 * * * /usr/share/astguiclient/AST_DB_dead_cb_purge.pl --purge-non-cb --quiet
# removal of duplicate scheduled callback records
#35 0 * * * /usr/share/astguiclient/AST_DB_dead_cb_purge.pl --purge-non-cb --remove-dup-cb --quiet

### inbound email parser should only be active on a single server
#* * * * * /usr/share/astguiclient/AST_inbound_email_parser.pl

### flush queue DB table every hour for entries older than 1 hour
11 * * * * /usr/share/astguiclient/AST_flush_DBqueue.pl -q

### remove and rotate old asterisk logs
29 0 * * * /usr/bin/find /var/log/asterisk -maxdepth 3 -type f -mtime +30 -print | xargs rm -f
30 0 * * * /usr/bin/find / -maxdepth 1 -name "screenlog.0*" -mtime +7 -print | xargs rm -f
31 0 * * * /usr/bin/find /tmp -maxdepth 1 -type f -mtime +7 -print | xargs rm -f
32 0 * * * /usr/bin/find /var/log/asterisk -maxdepth 1 -type f -mtime +1 -print | grep -v \.xz | xargs xz >/dev/null 2>&1

### recording mixing/compressing/ftping scripts
#0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl --MIX
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_VDonly.pl
1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3 --HTTPS
#2,5,8,11,14,17,20,23,26,29,32,35,38,41,44,47,50,53,56,59 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3 --run-check
#0 1 * * * /usr/share/astguiclient/AST_CRON_audio_4_ftp2.pl --ftp-server=server.ip --ftp-login=user --ftp-pass=pass --ftp-directory=/ --ftp-persistent --ftp-validate --transfer-limit=100000 --list-limit=100000

### remove old recordings more than 180 days old, and delete originals after 1 day
24 0 * * * /usr/bin/find /var/spool/asterisk/monitorDONE -maxdepth 2 -type f -mtime +180 -delete
24 1 * * * /usr/bin/find /var/spool/asterisk/monitorDONE/ORIG -maxdepth 2 -type f -mtime +1 -delete

### kill Hangup script for Asterisk updaters
* * * * * /usr/share/astguiclient/AST_manager_kill_hung_congested.pl

### updater for voicemail
* * * * * /usr/share/astguiclient/AST_vm_update.pl

### updater for conference validator
* * * * * /usr/share/astguiclient/AST_conf_update.pl

### reset several temporary-info tables in the database
2 1 * * * /usr/share/astguiclient/AST_reset_mysql_vars.pl

### Reboot nightly to manage asterisk issues and memory leaks - uncomment if issues arise
#0 3 * * 0 /sbin/reboot

### remove text to speech file more than 4 days old
#20 0 * * * /usr/bin/find /var/lib/asterisk/sounds/tts/ -maxdepth 2 -type f -mtime +4 -print | xargs rm -f

### Update agent records with the IP of the phone they are logging in on
#*/5 * * * * /usr/share/astguiclient/AST_phone_update.pl --agent-lookup

### Delete voicemail that is older then 60 days
#0 2 * * * /usr/local/bin/vmspool_manager.pl --active --age=60

### ViciBox integrated firewall, by default just load the VoIP Black list and reload it every 4 hours
### You can lock everyone out of your server if you set this wrong, so understand what you are doing!!!
#@reboot /usr/local/bin/VB-firewall.pl --voipbl --noblack --quiet
#0 */6 * * * /usr/local/bin/VB-firewall.pl --voipbl --noblack --flush --quiet

### Process dynamic and white lists for VB Firewall, comment out the black/block list entries above this
### Enable iptables white list rules with : touch /etc/sysconfig/scripts/SuSEfirewall2-viciwhite
### Enable iptables dynamic list rules with : touch /etc/sysconfig/scripts/SuSEfirewall2-vicidynamic
### Enable iptables full control with : touch /etc/sysconfig/scripts/SuSEfirewall2-vicifull
* * * * * /usr/local/bin/VB-firewall.pl --dynamic --white --flush --quiet

### Renew SSL certificate monthly

0 5 1 * * /usr/sbin/certbot-open.sh
1 5 1 * * /usr/bin/certbot certonly > /var/log/certbot.log
3 5 1 * * /usr/sbin/certbot-close.sh
5 5 1 * * /usr/sbin/apache2ctl -k graceful >/dev/null 2>&1

### Viciboard rapportage
*/5 * * * * /usr/bin/curl --silent https://localhost/viciboard/rolling_agent_stats.php > /dev/null 2>&1

### nightly backup
#0 2 * * * /usr/share/astguiclient/ADMIN_backup.pl

Yep, "/usr/share/astguiclient/AST_reset_mysql_vars.pl" would most likely cause that.

Thanks guys, i can confirm disabling these jobs has resolved the problem.

Code: Select all

### adjust the GMT offset for the leads in the vicidial_list table
1 1,7 * * * /usr/share/astguiclient/ADMIN_adjust_GMTnow_on_leads.pl --debug --list-settings
### reset several temporary-info tables in the database
2 1 * * * /usr/share/astguiclient/AST_reset_mysql_vars.pl  <<most likely because of this

This weekend i remained succesfully logged in for at least 40 hours.

I suppose the GMT offset job isn't important when you work in a single timezone?
What happens if you don't reset the mysql vars for a longer period? What would be a recommended period to reset these if this can't be done nightly?

You'd really have to customize that script if you wanted to run it while agents are logged in. Some of the queries in it will effectively log out all agents. You could change it to be run at reboot time, although you haven't mentioned if you are even doing scheduled reboots.

In typical 24/7 setups, we recommend a dedicated web/db server and two dialers. We recommend having all agents log in with Phone Aliases, and scheduling reboots of one of the two dialers every other weekend, setting the Active Agent Server = N about 12 hours before the scheduled reboot so it has no effect on production. We have set up several clients like this and it works out very well and allows for the asterisk servers to be rebooted regularly.

Hi Matt, normally we do scheduled reboots but in this case we have turned that off as you can see in the cronjob. Are scheduled reboots are neccessity? This server only has about 4-5 logged in agents. A cluster like that sounds like a good idea but a bit of overkill in this case.

We do have clients that are very low call volume that run as you describe with no reboots, but at some point the system will need to be rebooted, so you should plan on scheduled reboots at least once a year so they don't happen in an uncontrolled manner.

Does your system ever have "idle" time? It would be possible to set the system to "wait for idle" after being online for over 24 hours and reboot when there are no client calls. Thus the agents would be booted, but no clients would be inconvenienced unless they called during the reboot moment of the dialer.

Yes there often is idle time. However the agents are working on-hook andere not always behind their screen or in the vicidial tab at all times. So how would such an idle time reboot work? Would be nice to send an SMS to the logged in users too.

May be easier to generate an audio alert on the screen, but texting is possible as well, of course. Email would probably be easiest. Perhaps even generate a CALL to each of them on their 2nd lines to give them a heads-up.

In hindsight: A call with a callerID of "REBOOTING SYSTEM" and audio that says "REBOOTING NOW", then reboot 15 seconds later. Block inbound/outbound calls before generating the warning and you should be good to go.

Thanks for the tip, will try that!

Hi William, do you have an example of how to configure the cronjob so it only reboots when there are no active calls?

Nope. You can use "uptime" for load and

Code: Select all

asterisk -rx "core show channels"

Then filter out the agent sessions (with grep -v) and count what's left with wc -l