Call rec processing problem
Posted: Fri Jun 22, 2012 4:51 am
Hi forum!
I've encountered a problem on every dialer (x8) we have in production across two sites whereby the /var/spool/asterisk/monitorDONE/FTP folder has old random recordings, some months old:
This is growing slowly in size and in some cases up over 40GB in the FTP directory:
I've had to delete the wav files in the ORIG folder to make space but that is not a permanent fix.
I'm concerned that recordings older than 7 days are removed when they could not exist on the FTP server, resulting in lost recordings.
Cron:
I find that /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3 --debugX processes only current recordings:
I did notice a fourth script that is not included in the cron schedule, which does move recordings to FTP but also results in "File size mismatch" and random call recordings left behind on the dialer:
Results in some recordings being FTP'd and others in:
As a result, these dialers' disks are filling and the smallest disk needs manual cleaning and processing with that fourth script.
Q: Is it okay to just remove the wavs in the ORIG folder?
Q: Are all the dialer recording files supposed to be moved the the FTP by the end of the day?
Q: Can that fourth script be included in the cron schedule to move all recordings to the FTP regularly so none get left behind and alleviate this problem?
-----
VERSION: 2.6-365a BUILD: 120420-1620 and VERSION: 2.4-351a BUILD: 111223-0043 at different sites.
Asterisk 1.4
Multi-server: 1x DB, 1x Web, 4x Dialers
Sangoma/Digium hardware timing
G729
VoIP
OpenSuse - Vicibox Redux 3.1.5
No extra software after installation
I've encountered a problem on every dialer (x8) we have in production across two sites whereby the /var/spool/asterisk/monitorDONE/FTP folder has old random recordings, some months old:
- Code: Select all
dialler4:/var/spool/asterisk/monitorDONE/FTP # ls | wc -l
576544
dialler4:/var/spool/asterisk/monitorDONE/FTP # ls
...
-rw-r--r-- 1 root root 16K 2012-06-06 18:46 004_11428XXXXX_20120606-184330_FCU0767-all.mp3
-rw-r--r-- 1 root root 31K 2012-05-16 13:23 004_11428XXXXX_20120514-173206_FCU0630-all.mp3
-rw-r--r-- 1 root root 119K 2012-05-30 21:10 004_11428XXXXX_20120530-210535_FCU0313-all.mp3
-rw-r--r-- 1 root root 50K 2012-05-17 13:22 004_11428XXXXX_20120517-131741_FCU0032-all.mp3
-rw-r--r-- 1 root root 73K 2012-05-17 18:55 004_11428XXXXX_20120517-185058_FCU0821-all.mp3
-rw-r--r-- 1 root root 106K 2012-05-16 14:52 004_11428XXXXX_20120514-145009_FCU0767-all.mp3
...
This is growing slowly in size and in some cases up over 40GB in the FTP directory:
- Code: Select all
45G ./FTP
757M ./FTP2
4.0K ./GSM
4.0K ./GSW
496K ./MP3
4.0K ./OGG
1.6G ./ORIG
I've had to delete the wav files in the ORIG folder to make space but that is not a permanent fix.
I'm concerned that recordings older than 7 days are removed when they could not exist on the FTP server, resulting in lost recordings.
Cron:
- Code: Select all
### keepalive script for astguiclient processes
* * * * * /usr/share/astguiclient/ADMIN_keepalive_ALL.pl
### remove old vicidial logs and asterisk logs more than 2 days old
28 0 * * * /usr/bin/find /var/log/astguiclient -maxdepth 1 -type f -mtime +2 -print | xargs rm -f
29 0 * * * /usr/bin/find /var/log/asterisk -maxdepth 3 -type f -mtime +2 -print | xargs rm -f
30 0 * * * /usr/bin/find / -maxdepth 1 -name "screenlog.0*" -mtime +4 -print | xargs rm -f
### recording mixing/compressing/ftping scripts
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl --MIX
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_VDonly.pl
1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3
#Job to catch up recording processing backlog
#45 23 * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3
2,5,8,11,14,17,20,23,26,29,32,35,38,41,44,47,50,53,56,59 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3
### kill Hangup script for Asterisk updaters
* * * * * /usr/share/astguiclient/AST_manager_kill_hung_congested.pl
### updater for voicemail
* * * * * /usr/share/astguiclient/AST_vm_update.pl
### updater for conference validator
* * * * * /usr/share/astguiclient/AST_conf_update.pl
### flush queue DB table every hour for entries older than 1 hour
11 * * * * /usr/share/astguiclient/AST_flush_DBqueue.pl -q
### reset several temporary-info tables in the database
2 1 * * * /usr/share/astguiclient/AST_reset_mysql_vars.pl
### remove old recordings more than 7 days old
#24 0 * * * /usr/bin/find /var/spool/asterisk/monitorDONE -maxdepth 2 -type f -mtime +7 -print | xargs rm -f
0 8-20 * * * /usr/bin/find /var/spool/asterisk/monitorDONE/ORIG -maxdepth 2 -type f -mmin +120 -print | xargs rm -f
### Reboot nightly to manage asterisk issues and memory leaks - uncomment if issues arise
#0 6 * * * /sbin/reboot
### remove text to speech file more than 4 days old
#20 0 * * * /usr/bin/find /var/lib/asterisk/sounds/tts/ -maxdepth 2 -type f -mtime +4 -print | xargs rm -f
I find that /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3 --debugX processes only current recordings:
- Code: Select all
/var/spool/asterisk/monitorDONE/MP3/004_12536XXXXX_20120622-103427_FCU0914-all.mp3 35424
...
I did notice a fourth script that is not included in the cron schedule, which does move recordings to FTP but also results in "File size mismatch" and random call recordings left behind on the dialer:
- Code: Select all
/usr/share/astguiclient/AST_CRON_audio_4_ftp2.pl --MP3 --debugX --ftp-persistent
Results in some recordings being FTP'd and others in:
- Code: Select all
not transfering /var/spool/asterisk/monitorDONE/FTP/004_19378XXXXX_20120404-111306_FCU0010-all.mp3. File size mismatch 164448 !=
As a result, these dialers' disks are filling and the smallest disk needs manual cleaning and processing with that fourth script.
Q: Is it okay to just remove the wavs in the ORIG folder?
Q: Are all the dialer recording files supposed to be moved the the FTP by the end of the day?
Q: Can that fourth script be included in the cron schedule to move all recordings to the FTP regularly so none get left behind and alleviate this problem?
-----
VERSION: 2.6-365a BUILD: 120420-1620 and VERSION: 2.4-351a BUILD: 111223-0043 at different sites.
Asterisk 1.4
Multi-server: 1x DB, 1x Web, 4x Dialers
Sangoma/Digium hardware timing
G729
VoIP
OpenSuse - Vicibox Redux 3.1.5
No extra software after installation