vicidial.org

by **spacejanitor** » Thu Jul 21, 2011 1:08 pm

Asterisk 1.4 (installed from ViciBox Redux V 3.1.9 ISO)
VERSION: 2.4-304a
BUILD: 110301-1206
Single Server (HP XW8400)
No Digium/Sangoma Hardware
No Extra Software After Installation

Processors
4 (2 physical)
Model
Intel Xeon 5160
CPU Speed
3.0 GHz
Cache Size
4M Cache
----------------------------------------------------------------

Hi all,

This weekend I plan on doing a little infrastructure upgrading on our dialer. Namely, setting up a separate backup server as well as setting call recordings to be sent to another server.

I frequently make use of the call recording feature of ViciDial for QA purposes. For example, I'll search for a lead in the Lists that matches a particular phone number and then I will check the lead page to find the call recording link and listen to it. So I need to make sure that whichever recording backup method I use, it won't destroy these links. Hoes does ViciDial update the links for recordings? Does it simply take the path specified in etc/astguiclient.conf ?

So my next question is, what's the best method for offloading recordings to a separate server? If I specify the FTP details in astguiclient.conf of the backup server, will ViciDial "take care of the rest"? Meaning, it will automatically move all processed recordings to the FTP and delete them from the local directory?

I'm just wondering whether there's anything I should do like write a cron script to wipe all recordings locally every few hours, or do I just update the information in astguiclient.conf?

Thanks!

by **okli** » Fri Jul 22, 2011 10:49 am

Hoes does ViciDial update the links for recordings?

/usr/share/astguiclient/ADMIN_update_archive_url.pl :

Code: Select all: # ADMIN_update_archive_url.pl - updates the url for the archive server # in the recording_log table

Use --help option to find out command line parameters or just open the file and look at it, it's quite well commented as all files in Vicidial.
Amend astguiclient.conf file too with the new location.

So my next question is, what's the best method for offloading recordings to a separate server?

/usr/share/astguiclient/AST_CRON_audio_3_ftp.pl :

Code: Select all: #!/usr/bin/perl # # AST_CRON_audio_3_ftp.pl # # This is a STEP-3 program in the audio archival process # # runs every 3 minutes and copies the recording files to an FTP server # # put an entry into the cron of of your asterisk machine to run this script # every 3 minutes or however often you desire # # ### recording mixing/compressing/ftping scripts ##0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl # 0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_VDonly.pl # 1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --GSM # 2,5,8,11,14,17,20,23,26,29,32,35,38,41,44,47,50,53,56,59 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --GSM # # FLAGS FOR COMPRESSION FILES TO TRANSFER # --GSM = GSM 6.10 files # --MP3 = MPEG Layer3 files # --OGG = OGG Vorbis files # --WAV = WAV files # --GSW = GSM 6.10 codec with RIFF headers (.wav extension)

Meaning, it will automatically move all processed recordings to the FTP

yes

and delete them from the local directory?

No. Will move them to /var/spool/asterisk/monitorDONE/FTP. You might empty it via cron job however often you feel safe and suitable.

For our purposes, because disk space became an issue, I've modified the FTP script to delete the files if were successfully put on the FTP by comparing file sizes. Next we use slightly modified AutoIndex PHP to search and browse recordings, voicemails and reports, and play online the recordings with flash based WAV and MP3 players. If anyone is interested I can post more details. Some are here:
http://www.vicidial.org/VICIDIALforum/v ... =autoindex

by **callcrazy** » Fri Jul 22, 2011 12:27 pm

Our servers are all in the same network so instead of using ftp script to copy I've created the ramfs and mounted it to /var/spool/asterisk/monitor

Then using the script AST_CRON_audio_1_move_mix.pl it will copy the file from

/var/spool/asterisk/monitor (the ram drive)

to

/var/spool/asterisk/monitorDONE

The monitorDONE folder has been mounted to a remote filesystem running samba.

Code: Select all: mount -t cifs -o -username=user //remoteip/share /var/spool/asterisk/monitorDONE

I then use custom scripts on the remote server to perform the conversions from gsm to mp3 to keep the load on the pbx in check.

by **dj mondo** » Fri Jul 22, 2011 3:49 pm

i am also about to do the same thing tomorrow. I also downloaded and printed out this document:

http://www.poundteam.com/downloads/Vici ... v1%201.pdf

very handy information, now at the end of the pdf, there is a section about backing up to an archive server. If i followed that to the letter, would everything go well? I am very new to linux, spent the last 12 years working on nothing but windows.

The first backup that i am doing is going to be a one time backup and on monday i will be ordering 2 - 1 terabyte hard drives for raid. Also i will be using windows server 2003 as the backup server, along with filezilla (any recommendations on that??) Then once a month i will go the backup server and burn all the recordings for that month to dvd and get them off the server to free up space.

Please any help or advice would be greatly appreciated.

Thank you
Armando

by **dj mondo** » Sat Jul 23, 2011 10:44 am

@okli, yes please i would like more details. Seeing as i don't know linux very well yet, and and all advice would be greatly appreciated

thank you
armando

by **dj mondo** » Mon Jul 25, 2011 12:59 pm

Did anyone get a chance to look at the document that i posted. towards the end its mentions something about backup scripts. Is that all i would have to do, or is there more to it??

Thank you
Armando

by **Trying** » Mon Jul 25, 2011 1:32 pm

Those are the same steps we used for our system and there are no issues. Don't stress, just do it :lol:

by **dj mondo** » Mon Jul 25, 2011 1:47 pm

The only reason why i ask is i am wanting to know, do i have to add the server in the admin interface ( Admin>servers>add server), or that won't matter?

by **Trying** » Mon Jul 25, 2011 1:48 pm

No you don't have to add it there.

by **dj mondo** » Mon Jul 25, 2011 2:50 pm

so in order to backup the database also to the archive server, i have to install the database along with the archive setup? I would assume yes, but i would rather ask than assume.

Thank you
Armando

by **spacejanitor** » Mon Jul 25, 2011 3:16 pm

okli wrote:
Hoes does ViciDial update the links for recordings?

/usr/share/astguiclient/ADMIN_update_archive_url.pl :
Code: Select all
# ADMIN_update_archive_url.pl - updates the url for the archive server # in the recording_log table
Use --help option to find out command line parameters or just open the file and look at it, it's quite well commented as all files in Vicidial.
Amend astguiclient.conf file too with the new location.

So my next question is, what's the best method for offloading recordings to a separate server?

/usr/share/astguiclient/AST_CRON_audio_3_ftp.pl :
Code: Select all
#!/usr/bin/perl # # AST_CRON_audio_3_ftp.pl # # This is a STEP-3 program in the audio archival process # # runs every 3 minutes and copies the recording files to an FTP server # # put an entry into the cron of of your asterisk machine to run this script # every 3 minutes or however often you desire # # ### recording mixing/compressing/ftping scripts ##0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl # 0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_VDonly.pl # 1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --GSM # 2,5,8,11,14,17,20,23,26,29,32,35,38,41,44,47,50,53,56,59 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --GSM # # FLAGS FOR COMPRESSION FILES TO TRANSFER # --GSM = GSM 6.10 files # --MP3 = MPEG Layer3 files # --OGG = OGG Vorbis files # --WAV = WAV files # --GSW = GSM 6.10 codec with RIFF headers (.wav extension)

Meaning, it will automatically move all processed recordings to the FTP
yes

and delete them from the local directory?
No. Will move them to /var/spool/asterisk/monitorDONE/FTP. You might empty it via cron job however often you feel safe and suitable.

For our purposes, because disk space became an issue, I've modified the FTP script to delete the files if were successfully put on the FTP by comparing file sizes. Next we use slightly modified AutoIndex PHP to search and browse recordings, voicemails and reports, and play online the recordings with flash based WAV and MP3 players. If anyone is interested I can post more details. Some are here:
http://www.vicidial.org/VICIDIALforum/v ... =autoindex

Thanks for the help so far guys. I have created an FTP on another machine, verified it's working.

I've updated astguiclient.conf with the FTP credentials.

I've updated Admin -> Servers with the "Recordings Web Link" (set to ALT_IP).

Now I just need to get these recordings MOVING...

So how can I get AST_CRON_audio_3_ftp.pl running every 3 minutes?

by **spacejanitor** » Mon Jul 25, 2011 3:22 pm

The machine is already move/mixing and compressing my audio, so it seems only Step3 is not running...

Also, must I modify the variables in AST_CRON_audio_3_ftp.pl also? They appear as the defaults: $VARFTP_host = '10.0.0.4' etc...

by **Trying** » Mon Jul 25, 2011 4:05 pm

@ dj mondo...... I am not sure I understand your question. Are you asking if you must also install the database on the archive server in order to do backups? If that is your question then the answer is no. You only install the database on your db server.

by **williamconley** » Mon Jul 25, 2011 9:16 pm

spacejanitor wrote:The machine is already move/mixing and compressing my audio, so it seems only Step3 is not running...

Also, must I modify the variables in AST_CRON_audio_3_ftp.pl also? They appear as the defaults: $VARFTP_host = '10.0.0.4' etc...

No, those variables are "default examples" for those wishing to modify the script. The script itself at runtime will acquires these from astguiclient.conf and ignore the defaults.

But you DO have to verify that the ftp credentials will work from the server sending the files to the server receiving the files in the folder specified. (actually transfer a few files). Until that works, your script cannot succeed.

by **spacejanitor** » Tue Jul 26, 2011 8:22 am

williamconley wrote:
spacejanitor wrote:The machine is already move/mixing and compressing my audio, so it seems only Step3 is not running...

Also, must I modify the variables in AST_CRON_audio_3_ftp.pl also? They appear as the defaults: $VARFTP_host = '10.0.0.4' etc...
No, those variables are "default examples" for those wishing to modify the script. The script itself at runtime will acquires these from astguiclient.conf and ignore the defaults.

But you DO have to verify that the ftp credentials will work from the server sending the files to the server receiving the files in the folder specified. (actually transfer a few files). Until that works, your script cannot succeed.

Ok cool, I verified FTP connects successfully and transferred some files.
So how do I start executing the AST_CRON_audio_3_ftp.pl at regular intervals now? I would only need to transfer the MP3s (ViciBox appears to be processing to MP3 by default), and as I understand it I would need to add a --MP3 flag to the script... but still, where can I make sure that it's being called?

Thanks!

by **dj mondo** » Tue Jul 26, 2011 9:12 am

@trying, i would like the archive server to do 2 kind of backups, one for recordings and one for the database so if the main server goes down, i still have backups of the database which is the bread and butter, and i am surely not going through that whole process of importing every list one by one again. so installing the database portion of the install along with the archive setup, would allow me to backup the database?

by **williamconley** » Tue Jul 26, 2011 9:45 am

spacejanitor wrote:Ok cool, I verified FTP connects successfully and transferred some files.
So how do I start executing the AST_CRON_audio_3_ftp.pl at regular intervals now? I would only need to transfer the MP3s (ViciBox appears to be processing to MP3 by default), and as I understand it I would need to add a --MP3 flag to the script... but still, where can I make sure that it's being called?

Thanks!

standard crontab -e entry is already there, just uncomment it.

by **spacejanitor** » Tue Jul 26, 2011 12:37 pm

My etc/crontab has only 1 line and it doesn't seem to be vicidial related... is there another location where I should be looking?

by **okli** » Tue Jul 26, 2011 12:42 pm

That's wrong place. You were told about "crontab -e", not /etc/crontab

Type in command prompt:

Code: Select all: crontab -e

This should open the crontab entries in the default editor.

by **spacejanitor** » Wed Jul 27, 2011 12:58 pm

okli wrote:That's wrong place. You were told about "crontab -e", not /etc/crontab
Type in command prompt:
Code: Select all
crontab -e
This should open the crontab entries in the default editor.

Oh ok... I see it now

However there's a warning: DO NOT EDIT THIS FILE - edit the master and reinstall.

Any suggestions?

by **williamconley** » Wed Jul 27, 2011 3:33 pm

it's not like the /etc/asterisk/sip-vicidial.conf that actually DOES get regenerated quite often. i've never regenerated that file on a server (and I have a LOT of servers).

i always edit that file. it never gets regenerated unless you zypper (AFAIK) ... and we avoid zypper whenever possible if the system is already stable. a bad zypper up can require a reinstall. Kumba has the present system quite stable, I'm not into risk. LOL

by **dj mondo** » Fri Sep 23, 2011 10:10 am

k i got the ftp to work and i still can't get the ADMIN_backup.pl to work, any advice. If any information is needed please let me know and i post whatever is needed. Now if i wanted so that the recordings only move over at night after 11, what changed would i have to make so that it does the move after 11.

by **Arative** » Fri Sep 23, 2011 2:12 pm

If you look at the crontab, you'll see the audio_3_ftp.pl, you'll see the asterisks in front of the command. First one in minutes, second one is hour, third one day, fourth one is month and the fifth one is year.

I have mine set to ftp the recordings the archive server at 8:30 every night. It looks like this
30 20 * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3

by **dj mondo** » Fri Sep 30, 2011 11:16 am

k now i have a crazy question about that, i have about 4 months of recordings that i need to backup and move off the dialer. When i first set it up, it worked fine but there is so much that it froze the server i was backing up to, with those number, is there a way to have it do backup for like 2 hours and then stop??

by **joako** » Fri Sep 30, 2011 7:20 pm

dj mondo wrote:k now i have a crazy question about that, i have about 4 months of recordings that i need to backup and move off the dialer. When i first set it up, it worked fine but there is so much that it froze the server i was backing up to, with those number, is there a way to have it do backup for like 2 hours and then stop??

Have you tried rsync? You can run it for 2hrs a day and then stop it when you desire, start over again, etc.

by **dj mondo** » Fri Sep 30, 2011 11:32 pm

well i finally figured out what was going on with the servers i work with.

For you vets out there, the server that i "WAS" using, has ISA slots, and a ISA socket mother board, so i thought of a way to get a new motherboard, processor and ram.

The archive server that i tried to setup kept failing, and it was cause of the motherboard (which it is), its one of the first p4 when they first came out and i mean one of the first!!

So with their old school mentality, if it ain't broke why fix it, which is how they think, i am going to say that the server m.b. failed.

Then i am going to say, lucky i didn't delete everything off the dialer, but at least made a backup of the database, which i have manually and see if i can get a some new hardware that can handle all that data, and get a new board the supports something other than raid 1, 0 or both.

Once i get new hardware, i can be able to handle 4 months worth of recordings. Then be able to handle more. The database is not as large as the recordings.

Seriously, the most advanced hardware that i have in that office is the dialer and my laptop!!

by **actiondave** » Wed Nov 12, 2014 8:22 pm

I had someone set up the recordings and backups to automatically be ftp to my archive server. The recordings are moving perfectly but the backups stopped.

are the settings in astguiclient.conf?

this is what mine looks like.

# /etc/astguiclient.conf - configuration elements for the astguiclient/vicidial package
# this is the astguiclient configuration file
# all comments will be lost if you run install.pl again

# Paths used by astGUIclient
PATHhome => /usr/share/astguiclient
PATHlogs => /var/log/astguiclient
PATHagi => /var/lib/asterisk/agi-bin
PATHweb => /srv/www/htdocs
PATHsounds => /var/lib/asterisk/sounds
PATHmonitor => /var/spool/asterisk/monitor
PATHDONEmonitor => /var/spool/asterisk/monitorDONE

# The IP address of this machine
VARserver_ip => 192.168.1.xxx

# Database connection information
VARDB_server => localhost
VARDB_database => asterisk
VARDB_user => cron
VARDB_pass => xxxx
VARDB_custom_user => custom
VARDB_custom_pass => custom1234
VARDB_port => 3306

# Alpha-Numeric list of the astGUIclient processes to be kept running
# (value should be listing of characters with no spaces: 1234568)
# X - NO KEEPALIVE PROCESSES (use only if you want none to be keepalive)
# 1 - AST_update
# 2 - AST_send_listen
# 3 - AST_VDauto_dial
# 4 - AST_VDremote_agents
# 5 - AST_VDadapt (If multi-server system, this must only be on one server)
# 6 - FastAGI_log
# 7 - AST_VDauto_dial_FILL (only for multi-server, this must only be on one server)
# 8 - ip_relay (used for blind agent monitoring)
# 9 - Timeclock auto logout
# E - Email processor, (If multi-server system, this must only be on one server)
VARactive_keepalives => 123456789

# Asterisk version VICIDIAL is installed for
VARasterisk_version => 1.4

# FTP recording archive connection information
VARFTP_host => 192.168.1.xxx
VARFTP_user => username
VARFTP_pass => password
VARFTP_port => 21
VARFTP_dir => vicidial_recordings
VARHTTP_path => ftp://192.168.1.xxx/vicidial_recordings

# REPORT server connection information
#VARREPORT_host => 10.0.0.4
#VARREPORT_user => cron
#VARREPORT_pass => test
#VARREPORT_port => 21
#VARREPORT_dir => REPORTS

# backup ftp
VARREPORT_host => 192.168.1.xxx
VARREPORT_user => username
VARREPORT_pass => password
VARREPORT_port => 21
VARREPORT_dir => vicidial_backups

# Settings for FastAGI logging server
VARfastagi_log_min_servers => 3
VARfastagi_log_max_servers => 16
VARfastagi_log_min_spare_servers => 2
VARfastagi_log_max_spare_servers => 8
VARfastagi_log_max_requests => 1000
VARfastagi_log_checkfordead => 30
VARfastagi_log_checkforwait => 60

Does this: VARHTTP_path => ftp://192.168.1.xxx/vicidial_backups need to be the last line of # backup ftp?

by **okli** » Fri Nov 14, 2014 9:14 am

actiondave wrote:...
Does this: VARHTTP_path => ftp://192.168.1.xxx/vicidial_backups need to be the last line of # backup ftp?

I think not.
You could take a look at the server log files, and/or run the archive script manually with --debugX option to see what's happening.

by **actiondave** » Wed Jan 07, 2015 1:18 pm

can someone point me to which log would show any error of my backups not automatically moving to my archive server?

by **okli** » Thu Jan 08, 2015 9:50 am

You could run the archive script manually with --debugX option to see what's happening.

Or you can check mail for root, if there were any errors logged when the cron job run.

vicidial.org

Best method to backup recordings (off of production server)

Best method to backup recordings (off of production server)

Re: Best method to backup recordings (off of production serv

Re: Best method to backup recordings (off of production serv

Re: Best method to backup recordings (off of production serv

Re: Best method to backup recordings (off of production serv

Who is online