Page 1 of 1

[SOLVED] Differences in Perl Scripts

PostPosted: Fri May 15, 2020 11:36 am
by AlSam
I am interested in backing up recordings and am not sure which script I should use. The two that may help are AST_CRON_audio_3_ftp_FTPSSL.pl and AST_CRON_audio_4_ftp2_FTPSSL.pl but I don't know the difference in their functionality. Can anyone tell me what they are? Also, as I was reading the first few lines of each script, I found that one is a STEP-3 and the other a STEP-4 program. Must there be a STEP-3 program being run before running a STEP-4 program?

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 7:14 am
by AlSam
..or maybe is there documentation available on them (besides the comments at the beginning of them)?

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 9:54 am
by mflorell
Yes, a level-4 CRON-audio script can only run after the level-3 script has run, this is because of where the recordings are located. We created the level-4 scripts so that clients wanting to send recordings to 2 separate archive servers could do so in an automated way.

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 10:07 am
by AlSam
Thanks for the response. Do these scripts move recordings or just copy them?

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 10:40 am
by williamconley
AlSam wrote:Thanks for the response. Do these scripts move recordings or just copy them?


Vicidial Recording Management Clarified

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 12:41 pm
by AlSam
Thanks for the info, williamconley. I reviewed it and am a bit confused by the first two lines in point 4. The first bullet states that the mp3 file is moved from one folder to another but the following bullet states that it is now redundant. How is it redundant if it has been moved and not just copied?

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 12:50 pm
by AlSam
Also, one of the points references the astguiclient.conf. Is this meant to have the server directly deposit recordings to the server configured in the file? Does the perl script depend on the configurations in the astguiclient.conf file?

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 12:56 pm
by williamconley
AlSam wrote:Thanks for the info, williamconley. I reviewed it and am a bit confused by the first two lines in point 4. The first bullet states that the mp3 file is moved from one folder to another but the following bullet states that it is now redundant. How is it redundant if it has been moved and not just copied?

Because it was moved from one place to another on the Dialer. But it was first Copied to the archive server. Thus the local file is now the redundant copy. This is signified by moving it to the "you can delete me, because I'm redundant" FTP folder.

Once the upload to the archive server has completed, there is Zero need for the copy on the dialer. However: If you don't have a mirror or other copy of the audio files on the archive server, now you are left with the danger of the Archive server's HD dying the losing ALL your recordings now that they are conveniently all collected from all dialers to a single location. That single location is now prone to death of HD. You can either set up a mirror of that drive ... or use FTP2 to copy all the files AGAIN to another server. That's what FTP2 is for. If you don't use FTP2, then the files in FTP can be deleted any time you like. But if you DO use FTP2 to create your redundant backup of the archive server, then you can delete the files in FTP2 any time you like instead, but don't delete anything from FTP as those are still "in transit" and have not yet landed in the FTP2 server.

Described it like ... three times. I hope it sticks. lol 8-)

AlSam wrote:Also, one of the points references the astguiclient.conf. Is this meant to have the server directly deposit recordings to the server configured in the file? Does the perl script depend on the configurations in the astguiclient.conf file?

/usr/share/astguiclient/AST_CRON_audio_3_ftp.pl Uses credentials from /etc/astguiclient.conf, yes. AST_CRON_audio_4_ftp2.pl does not.

Updated Wiki. 8-)

Re: Differences in Perl Scripts

PostPosted: Tue May 19, 2020 1:37 pm
by AlSam
Thanks. That was very helpful.

Re: Differences in Perl Scripts

PostPosted: Tue Jun 09, 2020 1:36 pm
by AlSam
Tried running
Code: Select all
/usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl --help

which resulted in
Code: Select all
Can't locate Net/FTPSSL.pm in @INC (you may need to install the Net::FTPSSL module) (@INC contains: /usr/lib/perl5/site_perl/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.18.2 /usr/lib/perl5/vendor_perl/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.18.2 /usr/lib/perl5/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/5.18.2 /usr/lib/perl5/site_perl .) at /usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl line 392.
BEGIN failed--compilation aborted at /usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl line 392.

Took its suggestion (you may need to install the Net::FTPSSL module) and ran
Code: Select all
perl -MCPAN -e 'shell'

then
Code: Select all
install Net::FTPSSL

which resulted in
Code: Select all
....
Writing Makefile for Net::FTPSSL
Writing MYMETA.yml and MYMETA.json
Can't exec "make": No such file or directory at /usr/lib/perl5/5.18.2/CPAN/Distribution.pm line 2084.
  CLEACH/Net-FTPSSL-0.42.tar.gz
  make -- NOT OK
'YAML' not installed, will not store persistent state
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible
Failed during this command:
 CLEACH/Net-FTPSSL-0.42.tar.gz                : make NO

It references the make package. Do I download it from the OpenSUSE repositories or does it come from CPAN? I see it also states YAML not installed. Again, is this something that should be downloaded from OpenSUSE repositories or CPAN? Installation of the make package via OpenSUSE repository causes the addition of the site-config package. Will this addition have a negative impact on the server's proper operation?

Thanks

Re: Differences in Perl Scripts

PostPosted: Tue Jun 09, 2020 8:40 pm
by mflorell
On OpenSuSE, if you can install it through 'yast' then that should go much faster. The error you are getting is because you don't have the 'make' package installed, so cpan can't install anything. I'd suggest just trying to install the perl module though yast first.

Re: Differences in Perl Scripts

PostPosted: Wed Jun 10, 2020 9:12 am
by AlSam
Thanks for the quick response. I was able to install the required module. Is there a newer version (current is 180616-2248) of this script (AST_CRON_audio_3_ftp_FTPSSL.pl) which includes the --ftp-validate option referenced under the Suggestions section in the Vicidial Recording Management Clarified document? Also, is there any other reference material regarding the options available for the script? I am not clear on a few of them like debug, debugX, test, transfer-limit, list-limit, and max-files.

Re: Differences in Perl Scripts

PostPosted: Wed Jun 10, 2020 9:32 am
by mflorell
I think we updated that script in February, so I'd recommend updating to the latest svn/trunk revision.

As for documentation, all there is is right in the script, usually in the comments at the top.

Re: Differences in Perl Scripts

PostPosted: Thu Jun 11, 2020 11:09 am
by williamconley
Just because it's a good idea ... went ahead and moved this topic to "Support" instead of "General Discussion".

Re: Differences in Perl Scripts

PostPosted: Tue Jun 30, 2020 7:04 am
by AlSam
How does it determine which recordings it will process each time it runs? I find that my FTP server has recordings from yesterday yet there are still recordings from over a year ago on the Vicidial server.

Re: Differences in Perl Scripts

PostPosted: Tue Jun 30, 2020 8:49 am
by mflorell
Each script will only look in one specific folder for recordings to process. The folder it looks in depends on the script and the option flags.

Re: Differences in Perl Scripts

PostPosted: Tue Jun 30, 2020 8:56 am
by AlSam
I may not have provided enough detail in my question. I am using the --MP3 parameter and it, in fact, is processesing recordings from said folder. However, I don't understand the order in which recordings are selected to be processed from within the MP3 folder.

Re: Differences in Perl Scripts

PostPosted: Tue Jun 30, 2020 6:01 pm
by mflorell
The script uses the perl "readdir" function which really just calls the OS directory list function. It is not sorted by the script in any way, it uses whatever operating system order that the file list is supplied to it in. This might be the last modification date of the file or the creation date usually.

Re: Differences in Perl Scripts

PostPosted: Thu Jul 02, 2020 6:47 am
by AlSam
OK. I'll look into it. Thanks.
I noticed that the --max-files switch does nothing. I reviewed the script and found the only time it is referenced is in the help section. I am running the 200205-1716 version of the script.

Re: Differences in Perl Scripts

PostPosted: Mon Jul 13, 2020 2:09 pm
by AlSam
Thanks for all the help. I was able to setup the transfer of files successfully. I am going to be transferring recordings from a different Vicidial platform which has multiple Asterisk servers. Is it advisable to dump all recordings from the different Asterisk servers into a single directory within the storage server? What are the chances of having recordings with the same name in more than one Asterisk server? Are there any best practices that should be followed in this circumstance?

Re: Differences in Perl Scripts

PostPosted: Tue Jul 14, 2020 6:26 am
by mflorell
Due to filesystem limitations on the destination archive server, we always recommend using the dated directories that the Vicidial FTP perl scripts use.

Re: Differences in Perl Scripts

PostPosted: Tue Jul 14, 2020 7:10 am
by AlSam
I may not have explained myself properly. What I meant is that if it would be advisable to have the following structure or not.
Image
Or if recordings should go into the date folders regardless of the Asterisk server from whence they came.

Re: Differences in Perl Scripts

PostPosted: Tue Jul 14, 2020 7:42 am
by mflorell
We usually recommend putting each cluster's recordings in the same directory structure. So if your Astersk1, 2, 3... are all on the same VICIdial cluster, we would recommend sending them all to the same set of dated directories.

Re: Differences in Perl Scripts

PostPosted: Tue Jul 14, 2020 7:45 am
by AlSam
So there's no chance there would be two different recordings with the same name on different Asterisk servers (same cluster) which might end up having one overwrite the other?

Re: Differences in Perl Scripts

PostPosted: Tue Jul 14, 2020 9:15 am
by mflorell
I don't know how you are currently naming your recordings. If you use something like "FULLDATE_CUSTPHONE_AGENT" then yes there is no chance that recordings will have duplicate names.