Change recording URL to FQDN and https

[dspaan]

By default the links to recordings are shown as http://xxx.xxx.xxx.xxx/RECORDINGS/MP3/Y ... 22-all.mp3

I would like to replace the http part with https and the IP with the FQDN. Can anyone tell me how to do this?

[thephaseusa]

/etc/astguiclient.conf
Is where you set VARHTTP_path

As for the recordings already in the system you can change the path with
/usr/share/astguiclient/ADMIN_update_archive_url.pl —help (to look at options)

John M

[dspaan]

Hi phase, thanks for responding!

In astguiclient.conf i see:

Code: Select all

# FTP recording archive connection information
VARFTP_host => 10.0.0.4
VARFTP_user => cron
VARFTP_pass => test
VARFTP_port => 21
VARFTP_dir => RECORDINGS
VARHTTP_path => http://10.0.0.4


But i'm not using the FTP option. I'd like to change the URL for the recordings that are stored on the vicidial server itself.

I already was aware of the ADMIN_update_archive_url.pl script but as you said this is only for the old recordings.

Is there a way to change the URL for new recordings?

[blackbird2306]

The recording url ip / FQDN will be populated by "VARserver_ip" in astguiclient.conf file.

Code: Select all

# The IP address of this machine
VARserver_ip => xxxx


There is also "--HTTPS" flag option in AST_CRON_audio_2_compress.pl (and also in the other recordings process scripts). So try this in your crontab:

Code: Select all

1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3 --HTTPS


[dspaan]

Thanks! i added the --HTTPS flag to crontab

If i replace my IP in astguiclient.conf with an mydomain.com FQDN won't this break other functions of vicidial?

[williamconley]

Thanks! i added the --HTTPS flag to crontab

If i replace my IP in astguiclient.conf with an mydomain.com FQDN won't this break other functions of vicidial?

as long as you're only talking about VARHTTP_path ... that is its only use. To generate those links.

[dspaan]

But what i see in astguiclient.conf VARHTTP_path is part of the ftp archive process which i'm not using. Blackbird mentioned VARServer_ip

[williamconley]

If you are using the FTP process to send recordings to the archive server, the VARHTTP_path is used to set the ip/domain name of the final destination of the file.

Perhaps you should test a single file after hours and see how the process works. Note that you can (oddly enough) pull a file back from the /FTP/ folder to the /MP3/ folder and it will be processed again, and pushed to the FTP server again, and the URL will be updated in the DB again. There's no limit to the number of times you do this. So you can change that value and move a file and then run the archive script and check what happened to the recording url. As often as it takes to get "perfection".

[blackbird2306]

But i'm not using the FTP option. I'd like to change the URL for the recordings that are stored on the vicidial server itself.

But he's not using the FTP cron scripts, and therefore it is "VARserver_ip".

Of course you shouldn't change the VARserver_ip variable, if it is only for recording purposes! The question is why do you want to change the working ip in your recording links?
Here are two solutions:
1.
Admin --> Servers --> Select Server --> Select in "Recording Web Link" option "ALT_IP" and fill the "Alternate Recording Server IP" field with your FQDN.
This won't change the url in your database in real, but it will pretend the FQDN by replacing the database url.

2.
Open "AST_CRON_audio_2_compress.pl" and change at line 219:

Code: Select all

# Customized Variables
#$server_ip = $VARserver_ip;      # Asterisk server IP
$server_ip = "www.changeasyoulike.com";


[williamconley]

1) Then he SHOULD be using the FTP archive script (since it's very useful for this purpose: Storing the audio recordings on a named domain).

2) Do not change your server IP just to edit the domain of the recordings link. Unnecessary (and risky if done improperly).

* The server upon which Vicicial resides may have as many domains as you choose to point to that server's IP address. No limits. Vicidial neither knows nor cares.
* The domain in the URL will act exactly like the IP, you may substitute one for the other at will (unless you alter the apache configuration to block this: But if you do, you'll know you can point any domain anywhere you like without altering Vicidial's configuration).
* If you want the links Inside vicidial to change: Have a look at the options for this server under admin->Servers. Alternate IP, External IP ... these can be used to modify the URL presented to those who click on the recording links without altering any Vicidial configuration files or options. Once you've decided on "alternate" vs "external" (no difference really, but read the ?) then set "Recording Web Link" to your choice and the recording links are altered WITHOUT editing anything else in the system.

[blackbird2306]

1) Then he SHOULD be using the FTP archive script (since it's very useful for this purpose: Storing the audio recordings on a named domain).

He's not using them and that's OK. For me it's not a good idea to use this script only for his purpose. Push them to FTP directory, start a ftp transfer only to change url in database.

Apparently it's starting again, nice...

2) Do not change your server IP just to edit the domain of the recordings link. Unnecessary (and risky if done improperly).

Of course you shouldn't change the VARserver_ip variable, if it is only for recording purposes! The question is why do you want to change the working ip in your recording links?

* If you want the links Inside vicidial to change: Have a look at the options for this server under admin->Servers. Alternate IP, External IP ... these can be used to modify the URL presented to those who click on the recording links without altering any Vicidial configuration files or options. Once you've decided on "alternate" vs "external" (no difference really, but read the ?) then set "Recording Web Link" to your choice and the recording links are altered WITHOUT editing anything else in the system.

1.
Admin --> Servers --> Select Server --> Select in "Recording Web Link" option "ALT_IP" and fill the "Alternate Recording Server IP" field with your FQDN.
This won't change the url in your database in real, but it will pretend the FQDN by replacing the database url.

[williamconley]

start a ftp transfer only to change url in database

Actually, no: The Vicidial system stores ALL recordings in a single folder. This is a very bad thing.

* Over time that folder will fill up and it can literally become impossible to even view the list of files in the folder and very difficult to manage the files as a result.
* Storing the Audio Recordings on the same HD as the Vicidial Database allows those recordings to fill the HD and crash the system. Any time drive usage exceeds 75% we warn clients. Over 90% and production is notably affected. Over 95% and crashes begin to occur regularly. At 100%: System goes offline ... and now the client will begin to wonder if they should put the recordings somewhere else. We get these calls a lot.
* If the HD dies those recordings are all GONE. If you back it up and the HD dies, then you have thousands (hundreds of thousands?) of recordings to restore that have no reason to be on that HD any more. Takes much longer than restoring without those files. And Vicidial's backup script doesn't back them up in the first place, so you need a different method to back them up.
* Access to those files requires access to the Vicidial server itself. Even If you provide a direct link to just that one file, you must allow web access to the Vicidial web for it to work. If you move the files to an FTP server, however, access to that server is not related to Vicidial.

However: If you activate the FTP service and push those recordings to another server, those problems are solved.

Even if you use FTP on the SAME server you can solve all those problems pretty easily. Second HD mounted in the /home/ folder for instance will remove the need to restore that drive if the Vicidial HD dies. If you require redundancy, mirror that drive. The FTP system segregates the recordings into daily folders, so no single folder will ever be so full that it's difficult to manage.

So it's not just about one thing. FTP pushing is not difficult, but it has been known to save a lot of headaches over time (just like NOT using it has Caused a lot of headaches over time ... and resulted in support calls to us that we make money from, but we prefer Coding and Hosting for money more than fixing broken systems).

[blackbird2306]

Sure you mentioned points, which are true and very important, but we were not discussing the advantages of using an external place for recordings at this point. I'm sure he was aware of. His question was, how to change the recording link URL in database. And I referred to his question.

[williamconley]

Sure you mentioned points, which are true and very important, but we were not discussing the advantages of using an external place for recordings at this point. I'm sure he was aware of. His question was, how to change the recording link URL in database. And I referred to his question.

I try not to follow client requests out the window. Since it's actually the client that takes the fall. Give them what they need to make an informed decision. For instance: If they can simply change a value in Admin->Servers and remove the need to alter the server IP or all the database entries (both of which can and will break functionality), I prefer to inform rather than provide information that could lead to them needing an Emergency Fix down the road.

I don't like making money that way.

Capisce?

[blackbird2306]

Yes we all did capisce now! But he was asking how to change the url in database. And we didn't know what he tried to establish. That's why I asked him:

The question is why do you want to change the working ip in your recording links?

Perhaps, it is not just a simple change a value in Admin->Servers. Perhaps he wants to export the call report, where this simple change a value doesn't work or he is externally pulling these information directly from db or whatever. We don't know!

[williamconley]

Yes we all did capisce now! But he was asking how to change the url in database. And we didn't know what he tried to establish. That's why I asked him:

The question is why do you want to change the working ip in your recording links?

Perhaps, it is not just a simple change a value in Admin->Servers. Perhaps he wants to export the call report, where this simple change a value doesn't work or he is externally pulling these information directly from db or whatever. We don't know!

Thus telling him to change his server IP value is ... not the direction I'd take. Options of what can be done and what the results will be and the reasons for all of it. Full disclosure. Not a partial answer that can break the system.

[blackbird2306]

I have never told him to change this value. The only thing I wrote was:

Code: Select all

The recording url ip / FQDN will be populated by "VARserver_ip" in astguiclient.conf file.

and after that I advised him against it:

Code: Select all

Of course you shouldn't change the VARserver_ip variable...

Wait... But the strange thing is, why didn't you give him the same advice directly, if it could be misunderstood?

Or rather he asked:

If i replace my IP in astguiclient.conf with an mydomain.com FQDN won't this break other functions of vicidial?

and your only answer was:

as long as you're only talking about VARHTTP_path ... that is its only use. To generate those links.

[williamconley]

At the risk of stealing your thunder after you'd already done it? Really? I filled in the blanks.

[blackbird2306]

Bravo you won! You are right once more. But tomorrow is also a day...

[williamconley]

Bravo you won! You are right once more. But tomorrow is also a day...

Tomorrow never arrives.

[blackbird2306]

Perhaps tomorrow never arrives, yesterday never was or today never ends, but it will not change that tomorrow is also a day!

[williamconley]

"Saturday" is a day. "Tomorrow" never arrives. Thus it does not exist. It's an idea. A thought. But it's not real. Is something that's not real ... a "day"? At least yesterday "was" a day. But tomorrow will never actually be one. If you'd like to wait until tomorrow to answer, you may even find that out. IMHO.

[blackbird2306]

Bill you are not only mr. know-it-all you are also a philosopher too. But there is an error in your reasoning: Why do you think that something, which never arrives for you does not exist in reality? Jokes aside: For you it's now Tuesday, but I'm living just right now in your meaning of tomorrow (the oxford dictionary defines tomorrow: On the day after today), because it's now Wednesday for me here in Germany. Or something else: you are looking old and ugly in your web pics. You will never arrive something else, but for me it's real. Capisce?

[williamconley]

... which never arrives for you ...

Are you ... from the future?

PS: I don't have any web pics. Sorry to disappoint. I get younger every day.

[dspaan]

Goodmorning gentlemen! A new day has arrived once again
How nice to see this resulted in a philosophical debate but let me respond to the matter at hand.

Thanks! i added the --HTTPS flag to crontab

I enabled this but now of course the links to the recordings still have the server IP in the URL which will cause your browser to give an invalid certificate message, unless you install a Let's Encrypt certificate for the server ip i guess, didn't try.

Admin --> Servers --> Select Server --> Select in "Recording Web Link" option "ALT_IP" and fill the "Alternate Recording Server IP" field with your FQDN.
This won't change the url in your database in real, but it will pretend the FQDN by replacing the database url.

I have enabled this and this was an easy solution! Working nicely.

Do not change your server IP just to edit the domain of the recordings link. Unnecessary (and risky if done improperly).

* The server upon which Vicicial resides may have as many domains as you choose to point to that server's IP address. No limits. Vicidial neither knows nor cares.
* The domain in the URL will act exactly like the IP, you may substitute one for the other at will (unless you alter the apache configuration to block this: But if you do, you'll know you can point any domain anywhere you like without altering Vicidial's configuration).
* If you want the links Inside vicidial to change: Have a look at the options for this server under admin->Servers. Alternate IP, External IP ... these can be used to modify the URL presented to those who click on the recording links without altering any Vicidial configuration files or options. Once you've decided on "alternate" vs "external" (no difference really, but read the ?) then set "Recording Web Link" to your choice and the recording links are altered WITHOUT editing anything else in the system.

This is true but in this case, this is a small virtualized vicidial server with less then 10 agents on it and a single FQDN.

start a ftp transfer only to change url in database

Actually, no: The Vicidial system stores ALL recordings in a single folder. This is a very bad thing.

* Over time that folder will fill up and it can literally become impossible to even view the list of files in the folder and very difficult to manage the files as a result.
* Storing the Audio Recordings on the same HD as the Vicidial Database allows those recordings to fill the HD and crash the system. Any time drive usage exceeds 75% we warn clients. Over 90% and production is notably affected. Over 95% and crashes begin to occur regularly. At 100%: System goes offline ... and now the client will begin to wonder if they should put the recordings somewhere else. We get these calls a lot.
* If the HD dies those recordings are all GONE. If you back it up and the HD dies, then you have thousands (hundreds of thousands?) of recordings to restore that have no reason to be on that HD any more. Takes much longer than restoring without those files. And Vicidial's backup script doesn't back them up in the first place, so you need a different method to back them up.
* Access to those files requires access to the Vicidial server itself. Even If you provide a direct link to just that one file, you must allow web access to the Vicidial web for it to work. If you move the files to an FTP server, however, access to that server is not related to Vicidial.

However: If you activate the FTP service and push those recordings to another server, those problems are solved.

Even if you use FTP on the SAME server you can solve all those problems pretty easily. Second HD mounted in the /home/ folder for instance will remove the need to restore that drive if the Vicidial HD dies. If you require redundancy, mirror that drive. The FTP system segregates the recordings into daily folders, so no single folder will ever be so full that it's difficult to manage.

So it's not just about one thing. FTP pushing is not difficult, but it has been known to save a lot of headaches over time (just like NOT using it has Caused a lot of headaches over time ... and resulted in support calls to us that we make money from, but we prefer Coding and Hosting for money more than fixing broken systems).

Yes i have experienced many of the issues you mentioned in the past, servers getting stuck and not being able to browse the recording directory. What we now do is use this part of the crontab to remove them after 90 days:

Code: Select all

### remove old recordings more than 90 days old, and delete originals after 1 day
24 0 * * * /usr/bin/find /var/spool/asterisk/monitorDONE -maxdepth 2 -type f -mtime +90 -print | xargs rm -f
24 1 * * * /usr/bin/find /var/spool/asterisk/monitorDONE/ORIG -maxdepth 2 -type f -mtime +1 -print | xargs rm -f

But of course this still does not factor in how many agents and thus recordings will fill up the disk and the issue can still happen. So i will look into setting up some kind of extra storage for recordings. Ideally a central server for different clients.

Furthermore i noticed in this script the following:

/usr/share/astguiclient/AST_CRON_audio_2_compress.pl

Code: Select all

# 80302-1958 - First Build
# 80731-2253 - Changed size comparisons for more efficiency
# 90727-1417 - Added GSW format option
# 101207-1024 - Change to GSW option because of SoX flag changes in 14.3.0
# 110524-1059 - Added run-check concurrency check option
# 160523-0652 - Added --HTTPS option to use https instead of http in local location
# 170212-0732 - Added --file-sorting option to put files into dated directories (THIS WILL NOT ALLOW FTP ARCHIVING)

So it is possible to place recordings in different directories without the FTP option.

That's why I asked him:
The question is why do you want to change the working ip in your recording links?

The reason i started this thread is that the client who is using this server asked this question, the most important part was using https and not http but because of not having a cert for the IP the other part (using the FQDN in the URL) was still relevant.

[dspaan]

Posting back on this issue, i found out that if you setup the recording URL like above the URL is broken on the QC modify page. I posted this in Mantis:
http://www.vicidial.org/VICIDIALmantis/view.php?id=1148

[blackbird2306]

I also tried it but I can't replicate this issue. Are you experiencing this problem also in admin modify lead page (admin_modify_lead.php) and not only in QC Modify Lead page? Give us more details, what do you mean by "broken" or "The FQDN gets stripped from the URL..."? And where is it exactly "broken"?

[williamconley]

Posting back on this issue, i found out that if you setup the recording URL like above the URL is broken on the QC modify page. I posted this in Mantis:
http://www.vicidial.org/VICIDIALmantis/view.php?id=1148

Have you tested this on a more recent version? (You show SVN 2951 from nine months ago in your sig ...)

[dspaan]

Sorry for not responding sooner. I had already submitted this to the issue tracker: http://www.vicidial.org/VICIDIALmantis/view.php?id=1148