delete recording only if copied

[mav2287]

When setting up the recording deletion and recording ftp copy over script do they check to make sure the recording is copied over to the recording server? If not is there a way to set the scripts to only delete a recording if it has been verified and copied over to the recording server?

[williamconley]

Good question.

When files are copied using the ftp archive script in crontab -e, they are first copied and then (if successful, if I recall correctly) moved to the FTP folder. Thus, anything in the FTP folder can be deleted at will.

If not sure, you could certainly test this theory by waiting until after hours and turning off your ftp server. Then make a recording and see if it does or does not get moved to the FTP folder. It should not.

Note, of course, that it would be unwise to accept anyones (including my) word on this topic. Especially when you can just Test it.

[mav2287]

That is what I thought didn't know though if there was a script that did all that on it's own or if I need to make a hybrid script to ftp then delete.

[mav2287]

The FTP3 script will move them not delete them. If I get time I will rewrite it so that once everything is confirmed on the archive server it then delete them

[williamconley]

it is not expected to delete them. it copies them and then moves them to the FTP folder signifying "this file is done, and has been moved via FTP". There is a job in "crontab -e" that is commented out that can be used to delete these files when they reach a certain age. This gives your FTP server enough time to mirror them (for data redundancy) before you delete the originals.

Also, you can use the 2nd FTP script to copy them to another site (and that script will move them to the FTP2 folder), if you like. Assuming you didn't delete them when you copied them.

Check out the jobs in the crontab and execute all of them with --help (if they are external scripts) and see what possibilities exist. You don't actually have to write any code to get it working properly, specific to your business model and "safety zone".

[mav2287]

Thanks! that helped me knock this out. What I ended up doing setting up the FTP script to upload and validate so it would only move the file to FTP folder if it was uploaded and validated. I then setup the deleting script to only delete out of the FTP folder. This way it would only delete the items that had been uploaded. I then did various tests on the server and raid that was being used for storage. Worked out great. I have the FTP server on the raid set to Chroot the user to the raid. In the event the raid is down it will throw an error and when the script goes to validate it will fail and leave the file in place. Once it all comes back online the script will be able to connect transfer and validate. Thanks again!

[williamconley]

Sounds like a clean script. Now switch it to scp to make it perfect.

[mav2287]

Is there any existing SCP script I can start on? I don't mind putting one together from scratch, but having somewhere to start is always quicker.

[mflorell]

We have done an FTPSSL one before(included in svn/trunk), but the problem you usually run into on a busy system is that you quickly run out of bandwidth or time to transmit all of the recordings. Using encryption adds more CPU cycles and data needed to send the same files.

With the VDsales_export script we also have SFTP and SFTPSSH2 versions in svn/trunk that you can try.

[mav2287]

That for the heads up Matt. I think in this case I may just let it be for a while. We are really happy with what we have. One of the biggest things for us was making sure we never lost a call recording.

[williamconley]

Is there any existing SCP script I can start on? I don't mind putting one together from scratch, but having somewhere to start is always quicker.

Actually, we just modified the FTP version to use SCP instead. Then we added trust between the servers to avoid the need to log in. And we did not add SSL. The purpose of the method was to reduce the number of lines executing. We've also had a couple clients request bandwidth limiting (scp has this). And a few other clients requested rsync instead (which they do after hours). All sorts of interesting needs for moving files.

My favorite is one that uses scp and deletes upon success and then rsyncs to the amazon cloud for perma-storage (in case of catastrophic failure). They pay for SALE storage only, so only sale dispo categories are pushed via SCP. The rest are deleted after one week from all dialers.