I note that you provided your installer with version (very good), but left off your Vicidial Version with Build. Please remember to ALWAYS include that very important information.
More info is needed so we can to rule out some obvious causes. How many cores and at what speed plus the number of Live calls is very useful. Your inclusion of the brand name of the server and CPU are only cool from a "sell me a server!" standpoint, the brand name on your server is however only of interest in an "aside" way until we've ruled out all other issues. I admit, however, that there have been some brand-specific issues over the years, so not bad to include it. Just not likely "first expectation of the fault" being brand-related.
So some information that may be useful:
* Vicidial Version with Build
* Number of Cores (physical and logical if hyperthreading is on)
* Number of live calls when the problem begins (if it's always present, this is obviously also useful information)
* Average Server Load when the problem begins
* Are these agents able to log in to another server and get good call quality? Or is this problem possibly related to these agents? (IE: This call center could have seriously crappy audio cards in their workstations: switch to USB headsets!)
* Networking could still be the issue if this server is on a different switch to the gateway OR a different switch to these agents.
* Kumba mentioned a kernel option that may reduce server load due to standard inclusion of cpu intense modules that are not needed when not running in a Virtualized environment, your 8.1.1 install may be in need of that kernel modification. Vicibox 8.1.2 has been released, which may or may not include that modification out-of-the-box, but I've done no research and don't have time to reread the post in question:
viewtopic.php?f=4&t=38828