MySQL Connect Error

[getswole89]

Hey Guys,

We've been getting MySQL Connect Error: Cannot assign requested address Active Call Backs
Build: 231115-1646
SVN Version: 3838

We've tried reinstalling the database server completely because we had thought this issue was caused from a restore from our previous database server.

Here's the Server Specs

7 Dialers
2 Web
1 Master DB
1 Slave DB

SuperMicro Servers:
Dual E5-2680 v4
Cores: 2x 14x 2.40 GHz (Dual 14 Core)
RAM: 128 GB RAM DDR 4 ECC reg.
Storage: 2x 960 GB SSD HW Raid 1

Is this from Hardware limitations or an issue with MySQL configuration?

[mflorell]

I can't say I've run into that issue before, it doesn't appear to be a VICIdial issue really.

I'd suggest using an IP address for your DB server configuration instead of a hostname if you're doing that. Also, you don't mention what version of MariaDB you are using.

[getswole89]

SET timestamp=1718719269;
select distinct phone_code from vicidial_list;
# User@Host: cron[cron] @ [I hide this for security]
# Thread_id: 12330 Schema: asterisk QC_hit: No
# Query_time: 7.922623 Lock_time: 0.000147 Rows_sent: 1 Rows_examined: 1208173
# Rows_affected: 0 Bytes_sent: 110
# Tmp_tables: 1 Tmp_disk_tables: 0 Tmp_table_sizes: 6507520
# Full_scan: Yes Full_join: No Tmp_table: Yes Tmp_table_on_disk: No
# Filesort: No Filesort_on_disk: No Merge_passes: 0 Priority_queue: No
#
# explain: id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
# explain: 1 SIMPLE vicidial_list ALL NULL NULL NULL NULL 1208172 1208172.00 100.00 100.00 Using temporary
#

[carpenox]

they are using mariadb Ver 15.1 Distrib 10.5.22-MariaDB, for Linux (x86_64) using EditLine wrapper - matt

[getswole89]

I can't say I've run into that issue before, it doesn't appear to be a VICIdial issue really.

I'd suggest using an IP address for your DB server configuration instead of a hostname if you're doing that. Also, you don't mention what version of MariaDB you are using.

Would it be an issue if our Master DB was on an AMD processor while all the rest are on Intel?

[williamconley]

Would it be an issue if our Master DB was on an AMD processor while all the rest are on Intel?

No. Just the version of MySQL should matter. But communication issues can arise for other reasons ... just not CPU related, as a rule.

Is this to/from a workstation or replication server or ...?

What versions are on both master and slave (if replication is involved, of course)

Cannot assign requested address Active Call Backs

Given the odds, you have someone in networking who is very smart on staff. Ask him if he (upon occasion) outsmarts himself. Or to be less cryptic: If the system says it can't assign an address, it's entirely likely there's a reason for that. Something is blocking that assignment. As I've also never bumped into that specific error, it would have to be a pretty cool configuration complexity (the odds on Matt and I both saying that on the same post are pretty low).

If it keeps happening, you may want to describe the network both physically and logically (hardware pathways and subnets in use on that hardware). Possibly a simple collision. But simple collisions can be tricky to track down sometimes.

[getswole89]

This issue seems to only happen whenever we go above 80 agents. We're using WAN for all the servers and have a 1GBIT internet connection. We also we're having issues with Table Locks which we fixed now we don't have table locks that take over 1 to complete.

Is there any My.cnf items we can change?

we also tried applying the following:

echo "3" > /proc/sys/net/ipv4/tcp_fin_timeout
echo 450000 > /proc/sys/net/ipv4/tcp_max_tw_buckets

net.ipv4.tcp_fin_timeout=3

[williamconley]

Are all the servers on the same physical network?

I'm going to strongly recommend a shared private physical network, of course, but it's also useful information to find out if the servers are on the same physical network with ONE hop between them (ie: ONE switch with all servers connected).

Where do you see this error? On the DB server? On a Dialer? What interface/log/file/table are you seeing this error?

When the error occurs, what is your symptom?

[getswole89]

The new server we got with the upgrades specs even though its from the same hosting and same location they can't set all the servers up on LAN but this issue is effecting everything from the admin side to the agent side and it only last for 1-3 seconds but on the agent side it disconnects the call, doesn't allow them to receive calls and on the admin side we see it quite often in the real time dashboard while monitoring but we've even modified the my.cnf file as well to see if that helps but still haven't gotten any luck yet

[williamconley]

The new server we got with the upgrades specs even though its from the same hosting and same location they can't set all the servers up on LAN but this issue is effecting everything from the admin side to the agent side and it only last for 1-3 seconds but on the agent side it disconnects the call, doesn't allow them to receive calls and on the admin side we see it quite often in the real time dashboard while monitoring but we've even modified the my.cnf file as well to see if that helps but still haven't gotten any luck yet

OK: So the answer to the question "are they on the same physical network" is ... NO.

It is entirely possible (IMHO: Likely) that you will not solve this problem until that answer is "YES".

Consider this: Your network traffic can be very low, or very high/intense. During intense moments every packet counts and can slow down the dialer or disrupt your ability to pull a web page. If your colocation facility has you in a rack with other clients who are not using much bandwidth, the system may run perfectly for any/all loads.

But when OTHER clients in the same rack (or otherwise using the same routers/switches) increase their network traffic, your traffic will now suffer. And you have Zero insight into the reason behind the problems because of course you can't see their traffic, as it is likely not even on the same subnet as you. Logically your servers will ignore these packets, but physically they will be sharing the wires and routers and switches. The wires, switches and routers will not be ignoring this traffic.

So randomly your system will work great or horribly or anywhere in between, and you have no way to tell when or why this is happening if it is a result of bottlenecked networking.

This is why the private network is so important: NO firewall needed, no other traffic, just Vicidial and its dialers and database humming away all alone. But if the "private" network ends up sharing wires/switches/routers just like the public one ... the problem has still not been resolved.

There are some fairly deep networking tools that can be brought to bear on the servers to see if their packets are being transmitted at "link-rated" capacity. But enlisting someone to fire those up could be quite expensive. Have you considered changing to a different facility or having a Vicidial Professional look directly into your system?