vicidial.org

by **DruRoland** » Tue Sep 03, 2013 1:12 pm

I recently upgraded our cluster; I added a second Vicibox install, 5.0.3, to my existing 4.0.3 dialer. I also SVN'd the 4.0.3 install to current and upgraded asterisk to 1.8.23.

The system works correctly for ~30 minutes, and stops dialing.
Looking at the realtime monitor, it shows ~140 active calls (of 150 max), but asterisk -rx 'sip show channels' shows only the usual registered agents and 0 calls to carrier.

I'm not sure how to resolve the mismatch between actual channels in use and shown channels in use.

by **DruRoland** » Wed Sep 04, 2013 5:12 pm

The problem continues, and it's killing our performance!
More info:
When this occurs, asterisk on both servers essentially freezes. Agents attempting to log in to the server aren't presented with calls, and there's no indication in the CLI that a call is attempted. SIP register/deregister both work though.

Anyone have any ideas where to look to solve this?

by **mflorell** » Thu Sep 05, 2013 5:29 am

My guess is that there is a misconfiguration somewhere, because we have several heavy outbound dialing clients now using Asterisk 1.8 without any issues. I would suggest starting with checking the Asterisk version on all of the servers settings in the web interface.

by **DruRoland** » Mon Sep 09, 2013 3:03 pm

Everything is on 1.8.23-vici.
I added a 3rd dialer, and have been disabling dialers trying to determine if it's related to a specific dialer. So far, with any combination of dialers available, it still freezes 30-90 minutes after dialing starts.

SERVER LISTINGS:
SERVER ID NAME SERVER IP ACTIVE ASTERISK TRUNKS GMT MODIFY
dialer00 Server dialer00 10.0.0.4 N 1.8.23.0-vici 0 -8.00 MODIFY
dialer01 Server dialer01 10.0.0.12 Y 1.8.23.0-vici 50 -8.00 MODIFY
dialer2 Server dialer2 10.0.0.9 N 1.8.21.0-vici 0 -8.00 MODIFY

by **DruRoland** » Mon Sep 09, 2013 3:05 pm

This is the log right before the last crash.

Code: Select all: [Sep 9 12:42:19] NOTICE[11612] chan_local.c: No such extension/context XXYYXXYYXXYYXX48600054@default while calling Local channel [Sep 9 12:42:19] NOTICE[11612] channel.c: Unable to call channel Local/XXYYXXYYXXYYXX48600054@default [Sep 9 12:42:19] NOTICE[11619] chan_local.c: No such extension/context XXYYXXYYXXYYXX48600054@default while calling Local channel [Sep 9 12:42:19] NOTICE[11619] channel.c: Unable to call channel Local/XXYYXXYYXXYYXX48600054@default [Sep 9 12:42:19] NOTICE[11628] chan_local.c: No such extension/context XXYYXXYYXXYYXX48600054@default while calling Local channel [Sep 9 12:42:19] NOTICE[11628] channel.c: Unable to call channel Local/XXYYXXYYXXYYXX48600054@default [Sep 9 12:42:19] NOTICE[11633] chan_local.c: No such extension/context XXYYXXYYXXYYXX48600054@default while calling Local channel [Sep 9 12:42:19] NOTICE[11633] channel.c: Unable to call channel Local/XXYYXXYYXXYYXX48600054@default [Sep 9 12:43:22] NOTICE[3040] chan_sip.c: No compatible codecs, not accepting this offer! [Sep 9 12:44:15] WARNING[13511] app_meetme.c: Unable to write frame to channel SIP/airespring-00001acb [Sep 9 12:44:26] NOTICE[3040] chan_sip.c: No compatible codecs, not accepting this offer! [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:43] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:44] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:47:45] WARNING[17543] dsp.c: Can only calculate silence on signed-linear frames :( [Sep 9 12:48:37] WARNING[3040] chan_sip.c: Rejecting offer with image stream due to UDPTL initialization failure [Sep 9 12:48:37] NOTICE[3040] chan_sip.c: No compatible codecs, not accepting this offer! [Sep 9 12:49:15] WARNING[18964] app_meetme.c: Unable to write frame to channel SIP/airespring-00001cc3 [Sep 9 12:49:38] WARNING[18980] app_meetme.c: Unable to write frame to channel SIP/airespring-00001cc0 [Sep 9 12:51:26] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.57:7079, rtcp halted Network is unreachable [Sep 9 12:51:27] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.63:7079, rtcp halted Network is unreachable [Sep 9 12:51:27] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.85:7079, rtcp halted Network is unreachable [Sep 9 12:51:27] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 173.245.32.22:10545, rtcp halted Network is unreachable [Sep 9 12:51:27] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.51:7079, rtcp halted Network is unreachable [Sep 9 12:51:27] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.59:7079, rtcp halted Network is unreachable [Sep 9 12:51:27] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.61:7079, rtcp halted Network is unreachable [Sep 9 12:51:28] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.58:7079, rtcp halted Network is unreachable [Sep 9 12:51:28] ERROR[3040] res_rtp_asterisk.c: RTCP SR transmission error to 10.0.0.67:7079, rtcp halted Network is unreachable

by **DruRoland** » Thu Sep 12, 2013 5:15 pm

I disabled SQL replication to migrate the slave to a different server, and the freezing only happens once a day now.

This is getting very frustrating.

by **mflorell** » Fri Sep 13, 2013 5:57 am

I noticed that you are using MySQL 5.5. We have documented problems with this version, and we switched to MariaDB 5.5 because of them. I would recommend that as your next step.

Also, could you detail what kind of disk system you are using on your database server?

by **DruRoland** » Fri Sep 13, 2013 1:36 pm

Heh, that's exactly what I'm doing. SQL was living in XenServer as the sole VM in the node, which is kinda silly, so it's being rebuilt as a dedicated SQL box using the 5.0.3 ISO.

The server is a Proliant DL380 G5, with 6 10k SAS drives in RAID 6 using HP's P400 controller, with 16Gb FB-DIMM

by **mflorell** » Sat Sep 14, 2013 5:55 am

RAID 6 is bad for DBs, you should use a 4-drive RAID 10 for the DB data partition, and maybe a 2-drive RAID 1 for your OS to be on.

Also, we only recommend LSI Logic MegaRAID with caching RAID controllers for high-end systems. Although I'm not familiar with HP's offerings, so it may be fine for your purposes.

by **DruRoland** » Mon Sep 16, 2013 3:24 pm

I wiped the SQL server and installed vicibox 5.0.3 svn 2021 as a dedicated MariaDB server.
And the dialer has frozen twice today.

I Like LSI as well; HP's controllers are very similar with battery backed write caching.
I'll rebuild it as you say, though the server's disk I/O averages 2MB/sec, well below where different RAID levels matter for performance.

by **DruRoland** » Thu Sep 19, 2013 1:34 pm

Rebuilt SQL with 4x 172Gb 10k SAS drives in RAID 10.

Asterisk still locks, requiring a restart to restore audio.

by **mflorell** » Fri Sep 20, 2013 6:15 am

Do you have slow query logging enabled?

by **DruRoland** » Mon Oct 07, 2013 3:43 pm

I did not; I had everything still at default.
I enabled slow logging, I'll report back what it turns up.

I saw this in the mysql_error.log file:

Code: Select all: 131007 10:41:54 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 10:42:37 [Note] /usr/sbin/mysqld: Normal shutdown 131007 10:42:37 [Note] Event Scheduler: Purging the queue. 0 events 131007 10:42:37 InnoDB: Starting shutdown... 131007 10:42:38 InnoDB: Shutdown completed; log sequence number 384606803 131007 10:42:38 [Note] /usr/sbin/mysqld: Shutdown complete 131007 10:42:38 mysqld_safe mysqld from pid file /var/run/mysql/mysqld.pid ended 131007 10:42:38 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 131007 10:42:38 [Warning] The syntax '--log-slow-queries' is deprecated and will be removed in a future release. Please use '--slow-query-log'/'--slow-query-log-file' instead. 131007 10:42:38 [Warning] Using unique option prefix myisam_recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead. 131007 10:42:38 InnoDB: The InnoDB memory heap is disabled 131007 10:42:38 InnoDB: Mutexes and rw_locks use GCC atomic builtins 131007 10:42:38 InnoDB: Compressed tables use zlib 1.2.7 131007 10:42:38 InnoDB: Using Linux native AIO 131007 10:42:38 InnoDB: Initializing buffer pool, size = 128.0M 131007 10:42:38 InnoDB: Completed initialization of buffer pool 131007 10:42:39 InnoDB: highest supported file format is Barracuda. 131007 10:42:39 InnoDB: Waiting for the background threads to start 131007 10:42:40 Percona XtraDB (http://www.percona.com) 5.5.33-MariaDB-31.1 started; log sequence number 384606803 131007 10:42:40 [Note] Server socket created on IP: '0.0.0.0'. 131007 10:42:40 [Warning] 'proxies_priv' entry '@ root@sql00' ignored in --skip-name-resolve mode. 131007 10:42:40 [Note] Event Scheduler: Loaded 0 events 131007 10:42:40 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.33-MariaDB-log' socket: '/var/run/mysql/mysql.sock' port: 3306 openSUSE package 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:05 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:08 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:09 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:09 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:09 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:11 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:13 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:13 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:13 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:13 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:13 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:13 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:13 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:14 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:15 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:15 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:15 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:15 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:15 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:15 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:16 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:16 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:16 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:16 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:16 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:19 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:19 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:19 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:19 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:19 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:19 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:20 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:20 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:22 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:22 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:22 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:22 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:22 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:23 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:23 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:23 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:23 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:26 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:28 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:28 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:28 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:28 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:28 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:30 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:30 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:30 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:31 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:31 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:31 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:31 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:31 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:31 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:32 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:32 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:33 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:34 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:35 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:35 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:35 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:35 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:35 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:38 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:38 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:38 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:38 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:39 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:44 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:44 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:44 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:44 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:44 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:44 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:45 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:45 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:46 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:47 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:47 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:47 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:47 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:48 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:52 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:52 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:52 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:52 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:52 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:53 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:53 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:53 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:56 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:56 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:56 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:58 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:58 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:59 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:59 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:59 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:38:59 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:02 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:02 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:03 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:04 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:04 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:07 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:07 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:12 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:18 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:24 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:32 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:34 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:34 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:43 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:50 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:55 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:55 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:56 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:56 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:39:59 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:03 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:03 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:06 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:08 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:08 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:08 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:08 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:09 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:09 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:17 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:20 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:20 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:20 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:20 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:20 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:21 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:22 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:22 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:25 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:26 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:26 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:28 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:28 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:29 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:32 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:32 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:32 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:35 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:35 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:36 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:36 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:39 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:39 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:39 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:42 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:42 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:47 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:48 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:51 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:56 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:40:56 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:41:02 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:41:07 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:41:08 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:41:10 [ERROR] mysqld: The table 'vicidial_manager' is full 131007 12:41:10 [ERROR] mysqld: The table 'vicidial_manager' is full

by **DruRoland** » Mon Oct 07, 2013 6:02 pm

I enabled slow logging, and this is from the latest crash:
mysqld.log:

Code: Select all: 131007 13:38:15 InnoDB: Initializing buffer pool, size = 128.0M 131007 13:38:15 InnoDB: Completed initialization of buffer pool 131007 13:38:15 InnoDB: highest supported file format is Barracuda. 131007 13:38:15 InnoDB: Waiting for the background threads to start 131007 13:38:16 Percona XtraDB (http://www.percona.com) 5.5.33-MariaDB-31.1 started; log sequence number 384824480 131007 13:38:16 [Note] Server socket created on IP: '0.0.0.0'. 131007 13:38:16 [Warning] 'proxies_priv' entry '@ root@sql00' ignored in --skip-name-resolve mode. 131007 13:38:16 [Note] Event Scheduler: Loaded 0 events 131007 13:38:16 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.33-MariaDB-log' socket: '/var/run/mysql/mysql.sock' port: 3306 openSUSE package 131007 15:54:18 [ERROR] mysqld: The table 'vicidial_manager' is full

mysql-slow.log

Code: Select all: /usr/sbin/mysqld, Version: 5.5.33-MariaDB-log (openSUSE package). started with: Tcp port: 3306 Unix socket: /var/run/mysql/mysql.sock Time Id Command Argument # Time: 131007 14:18:37 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 59912 Schema: asterisk QC_hit: No # Query_time: 1.031485 Lock_time: 0.000077 Rows_sent: 92 Rows_examined: 915707 use asterisk; SET timestamp=1381180717; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE order by list_id; # Time: 131007 14:31:13 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 120036 Schema: asterisk QC_hit: No # Query_time: 1.027506 Lock_time: 0.000074 Rows_sent: 92 Rows_examined: 915717 SET timestamp=1381181473; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 14:35:00 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 138855 Schema: asterisk QC_hit: No # Query_time: 1.035044 Lock_time: 0.000075 Rows_sent: 92 Rows_examined: 915724 SET timestamp=1381181700; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 14:48:47 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 208646 Schema: asterisk QC_hit: No # Query_time: 1.051341 Lock_time: 0.000081 Rows_sent: 92 Rows_examined: 915749 SET timestamp=1381182527; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 14:49:59 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 214889 Schema: asterisk QC_hit: No # Query_time: 1.009476 Lock_time: 0.000079 Rows_sent: 92 Rows_examined: 915750 SET timestamp=1381182599; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 15:01:45 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 281257 Schema: asterisk QC_hit: No # Query_time: 1.027571 Lock_time: 0.000059 Rows_sent: 92 Rows_examined: 915755 SET timestamp=1381183305; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 15:21:42 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 380651 Schema: asterisk QC_hit: No # Query_time: 1.022164 Lock_time: 0.000060 Rows_sent: 92 Rows_examined: 915779 SET timestamp=1381184502; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 15:22:10 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 382864 Schema: asterisk QC_hit: No # Query_time: 1.017879 Lock_time: 0.000060 Rows_sent: 92 Rows_examined: 915780 SET timestamp=1381184530; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 15:32:34 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 435769 Schema: asterisk QC_hit: No # Query_time: 1.026298 Lock_time: 0.000075 Rows_sent: 92 Rows_examined: 915791 SET timestamp=1381185154; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 15:39:56 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 473205 Schema: asterisk QC_hit: No # Query_time: 1.030682 Lock_time: 0.000059 Rows_sent: 92 Rows_examined: 915799 SET timestamp=1381185596; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; # Time: 131007 15:48:23 # User@Host: vicidbadmin[vicidbadmin] @ [10.0.0.12] # Thread_id: 522688 Schema: asterisk QC_hit: No # Query_time: 1.035337 Lock_time: 0.000074 Rows_sent: 92 Rows_examined: 915812 SET timestamp=1381186103; SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id; /usr/sbin/mysqld, Version: 5.5.33-MariaDB-log (openSUSE package). started with: Tcp port: 3306 Unix socket: /var/run/mysql/mysql.sock Time Id Command Argument

by **DruRoland** » Mon Oct 07, 2013 6:44 pm

mysqldumpslow -a -s r -t 5 /var/log/mysql/mysql-slow.log

Code: Select all: Reading mysql slow query log from /var/log/mysql/mysql-slow.log Count: 11 Time=1.03s (11s) Lock=0.00s (0s) Rows_sent=92.0 (1012), Rows_examined=915760.3 (10073363), vicidbadmin[vicidbadmin]@[10.0.0.12] SELECT vls.list_id,list_name,list_description,count(*) as tally,active,list_lastcalldate,campaign_id,reset_time,DATE_FORMAT(expiration_date,'%Y%m%d') from vicidial_lists vls,vicidial_list vl where vls.list_id=vl.list_id group by list_id order by list_id Died at /usr/bin/mysqldumpslow line 179, <> chunk 11.

by **DruRoland** » Mon Oct 07, 2013 11:18 pm

I don't think it's a slow query problem. I think it's vicidial_manager.
I verified both dialers crontabs had AST_flush_DBqueue.pl, and ran on both:

Code: Select all: dialer01:~ # perl /usr/share/astguiclient/AST_flush_DBqueue.pl -t -----TESTING ----- TEST NOW DATETIME: 2013-10-07 21:04:00 1 HOUR AGO DATETIME: 2013-10-07 20:04:00 - vicidial_manager flush - OPTIMIZE vicidial_manager - OPTIMIZE vicidial_live_agents - OPTIMIZE vicidial_drop_rate_groups - OPTIMIZE vicidial_campaigns - OPTIMIZE vicidial_lists

I'm not sure where else to look

by **silent_ukr** » Sat Nov 02, 2013 6:00 pm

Hi,

did you have any luck with this issue?

Thanks

by **williamconley** » Sat Nov 02, 2013 8:24 pm

mysqld: The table 'vicidial_manager' is full

this was likely the primary issue.

by **silent_ukr** » Sun Nov 03, 2013 3:41 pm

Thank you for clarification. I have similar symptoms and found the same errors in my:
/var/www/html/agc/vicidial_mysql_errors.txt

Do you by chance know what was done to resolve it?

by **williamconley** » Sun Nov 03, 2013 8:14 pm

1) increase the size possible for memory tables

2) decrease the size of this table

For #1, you may need to modify mysql itself, hopefully just my.cnf, but possibly a recompile (and having more memory than the average computer may not be a bad idea ...)
For #2, you need to look at setting the scripts that prune this data to run more often or reduce the amount of time before the records are pruned in the script or preferences

by **silent_ukr** » Mon Nov 04, 2013 10:07 pm

Thank you. I will try.

by **Nefariousparity** » Sat Dec 10, 2016 5:19 pm

I have had similar issues with databases that 16GB of memory. I have had to adjust the script that prunes the vicidial_manager table to keep it from locking up and stop dialing when
hitting a high number of channels.

The script is
/usr/share/astguiclient/AST_flush_DBqueue.pl

The lines you need to edit are number 80 and 81

80 $purge_seconds = '240';
81 $purge_seconds_half = '180';

=====^ Not default settings
###Defaut####

80 $purge_seconds = '3600';
81 $purge_seconds_half = '1800';

I have also seen this issue arise with people who are dialing hard on servers with 32GB of memory. I am increasing server memory to 64GB this weekend.

My question would be is there anything I have to adjust after increasing server memory? This is what my my.cnf looks like now.

# The following options will be passed to all MariaDB clients
[client]
# password = your_password
port = 3306
socket = /var/run/mysql/mysql.sock
loose-local-infile=1

[mysqld]
log_bin=/var/lib/mysql/mysql-bin
binlog_format=mixed
server-id=1
relay-log = /var/lib/mysql/mysqld-relay-bin
slave-skip-errors = 1032,1690,1062
datadir=/var/lib/mysql
sql_mode=NO_ENGINE_SUBSTITUTION
port = 3306
socket = /var/run/mysql/mysql.sock
skip-external-locking
skip-name-resolve
connect_timeout=60
long_query_time=3
slow_query_log=1
slow-query-log-file=/var/log/mysql/mysqld-slow.log
max_connections=2048
key_buffer_size=4G
max_allowed_packet=16M
table_open_cache=512
table_definition_cache=2048
open_files_limit=24576
sort_buffer_size=4M
net_buffer_length=8K
read_buffer_size=4M
read_rnd_buffer_size=16M
myisam_sort_buffer_size=128M
join_buffer_size=1M
thread_cache_size=100
query_cache_size=32M
thread_concurrency=8
default-storage-engine=MyISAM
expire_logs_days=3
concurrent_insert=2
myisam_repair_threads=2
myisam_use_mmap=1
skip-innodb
delay_key_write=ALL
max_write_lock_count=1

[mysqld_multi]
mysqld = /usr/bin/mysqld_safe
mysqladmin = /usr/bin/mysqladmin
log = /var/log/mysqld_multi.log

# If you want to use mysqld_multi uncomment 1 or more mysqld sections
# below or add your own ones.

# WARNING
# --------
# If you uncomment mysqld1 than make absolutely sure, that database mysql,
# configured above, is not started. This may result in corrupted data!
#
# [mysqld1]
# port = 3306
# datadir = /var/lib/mysql
# pid-file = /var/lib/mysql/mysqld.pid
# socket = /var/lib/mysql/mysql.sock
# user = mysql

# [mysqld2]
# port = 3307
# datadir = /var/lib/mysql-databases/mysqld2
# pid-file = /var/lib/mysql-databases/mysqld2/mysql.pid
# socket = /var/lib/mysql-databases/mysqld2/mysql.sock
# user = mysql

# [mysqld3]
# port = 3308
# datadir = /var/lib/mysql-databases/mysqld3
# pid-file = /var/lib/mysql-databases/mysqld3/mysql.pid
# socket = /var/lib/mysql-databases/mysqld3/mysql.sock
# user = mysql

# [mysqld6]
# port = 3309
# datadir = /var/lib/mysql-databases/mysqld6
# pid-file = /var/lib/mysql-databases/mysqld6/mysql.pid
# socket = /var/lib/mysql-databases/mysqld6/mysql.sock
# user = mysql

[myisamchk]
key_buffer = 1G
sort_buffer = 384M
read_buffer = 512M
write_buffer = 256M

!includedir /etc/my.cnf.d

Sorry to revive so an old thread, I feel like this should be a sticky.

by **williamconley** » Sun Dec 25, 2016 9:46 pm

I'm sure you have a question in there somewhere. But posting an entire configuration file instead of only the relevant changes/differences from the original can cause a bit of a problem.

There are two primary my.cnf sets, the default one and the big-vici cnf file for DB servers in large clusters.

If you were to post your differences from whichever one of those is closest to yours, you may get a better response.

But, in advance: That's the purpose of the big-vici cnf file .... "those are the best settings" for the bigger servers. As for changes necessary when increasing memory, nope. Not really necessary to alter the cnf in response to more memory. However: individual servers are often used differently. Keep an eye on mysql and resources and see if anything is amiss, and adjust accordingly. There are tools available to see what's happening.

vicidial.org

Cluster call filling problem

Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Re: Cluster call filling problem

Who is online