I am having an issue on multi-server system. Calls are waiting for agents while agents is waiting aswell. They are getting stuck in the queue for sometime. Even the customer is complaining regarding the issue that he has to wait for the agent.
Server is running on CentOS 6.8, scratch installed from guides. VERSION: 2.14-583a , BUILD: 161226-2224, No Sangoma/Digium, same specifications as 2 x (E3-1270v3), 32g, 2x240g SSD.
I have done some research but i don't think its overloaded since they are running on average load of 0.7 with a spike of 1.00 at times. I have done mysql tuning via the shell script inside the svn. Since the server is in remote location in a reputable DC, confirmed that there firewall or routers are not choking either, the ping also seems stable and mtr also seems to be stable with zero packet loss/chokes. Agents are running Firefox as their browser. MySQL DB size is about 15g with about 3million records in vicidial_log and fairly large amount of vicidial_list. (DB was archived but nothing purdged). dial_logs,carriers_logs,call_logs,log_extended are removed on weekly basis). <0.05% i/o wait time.
Have not got hands on my-big-vici.cnf by Kumba.
- Code: Select all
Uptime = 2 days 23 hrs 49 min 3 sec
Avg. qps = 649
Total Questions = 167928244
Threads Connected = 37
Server has been running for over 48hrs.
It should be safe to follow these recommendations
To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service
SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 1.000000 sec.
You have 1958 out of 167928505 that take longer than 1.000000 sec. to complete
Your long_query_time seems to be fine
BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/5.1/en/point-in-time-recovery.html
WORKER THREADS
Current thread_cache_size = 16
Current threads_cached = 10
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine
MAX CONNECTIONS
Current max_connections = 1200
Current threads_connected = 38
Historic max_used_connections = 193
The number of used connections is 16% of the configured maximum.
Your max_connections variable seems to be fine.
MEMORY USAGE
Max Memory Ever Allocated : 6 G
Configured Max Per-thread Buffers : 31 G
Configured Max Global Buffers : 1 G
Configured Max Memory Limit : 33 G
Physical Memory : 31.22 G
Max memory limit exceeds 90% of physical memory
KEY BUFFER
Current MyISAM index space = 5 G
Current key_buffer_size = 1 G
Key cache miss rate is 1 : 140
Key buffer fill ratio = 86.00 %
You could increase key_buffer_size
It is safe to raise this up to 1/4 of total system memory;
assuming this is a dedicated database server.
QUERY CACHE
Query cache is enabled
Current query_cache_size = 32 M
Current query_cache_used = 10 M
Current query_cache_limit = 1 M
Current Query cache Memory fill ratio = 32.65 %
Current query_cache_min_res_unit = 4 K
MySQL won't cache query results that are larger than query_cache_limit in size
SORT OPERATIONS
Current sort_buffer_size = 6 M
Current read_rnd_buffer_size = 16 M
Sort buffer seems to be fine
JOINS
Current join_buffer_size = 1.00 M
You have had 16 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.
Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.
OPEN FILES LIMIT
Current open_files_limit = 6000 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine
TABLE CACHE
Current table_open_cache = 1024 tables
Current table_definition_cache = 256 tables
You have a total of 226 tables
You have 836 open tables.
The table_cache value seems to be fine
TEMP TABLES
Current max_heap_table_size = 16 M
Current tmp_table_size = 128 M
Of 582392 temp tables, 0% were created on disk
Effective in-memory tmp_table_size is limited to max_heap_table_size.
Created disk tmp tables ratio seems fine
TABLE SCANS
Current read_buffer_size = 4 M
Current table scan ratio = 647 : 1
read_buffer_size seems to be fine
TABLE LOCKING
Current Lock Wait ratio = 1 : 56
You may benefit from selective use of InnoDB.
If you have long running SELECT's against MyISAM tables and perform
frequent updates consider setting 'low_priority_updates=1'
If you have a high concurrency of inserts on Dynamic row-length tables
consider setting 'concurrent_insert=2'.
