Intermittent Time Synchronization Errors

[adamious]

Hello ViciDial Experts,

We have installed a Vicidial server and it will periodically have Time Synchronization Errors.
Sometimes it will work perfectly for 2-3 weeks without an issue and other times it could happen twice a day.

Specifications of the server

Version - 2.2.0-235
Build - 100319-1708
Asterisk - 1.4.28
No Digium/Sangoma Hardware
No Extra Software After Installation
OS: Debian 4.1.1-21

Upon a reboot it will fix the issue, but would prefer a permanent resolution.
We have tried suggestions on the forums such as with firewall rules, restarting apache2, even tried disabling NTP on the vicidial server but to no avail.

Its quite difficult for us to troubleshoot as it can be fine for weeks before then suddenly this happens again.
Is there anything else that can be suggested? or perhaps anything that could be enabled on the server to help diagnose the problem?

Thanks in advance

[williamconley]

excellent post (almost all the server specs:) )

please add: installation method (link to instructions?); and cluster information

1) noone has (to my knowledge) suggested DISABLING ntp on the server.

2) if you have a cluster or have even moved mysql from the base system to a sep server, you MUST time sync the servers together. the recommended method is to actually use ntp to sync them to a master (not to the world in general, but to allow them to sync with iburst to one specific server permanently, which may actually be a Vicidial server). this is better than issuing timed "sync now" using cron.

3) it is also recommended to sync the agents to the server (yes, the NTP service built into a standard installation will provide NTP service to workstations)

4) what is the server load when this happens?

[adamious]

Thank you for the prompt reply, its appreciated

To answer the missing information, the installation method used was (seems im not allowed to post links unless i have been here for 5 days)

Cluster information: The SQL server is running on the vicidial server itself 2 servers are not being used.

In relation to point 4, we have noticed when this occurs the CPU usage goes up quite considerably. We believe it may be an issue with the asterisk version. We are currently running on the server (1.4.28 )
At this point we are looking at either downgrading to version 1.4.27 or upgrading to 1.4.30. Is there any particular version you would recommend?

Thanks again

[adamious]

The installation Method is named Ubuntu VICIDIAL Install: 2009-12-02 from the vicidial Wiki

[williamconley]

Thank you for the prompt reply, its appreciated

To answer the missing information, the installation method used was (seems im not allowed to post links unless i have been here for 5 days)

Cluster information: The SQL server is running on the vicidial server itself 2 servers are not being used.

In relation to point 4, we have noticed when this occurs the CPU usage goes up quite considerably. We believe it may be an issue with the asterisk version. We are currently running on the server (1.4.28 )
At this point we are looking at either downgrading to version 1.4.27 or upgrading to 1.4.30. Is there any particular version you would recommend?

Thanks again

if you will be making a living with this software, and you're the technician posting on this forum for free help, i would strongly suggest (not just because it's a requirement, but because it may cause you to get more SLEEP between now and Christmas) that you read all the Stickies on all the forum categories on this site (what are there now, seven?).

Here's one of them you may be interested in:
http://www.vicidial.org/VICIDIALforum/v ... hp?t=12501

[dspaan]

Hello ViciDial Experts,

We have installed a Vicidial server and it will periodically have Time Synchronization Errors.
Sometimes it will work perfectly for 2-3 weeks without an issue and other times it could happen twice a day.

Upon a reboot it will fix the issue, but would prefer a permanent resolution.

Its quite difficult for us to troubleshoot as it can be fine for weeks before then suddenly this happens again.
Is there anything else that can be suggested? or perhaps anything that could be enabled on the server to help diagnose the problem?

Thanks in advance

We have the same problem but we are using ViciBox. Restarting helps but my main question is what we can do to analyze this problem so a permanent fix can be found.

[williamconley]

there's already a permanent fix: ntp time sync between all servers + reliable internet connection between agents and all servers. When you achieve this, you will no longer have this issue. We have had many rooms with this complaint, and so far this has always resolved the issue (no changes to Vicidial itself, but to the time sync and interconnecting network reliability).

this assumes, of course, a STOCK vicibox install

start here ... on each server:

Code: Select all

ntpq -p

[boybawang]

try putting this in your cronjob ntpdate -u 0.pool.ntp.org , I have experienced this in one of my clients before, the problem was their servers time was being reset to hardware time

[williamconley]

but beware because ntpdate does not "sync", it merely "sets". so the servers will not move forward at the same time. under heavy load, one may skew in a different direction and the sync may fail.

if you actually use ntp, however, they will alter their timing to match whichever is the master server and even under heavy load they do not fail.

[dspaan]

Having the same problem again right now!

vicibox:~ # ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*time.nist.gov .ACTS. 1 u 153 1024 377 145.133 -0.446 1.971
+ntp.yoinks.net 130.207.244.240 2 u 16 1024 377 131.154 3.722 2.211
clock.trit.net 192.43.244.18 2 u 645 1024 377 173.683 3.916 0.800
+sulfur.mednor.n 207.200.81.113 2 u 19 1024 375 164.284 4.067 1.815
-a.ntp.br 200.160.7.186 2 u 144 1024 377 210.081 4.275 0.377
LOCAL(0) .LOCL. 10 l 23 64 377 0.000 0.000 0.001

Any suggestions?

[williamconley]

if you have a single server setup and you're getting that message, you likely have connectivity issues with your agents to your server. dropping packets.

turn off ALL nonessential internet usage (ie: if it ain't vicidial, it should not be running)

[dspaan]

I do have one other asterisk machine that the ViciDial server uses to pass calls through. Could that server cause the issue as well? I could not do the ntpq -p on that machine, it is Asterisk 1.6.

[williamconley]

that should not be related as long as it's not dropping calls. the problem is between your agents and the vicidial server. are they Local to the vicidial server?

[dspaan]

Yes, all agents are in the LAN. Next time it happens i will turn off all work stations and turn them on one by one to see which station is causing this.

[williamconley]

Start by closing all outbound ports on your router except for SIP

If that fixes it, you can search for the culprit. Remember that Limewire and other Peer to Peer software can also query the heck out of your local network and screw it up. So you may have to do both

[soxito]

I solve the server sync issue.

log on the admin site

goto : admin > server > your server > then change the Telnet Host from localhost to your server IP address