[HELP] Another Time Synchronization Problem - 4Days Stuck!

[darkdemin]

Hi Guys,

I need your help.. I've been stuck for 4 days straight now in my "There is a time sychronization problem.." issue. Conducted multiple Google searches and browsed all related topics here but none seems to work on my end.



Your feedback will be highly appreciated. Thanks in advance!

[carpenox]

OK lets start with a little more info. What have you tried already?

[darkdemin]

OK lets start with a little more info. What have you tried already?

I've tried both the vicibox-timezone and yast timezone commands and it perfectly sync with the configured timezone. After a while, after the boot up of vicibox-install, I've noticed that both DB and PHP time are synched but not the Server Time (Telephony?) as seen on my previously attached image.

Is there any way to manully re-sync this? Sorry, I am a bit newbie on Linux CLI commands.

[ed123]

What is the output if you execute screen -ls?

[salartelcast]

Did you used vicibox.iso or you installed it from scratch ?

[carpenox]

ok from what you have reported, i am willing to bet your listen and send sockets are not running correctly. Run these commands at linux CLI:

screen -ls

[darkdemin]

Hi Guys,

Appreciate your responses! Here is my screen -ls output:

vicidial:~ # screen -ls
There are screens on:
153359.ASTVDauto (Detached)
120948.ASTVDadapt (Detached)
58986.ASTemail (Detached)
2253.ASTVDadFILL (Detached)
2250.ASTfastlog (Detached)
5 Sockets in /var/run/screens/S-root.
vicidial:~ #

[carpenox]

Ok so yea, your listen and send aren't running. Your manager.conf file probably corrupt.

[salartelcast]

Your linux time will not be set I guess,

Run this code on terminal

Code: Select all

rm /etc/localtime

ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime

rdate -s nist1-ny.ustiming.org


Set time zone according to your php.ini and my.cnf

[darkdemin]

Ok so yea, your listen and send aren't running. Your manager.conf file probably corrupt.

hi carpenox, here is my manager.conf output. Can you tell me how to fix it please?

[general]
enabled = yes
port = 5038
bindaddr = 0.0.0.0
timestampevents = yes

[cron]
secret = 1234
read = system,call,log,verbose,command,agent,user,originate
write = system,call,log,verbose,command,agent,user,originate

[updatecron]
secret = 1234
read = command,reporting
write = command,reporting

eventfilter=Event: CoreShowChannel


[listencron]
secret = 1234
read = system,call,log,verbose,command,agent,user,dtmf
write = command

eventfilter=Event: Shutdown
eventfilter=Event: DTMFBegin
eventfilter=Event: DTMFEnd
eventfilter=Event: NewCallerid
eventfilter=Event: Newstate
eventfilter=Event: Hangup
eventfilter=!Event: HangupRequest


[sendcron]
secret = 1234
read = command
write = system,call,log,verbose,command,agent,user,originate

[sipcron]
secret = 1234
read = call
write = command

eventfilter=Event: SIPInvite
eventfilter=Event: SIPResponse

[darkdemin]

Your linux time will not be set I guess,

Run this code on terminal

Code: Select all

rm /etc/localtime

ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime

rdate -s nist1-ny.ustiming.org


Set time zone according to your php.ini and my.cnf

Thanks salartelcast. Can you tell me where can I find my.cnf please?

[carpenox]

your manager.conf file looks fine. the time settings can be found at /etc/php7/apache/php.ini i believe and /etc/my.cnf

but the timezone has nothing to do with the your send and listen sockets not running. What happens when you run /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debug

[darkdemin]

your manager.conf file looks fine. the time settings can be found at /etc/php7/apache/php.ini i believe and /etc/my.cnf

but the timezone has nothing to do with the your send and listen sockets not running. What happens when you run /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debug

Here is the output:

vicidialer:~ # /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debug
AST_update set to keepalive
AST_send_listen set to keepalive
AST_VDauto_dial set to keepalive
AST_VDremote_agents set to keepalive
FastAGI_log set to keepalive
ip_relay set to keepalive
asterisk RUNNING: | 1278 /usr/sbin/asterisk -fn|
ip_relay RUNNING: | 2229 ip_relay 40569 127.0.0.1 4569 9999999|
ip_relay RUNNING: | 2230 ip_relay 41569 127.0.0.1 4569 9999999|
ip_relay RUNNING: | 2231 ip_relay 42569 127.0.0.1 4569 9999999|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
AST_VDauto_dial RUNNING: |/usr/share/astguiclient/AST_VDauto_dial.pl|
double check that processes are not running...
asterisk RUNNING: | 1278 /usr/sbin/asterisk -fn|
ip_relay RUNNING: | 2229 ip_relay 40569 127.0.0.1 4569 9999999|
ip_relay RUNNING: | 2230 ip_relay 41569 127.0.0.1 4569 9999999|
ip_relay RUNNING: | 2231 ip_relay 42569 127.0.0.1 4569 9999999|
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
FastAGI_log RUNNING: |/usr/share/astguiclient/FastAGI_log.pl --debug |
AST_VDauto_dial RUNNING: |/usr/share/astguiclient/AST_VDauto_dial.pl|
starting AST_update...
starting AST_manager_send...
starting AST_manager_listen...
starting AST_VDremote_agents...
|SELECT count(*) from system_settings where timeclock_end_of_day LIKE "%1220%";|
voicemail server configuration: 0|N
old live scheduled callbacks move active campaigns: 0
CID Groups with Auto-Rotate enabled: 0
DONE
vicidialer:~ #

[mugogo]

pls send here the output of this debug

perl /usr/share/astguiclient/AST_manager_listenBUFFER.pl --debugX

[ed123]

what is the result if you execute this command?

lsmod | grep dahdi

[williamconley]

1) Welcome to the Party!

2) As you are obviously new here, I have some suggestions to help us all help you:

When you post, please post your entire configuration including (but not limited to) your installation method (7.X.X?) and vicidial version with build (VERSION: 2.X-XXXx ... BUILD: #####-####).

This IS a requirement for posting along with reading the stickies (at the top of each forum) and the manager's manual (available on EFLO.net, both free and paid versions)

You should also post: Asterisk version, telephony hardware (model number is helpful here), cluster information if you have one, and whether any other software is installed in the box. If your installation method is "manual/from scratch" you must post your operating system with version (and the .iso version from which you installed your original operating system) plus a link to the installation instructions you used. If your installation is "Hosted" list the site name of the host.

If this is a "Cloud" or "Virtual" server, please note the technology involved along with the version of that techology (ie: VMware Server Version 2.0.2). If it is not, merely stating the Motherboard model # and CPU would be helpful.

Similar to This:

Vicibox X.X from .iso | Vicidial X.X.X-XXX Build XXXXXX-XXXX | Asterisk X.X.X | Single Server | No Digium/Sangoma Hardware | No Extra Software After Installation | Intel DG35EC | Core2Quad Q6600

3) And now to our issue ASSUMING: Single Server Vicibox X.X.X installed system with no special hardware ...

Hi Guys,

Appreciate your responses! Here is my screen -ls output:

vicidial:~ # screen -ls
There are screens on:
153359.ASTVDauto (Detached)
120948.ASTVDadapt (Detached)
58986.ASTemail (Detached)
2253.ASTVDadFILL (Detached)
2250.ASTfastlog (Detached)
5 Sockets in /var/run/screens/S-root.
vicidial:~ #

Ok so yea, your listen and send aren't running. Your manager.conf file probably corrupt.

Really? lol

There is no asterisk screen. Finding out why there is no asterisk screen seems like a much more likely method rather than guessing about a "corrupt" file (IMHO)

Where'd you get that? You have a lot of corrupt manager.conf files with your clients? That's ... odd.

Asterisk crashing is fairly common, ... but ordinarily, if Asterisk itself crashes, the screen keeps running. So something is truly odd about this fail.

Let's try this: Set the server (under admin->servers) to "Auto-Restart Asterisk = Y" if it is not already. Also verify "Active Asterisk Server = Y" and "Active = Y".

Also verify /etc/astguiclient.conf value for "VARactive_keepalives => 123456789E" and the VARserver_ip is correct (should be the IP of this server), and the VARDB_server is "localhost"

Then run

Code: Select all

perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

And then check "screen -ls" again and see if the asterisk screen is at least present.

Has this server ever run correctly? Is it a fresh "first" install or a machine that was running for a few weeks and suddenly failed?

Is the HD full!?! (df -h) Are there corrupt tables from a power outage?

Code: Select all

VARDB_pass=`awk '/VARDB_pass/{print $NF}' /etc/astguiclient.conf`; mysql asterisk -ucron -p$VARDB_pass -e "show table status where comment like '%crashed%'"

[darkdemin]

pls send here the output of this debug

perl /usr/share/astguiclient/AST_manager_listenBUFFER.pl --debugX

Hi, below is the output:

vicidialer:~ # perl /usr/share/astguiclient/AST_manager_listenBUFFER.pl --debugX

----- DEBUGGING ENABLED -----


----- SUPER-DUPER DEBUGGING -----

pattern match timed-out at /usr/share/astguiclient/AST_manager_listenBUFFER.pl line 236

vicidialer:~ #

[darkdemin]

1) Welcome to the Party!

2) As you are obviously new here, I have some suggestions to help us all help you:

When you post, please post your entire configuration including (but not limited to) your installation method (7.X.X?) and vicidial version with build (VERSION: 2.X-XXXx ... BUILD: #####-####).

This IS a requirement for posting along with reading the stickies (at the top of each forum) and the manager's manual (available on EFLO.net, both free and paid versions)

You should also post: Asterisk version, telephony hardware (model number is helpful here), cluster information if you have one, and whether any other software is installed in the box. If your installation method is "manual/from scratch" you must post your operating system with version (and the .iso version from which you installed your original operating system) plus a link to the installation instructions you used. If your installation is "Hosted" list the site name of the host.

If this is a "Cloud" or "Virtual" server, please note the technology involved along with the version of that techology (ie: VMware Server Version 2.0.2). If it is not, merely stating the Motherboard model # and CPU would be helpful.

Similar to This:

Vicibox X.X from .iso | Vicidial X.X.X-XXX Build XXXXXX-XXXX | Asterisk X.X.X | Single Server | No Digium/Sangoma Hardware | No Extra Software After Installation | Intel DG35EC | Core2Quad Q6600

3) And now to our issue ASSUMING: Single Server Vicibox X.X.X installed system with no special hardware ...

Hi Guys,

Appreciate your responses! Here is my screen -ls output:

vicidial:~ # screen -ls
There are screens on:
153359.ASTVDauto (Detached)
120948.ASTVDadapt (Detached)
58986.ASTemail (Detached)
2253.ASTVDadFILL (Detached)
2250.ASTfastlog (Detached)
5 Sockets in /var/run/screens/S-root.
vicidial:~ #

Ok so yea, your listen and send aren't running. Your manager.conf file probably corrupt.

Really? lol

There is no asterisk screen. Finding out why there is no asterisk screen seems like a much more likely method rather than guessing about a "corrupt" file (IMHO)

Where'd you get that? You have a lot of corrupt manager.conf files with your clients? That's ... odd.

Asterisk crashing is fairly common, ... but ordinarily, if Asterisk itself crashes, the screen keeps running. So something is truly odd about this fail.

Let's try this: Set the server (under admin->servers) to "Auto-Restart Asterisk = Y" if it is not already. Also verify "Active Asterisk Server = Y" and "Active = Y".

Also verify /etc/astguiclient.conf value for "VARactive_keepalives => 123456789E" and the VARserver_ip is correct (should be the IP of this server), and the VARDB_server is "localhost"

Then run

Code: Select all

perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

And then check "screen -ls" again and see if the asterisk screen is at least present.

Has this server ever run correctly? Is it a fresh "first" install or a machine that was running for a few weeks and suddenly failed?

Is the HD full!?! (df -h) Are there corrupt tables from a power outage?

Code: Select all

VARDB_pass=`awk '/VARDB_pass/{print $NF}' /etc/astguiclient.conf`; mysql asterisk -ucron -p$VARDB_pass -e "show table status where comment like '%crashed%'"

Thanks for the reminder friend. Anyways, here is the output of the screen -ls after I set "Auto-Restart Asterisk = Y"

vicidialer:~ # screen -ls
There are screens on:
2219.ASTfastlog (Detached)
2217.ASTVDremote (Detached)
2215.ASTVDauto (Detached)
2213.ASTlisten (Detached)
2211.ASTsend (Detached)
2209.ASTupdate (Detached)
6 Sockets in /var/run/screens/S-root.

vicidialer:~ # screen -ls
There is a screen on:
2219.ASTfastlog (Detached)
1 Socket in /var/run/screens/S-root.

vicidialer:~ #

Apparently, ASTlisten, ASTsend and ASTupdate was lost in screen -ls command after 3-5 seconds. Please help

[ed123]

what is the result if you execute this command?

lsmod | grep dahdi

[darkdemin]

what is the result if you execute this command?

lsmod | grep dahdi

Hi @ed123, no output for 'lsmod | grep dahdi"

[williamconley]

If the asterisk screen is not running, you need to find out why.

Code: Select all

/usr/sbin/asterisk -vvvvvvvvvvvvvvvvvvvvvgcT

You can try to start asterisk manually (temporarily) outside the screen. But your best bet is to run the keepalive script in debugX mode and locate the asterisk startup command which *should* be run to try to start asterisk since it's not running. There are criteria specific to your installation that the keepalive script will take into account, and then it will generate a start command and execute it ... your job is to locate that start command, execute it, and find out why it fails ... if you can't find out just by running "/usr/sbin/asterisk -vvvvvvvvvvvvvvvvvvvvvgcT" and getting some sort of error message.

Remember that if you *do* get it to run, and decide to try leaving it running ... kill it and start it in a screen named "asterisk" so Vicidial can find it.

[ed123]

Try reinstall your dahdi.

[ed123]

https://striker24x7.blogspot.com/2013/0 ... dahdi.html

[williamconley]

On the other hand, if this server has NEVER run correctly, a full reinstall may be in order. If one thing is wrong ... usually there are many other things as well. Never fear the re-install.

[mugogo]

Similar to This:

Vicibox X.X from .iso | Vicidial X.X.X-XXX Build XXXXXX-XXXX | Asterisk X.X.X | Single Server | No Digium/Sangoma Hardware | No Extra Software After Installation | Intel DG35EC | Core2Quad Q6600

3) And now to our issue ASSUMING: Single Server Vicibox X.X.X installed system with no special hardware ...

Hi Guys,

Appreciate your responses! Here is my screen -ls output:

vicidial:~ # screen -ls
There are screens on:
153359.ASTVDauto (Detached)
120948.ASTVDadapt (Detached)
58986.ASTemail (Detached)
2253.ASTVDadFILL (Detached)
2250.ASTfastlog (Detached)
5 Sockets in /var/run/screens/S-root.
vicidial:~ #

Ok so yea, your listen and send aren't running. Your manager.conf file probably corrupt.

Really? lol

There is no asterisk screen. Finding out why there is no asterisk screen seems like a much more likely method rather than guessing about a "corrupt" file (IMHO)

Where'd you get that? You have a lot of corrupt manager.conf files with your clients? That's ... odd.

Asterisk crashing is fairly common, ... but ordinarily, if Asterisk itself crashes, the screen keeps running. So something is truly odd about this fail.

Let's try this: Set the server (under admin->servers) to "Auto-Restart Asterisk = Y" if it is not already. Also verify "Active Asterisk Server = Y" and "Active = Y".

Also verify /etc/astguiclient.conf value for "VARactive_keepalives => 123456789E" and the VARserver_ip is correct (should be the IP of this server), and the VARDB_server is "localhost"

Then run

Code: Select all

perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

And then check "screen -ls" again and see if the asterisk screen is at least present.

Has this server ever run correctly? Is it a fresh "first" install or a machine that was running for a few weeks and suddenly failed?

Is the HD full!?! (df -h) Are there corrupt tables from a power outage?

Code: Select all

VARDB_pass=`awk '/VARDB_pass/{print $NF}' /etc/astguiclient.conf`; mysql asterisk -ucron -p$VARDB_pass -e "show table status where comment like '%crashed%'"

Thanks for the reminder friend. Anyways, here is the output of the screen -ls after I set "Auto-Restart Asterisk = Y"

vicidialer:~ # screen -ls
There are screens on:
2219.ASTfastlog (Detached)
2217.ASTVDremote (Detached)
2215.ASTVDauto (Detached)
2213.ASTlisten (Detached)
2211.ASTsend (Detached)
2209.ASTupdate (Detached)
6 Sockets in /var/run/screens/S-root.

vicidialer:~ # screen -ls
There is a screen on:
2219.ASTfastlog (Detached)
1 Socket in /var/run/screens/S-root.

vicidialer:~ #

Apparently, ASTlisten, ASTsend and ASTupdate was lost in screen -ls command after 3-5 seconds. Please help

I also agree on a Full reinstall of the server advise given by William but the ASTlisten, ASTsend and ASTupdate, i had the same issue and this sorted me out https://itgala.xyz/vicidial-there-is-a-time-synchronization-problem-with-your-system-please-tell-your-system-administrator/ .

This might be the same reason i believe you are getting pattern match timed-out at line 236 on

Hi, below is the output:

vicidialer:~ # perl /usr/share/astguiclient/AST_manager_listenBUFFER.pl --debugX

----- DEBUGGING ENABLED -----


----- SUPER-DUPER DEBUGGING -----

pattern match timed-out at /usr/share/astguiclient/AST_manager_listenBUFFER.pl line 236

vicidialer:~ #

If above works pls run

Code: Select all

perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

an post the output of screen -ls

[carpenox]

pattern match timed-out is usually due to a wrong password. My suggestion is to delete your manager.conf file and remake it with this one:

Code: Select all

[general]
enabled = yes
port = 5038
bindaddr = 0.0.0.0
timestampevents = yes

[cron]
secret = 1234
read = system,call,log,verbose,command,agent,user,originate
write = system,call,log,verbose,command,agent,user,originate

[updatecron]
secret = 1234
read = command,reporting
write = command,reporting

eventfilter=Event: CoreShowChannel


[listencron]
secret = 1234
read = system,call,log,verbose,command,agent,user,dtmf
write = command

eventfilter=Event: Shutdown
eventfilter=Event: DTMFBegin
eventfilter=Event: DTMFEnd
eventfilter=Event: NewCallerid
eventfilter=Event: Newstate
eventfilter=Event: Hangup
eventfilter=!Event: HangupRequest


[sendcron]
secret = 1234
read = command
write = system,call,log,verbose,command,agent,user,originate

[sipcron]
secret = 1234
read = call
write = command

eventfilter=Event: SIPInvite
eventfilter=Event: SIPResponse

I know it looks the same as what you posted, but try it

[darkdemin]

Thanks for the reminder friend. Anyways, here is the output of the screen -ls after I set "Auto-Restart Asterisk = Y"

vicidialer:~ # screen -ls
There are screens on:
2219.ASTfastlog (Detached)
2217.ASTVDremote (Detached)
2215.ASTVDauto (Detached)
2213.ASTlisten (Detached)
2211.ASTsend (Detached)
2209.ASTupdate (Detached)
6 Sockets in /var/run/screens/S-root.

vicidialer:~ # screen -ls
There is a screen on:
2219.ASTfastlog (Detached)
1 Socket in /var/run/screens/S-root.

vicidialer:~ #

Apparently, ASTlisten, ASTsend and ASTupdate was lost in screen -ls command after 3-5 seconds. Please help

I also agree on a Full reinstall of the server advise given by William but the ASTlisten, ASTsend and ASTupdate, i had the same issue and this sorted me out https://itgala.xyz/vicidial-there-is-a-time-synchronization-problem-with-your-system-please-tell-your-system-administrator/ .

This might be the same reason i believe you are getting pattern match timed-out at line 236 on

Hi, below is the output:

vicidialer:~ # perl /usr/share/astguiclient/AST_manager_listenBUFFER.pl --debugX

----- DEBUGGING ENABLED -----


----- SUPER-DUPER DEBUGGING -----

pattern match timed-out at /usr/share/astguiclient/AST_manager_listenBUFFER.pl line 236

vicidialer:~ #

If above works pls run

Code: Select all

perl /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

an post the output of screen -ls

Finally!! Thank to you my friend, I am now able to move forward. Works like a charm!
Thank you all guys for the help. Appreciate it.

[carpenox]

What exactly did you do to fix it incase others have the same problem? Can you list the steps you took

[darkdemin]

What exactly did you do to fix it in case others have the same problem? Can you list the steps you took

1.) Fresh cluster installation - Web/DB/Telephony/Archive
2.) Follow this guide: https://itgala.xyz/vicidial-there-is-a- ... nistrator/
3.) Reboot DB server first then reboot other servers in any order

[carpenox]

Good postback, thank you

[mugogo]

Glad it worked out for you... enjoy.