asterisk stopped working

[dreedy]

Hello everyone,

We have 3 dialer and 1 web server. Out of the blue yesterday dialer 2 asterisk started to just stop. this would cause the agent screen for anyone using this dialer to have a time synchronize error. so i shifted everyone on dialer 2 to dialer 1 and dialer 3. both dialer 1 and 3 had no issues yesterday. Today at 1 pm dialer 3 asterisk stopped and restarted as i had the option for asterisk to restart. after it did it the first time i disabled the option to restart asterisk. at 4:40 today dialer 1 asterisk stopped and i tried looking through the screenlog.0 to see if i could see anything off. nothing really looked off to me so i did a service vicidial restart and it came up fine and started working agian.


Does anyone have any suggestions how to diagnose this? is there some log i am missing to see errors?

[williamconley]

Code: Select all

screen -r asterisk

this will show you what the asterisk crash provided (if anything). it is possible to have deep troubleshooting with a crashed asterisk, but not fun and not for those unfamiliar with core dumps.

however: asterisk has been known to be susceptible to attack, so a whitelisting firewall system to avoid asshats hitting your system (and causing asterisk to crash) is one "just in case" method. Another is to turn off unneeded and annoying modules, such as sqlite.

/etc/asterisk/modules.conf

Code: Select all

noload => res_config_sqlite.so
noload => res_config_sqlite3.so
noload => cdr_sqlite.so
noload => cel_sqlite3_custom.so
noload => cdr_sqlite3_custom.so

noload => chan_ooh323.so

the asterisk logs themselves are in /var/log/asterisk

[dreedy]

Thanks William i will try that when if asterisk crashes again. Strange it has been working great for years and all of a sudden starts crashing. i will double check my firewall i am behind a meraki on my data side and a sbc on my voip side.

[dreedy]

I will disable those modules also anything to make it easier for asterisk. Hope i can catch this bugger. its been flawless finally after a lot of work. and the web phone is great.

[dreedy]

if my reading is correct to exit safely from screen it would be ctrl A and then ctrl D is this the best way to exit from screen?

[williamconley]

if my reading is correct to exit safely from screen it would be ctrl A and then ctrl D is this the best way to exit from screen?

Yep.

I will disable those modules also anything to make it easier for asterisk. Hope i can catch this bugger. its been flawless finally after a lot of work. and the web phone is great.

Years? With 8.1.2? You wanna say that again?

But yes: If you run for too long, the berkeley DB in use (sqlite) grows to such a level that asterisk crashes ... merely turning that feature off could resolve your problem entirely.

Although I still don't get how you have an 8.1.2 server that's been running for years.

[dreedy]

I was including reload. I just upgraded to 8.1.2 when it came out and had the viciphone. previously we were on 7.0.3 i believe and that was stable also.

[dreedy]

I checked all the modules that you have listed and they all match the format that you listed. That means the modules are not being used correct?

Code: Select all


    noload => res_config_sqlite.so
    noload => res_config_sqlite3.so
    noload => cdr_sqlite.so
    noload => cel_sqlite3_custom.so
    noload => cdr_sqlite3_custom.so

    noload => chan_ooh323.so


[williamconley]

I checked all the modules that you have listed and they all match the format that you listed. That means the modules are not being used correct?

Code: Select all


    noload => res_config_sqlite.so
    noload => res_config_sqlite3.so
    noload => cdr_sqlite.so
    noload => cel_sqlite3_custom.so
    noload => cdr_sqlite3_custom.so

    noload => chan_ooh323.so


"match the format"? Not sure what that means. If the question is "does noload for a module mean the module is not loaded and thus not in use?" the answer is yes.

I was including reload. I just upgraded to 8.1.2 when it came out and had the viciphone. previously we were on 7.0.3 i believe and that was stable also.

"reload"? As in delete everything on the server and reinstall a new version of Vicibox which deletes all information on the hard drive? Or did you perform some sort of "upgrade"? Very different scenarios. The sqlite3/berkeley database gets wiped away with a re-install on the dialer, but does not get wiped clean with an upgrade. But: you can't "upgrade" from 7.0.3 to 8.1.2, you have to re-install. ViciDIAL can be upgraded, but ViciBOX requires a "start over" fresh install. Vicidial is presently in Version 2.14 whereas Vicibox is in version 8.1.2.

[dreedy]

sorry for the lack of detail there. When vicidial 8.1.2 came out i wiped the web and the 3 dialers. They are now clean installs that have the 8.1.2. I have to schedule the re installation of the mysql server and that is 7.0.3. So that has stayed on the previous install.

I finally had the asterisk dump on my and the last thing is sayed was.

Segmentation fault (core dumped)

[williamconley]

i'm not sure i ever got an answer: is the system whitelisted? (ie: whitelist firewall in place?)

analyzing an astgerisk core dump after a segmentation fault is outside the realm of this forum. there are asterisk forums that may be able to assist. but most of the time the problem is resolved by whitelist and noloading all unused modules.

[dreedy]

This turned at to be an asterisk 13.24.0 issue. the reps were using the volume up and down on the channels in the agent screen. we had to hire vicidial and there great techs were able to localize this issue. vicidual had to exempt out the volume controls for the agent screen so that it would not cause the asterisk to crash.

--Update--
I currently have 3 dialers that are on 13.24.0 and 1 dialer that is on 13.27.1. the newer astersik seems to have no reported issue with the volume controls at this time. I am running test on that dialer to see if i can update the asterisk to the newest one and remove the exemptions. i will update further once i have completed testing.

[williamconley]

That is an excellent postback!

[garski]

How was it? Did it solved your issue? We encounter the same error. Did you remove the volume up and down of the agent screen? I did $volumecontrol_active = '0'; but there is a bug where it would drop the call for each working call receive on autocall dialing.

I tried increasing TaskMax to Infinity but having the same error.

[dreedy]

Yes the vicidial tech remoted in to our dialers and in the /etc/extensions.conf he commented out the volume controls for that part of the agent screen.

[dreedy]

;exten => _X48600XXX,1,MeetMeAdmin(${EXTEN:2},T,${EXTEN:0:1})
;exten => _X38600XXX,1,MeetMeAdmin(${EXTEN:2},t,${EXTEN:0:1})
;exten => _X28600XXX,1,MeetMeAdmin(${EXTEN:2},m,${EXTEN:0:1})
;exten => _X18600XXX,1,MeetMeAdmin(${EXTEN:2},M,${EXTEN:0:1})

[dreedy]

I have sent a ticket on this. it has been happening from some time. so let see if we can fix this. Here is the ticket number.


http://www.vicidial.org/VICIDIALmantis/view.php?id=1186

[snoel]

I made all the changes that were requested upward but the asterisk still stops every 10 minutes, even if I checked the autorestart its very disturbing. How can I resolve it? There are only 3 agents logged in. vicibox9:~ # uptime
11:24:29 up 0:43, 2 users, load average: 0.52, 0.68, 0.61

- ViciBox v.9.0.0 190913-1108///asterisk 13.27///Version: 2.14b0.5
SVN Version: 3149
DB Schema Version: 1577