Some questions about vicibox 9 clustering

[dspaan]

It's been a while since i've made a cluster and have been working on that today with Vicibox 9.0.1 and SVN 3204.

I have created a cluster with 1 database server, 1 slave server, 1 web server and 2 dialers.

Up till Vicibox 8 i was used to having to setup NTP on the cluster so all other servers would sync with the database server (page 12 of the Poundteam Multi server manual V1.1).
But on Vicibox 9 i don't even see /etc/ntp.conf anymore. The only way i can access NTP settings is by using yast > System > Date and time

I created a slave database server. I couldn't find any documentation about it. I found in the forum (thanks Bill) that you can use the System settings section to offload some reports to the Slave server. But what i would like to use it for mainly is a backup server placed in another datacenter so in the event of a calamity we could make some changes and continue working.

I noticed the crontab for the webserver and slave server is empty. I understand there are specific cronjobs for the database and some are for the dialers. But are there no cronjobs needed by default for the webserver? What about nightly rebooting?

Questions:

1) How do i setup cluster timesync up Vicibox 9?
2) What is the primairy purpose for a slave server and what i if i want to use this as a backup server for redundancy (or is this the wrong way to go?)
3) Should the webserver have no cronjobs by default?

[williamconley]

1) ntp is a system level service. They like to move config files around and even rename them. I bet if you look you'll find it in a subfolder named ntp or the .conf file named something close but not identical to ntp. Or both.

2) ONLY dialers require locally running cron jobs. The DB and web servers merely provide DB and web services. In fact, these two roles don't even need Vicidial installed at all. Just DB or web processes on any OS that matches the version required for the cluster (ie: you can't use a more recent version of apache than is compatible with Vicidial, that sort of thing).

3) It has been a long-standing practice to run all the "has to run on only ONE server" processes on the DB server. It's also been a long-standing practice to run all the DB query routines on the DB server. But that's not even close to being a requirement. It's also rare to run any processes at all in the crontab for a dedicated web server. We tend to install all roles in all servers, but when a server becomes a dedicated web, we'll remove all crontab entries as part of that process.

4) DB replication is not actually a Vicidial-specific function at all. However, the "slave/replication/report" server CAN be set up with a Vicidial install .iso anywhere in the world and will replicate accordingly. That in no way means you need to use the Vicibox installer. You can just install an identical version of MySQL in any server on the web and configure normal replication with the replication settings recommended by the Vicibox installer and it'll work to save the DB. When the time comes to make that the LIVE db, that server need only be accessible LOCALLY to the rest of the servers in the cluster. Can't be in a different physical location if you expect any real load (ie: more than one agent). However, it could then be used to rebuild the local DB server via mysqldump from "up to the moment of death" data.

[dspaan]

1) No vicibox 9 clusters at poundteam yet? This is what i find when i look for ntp. I inspected a few files but couldn't find anything like the ntp.conf that was used in vicibox 8.

Code: Select all

/etc/netconfig.d/ntp-runtime
/usr/lib/firewalld/services/ntp.xml
/usr/lib/perl5/vendor_perl/5.26.1/LWP/Protocol/nntp.pm
/usr/lib/perl5/vendor_perl/5.26.1/URI/nntp.pm
/usr/lib/systemd/ntp-units.d
/usr/lib/systemd/ntp-units.d/50-chronyd.list
/usr/lib64/python2.7/nntplib.py
/usr/lib64/python2.7/nntplib.pyc
/usr/lib64/python2.7/nntplib.pyo
/usr/lib64/python2.7/ntpath.py
/usr/lib64/python2.7/ntpath.pyc
/usr/lib64/python2.7/ntpath.pyo
/usr/lib64/python3.6/nntplib.py
/usr/lib64/python3.6/ntpath.py
/usr/lib64/python3.6/__pycache__/nntplib.cpython-36.opt-1.pyc
/usr/lib64/python3.6/__pycache__/nntplib.cpython-36.opt-2.pyc
/usr/lib64/python3.6/__pycache__/nntplib.cpython-36.pyc
/usr/lib64/python3.6/__pycache__/ntpath.cpython-36.opt-1.pyc
/usr/lib64/python3.6/__pycache__/ntpath.cpython-36.opt-2.pyc
/usr/lib64/python3.6/__pycache__/ntpath.cpython-36.pyc
/usr/share/YaST2/clients/ntp-client.rb
/usr/share/YaST2/clients/ntp-client_auto.rb
/usr/share/YaST2/clients/ntp-client_finish.rb
/usr/share/YaST2/clients/ntp-client_proposal.rb
/usr/share/YaST2/data/ntp_servers.yml
/usr/share/YaST2/include/ntp-client
/usr/share/YaST2/include/ntp-client/commandline.rb
/usr/share/YaST2/include/ntp-client/dialogs.rb
/usr/share/YaST2/include/ntp-client/helps.rb
/usr/share/YaST2/include/ntp-client/misc.rb
/usr/share/YaST2/include/ntp-client/wizards.rb
/usr/share/YaST2/lib/y2ntp_client
/usr/share/YaST2/lib/y2ntp_client/client
/usr/share/YaST2/lib/y2ntp_client/dialog
/usr/share/YaST2/lib/y2ntp_client/widgets
/usr/share/YaST2/lib/y2ntp_client/client/auto.rb
/usr/share/YaST2/lib/y2ntp_client/client/finish.rb
/usr/share/YaST2/lib/y2ntp_client/dialog/add_pool.rb
/usr/share/YaST2/lib/y2ntp_client/dialog/main.rb
/usr/share/YaST2/lib/y2ntp_client/dialog/pool.rb
/usr/share/YaST2/lib/y2ntp_client/widgets/main_widgets.rb
/usr/share/YaST2/lib/y2ntp_client/widgets/pool_widgets.rb
/usr/share/YaST2/locale/af/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ar/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ast/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/bg/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/bn/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/bs/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ca/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/cs/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/cy/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/da/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/de/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/el/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/en_GB/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/es/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/et/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/fa/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/fi/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/fr/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/gl/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/gu/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/hi/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/hr/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/hu/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/id/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/it/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ja/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/jv/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ka/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/km/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/kn/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ko/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ku/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/lo/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/lt/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/mk/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/mr/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/nb/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/nds/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/nl/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/nn/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/pa/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/pl/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/pt/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/pt_BR/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ro/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ru/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/si/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/sk/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/sl/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/sr/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/sv/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/sw/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/ta/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/tg/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/th/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/tr/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/uk/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/vi/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/wa/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/xh/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/zh_CN/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/zh_TW/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/locale/zu/LC_MESSAGES/ntp-client.mo
/usr/share/YaST2/schema/autoyast/rnc/ntpclient.rnc
/usr/share/applications/YaST2/ntp-client.desktop
/usr/share/augeas/lenses/dist/ntp.aug
/usr/share/augeas/lenses/dist/ntpd.aug
/usr/share/bash-completion/completions/mountpoint
/usr/share/bash-completion/completions/ntpdate
/usr/share/icons/hicolor/scalable/apps/yast-ntp-client.svg
/usr/share/licenses/yast2-ntp-client
/usr/share/licenses/yast2-ntp-client/COPYING
/usr/share/nmap/scripts/http-frontpage-login.nse
/usr/share/nmap/scripts/nntp-ntlm-info.nse
/usr/share/nmap/scripts/ntp-info.nse
/usr/share/nmap/scripts/ntp-monlist.nse

2) Clear, thanks.

3) Alright so just run vicibox-express script on all servers and then tie them together by modifying the astguiclient file and crontjobs and adding servers and conferences (like in the poundteam manual)?

4) I understand, do you know what the main purpose was for adding this feature in the vicibox install script then? I'm just curious what this is most commonly used for. Can you recommend a good guide for setting up replication with vicibox? By this i mean the mysql setup steps but also the procedure how to change the role of the slave server into a master server. Based on your comments i'm thinking now to setup a slave server which also has the web role and making two 'sleeping' dialers in another datacenter so in case of calamity i have a complete backup cluster that i can bring to life.

[williamconley]

NEVER use express. use Standard.

There is NO reason to NOT use the Vicibox installer for replication. It's just not required. MySQL replication is covered ad-nauseum all over the web if you don't want to use the standard install script for vicibox. We generally don't set a vicidial server up as replication using the vicibox method. We just install web/dialer functions (so the server CAN be used if needed) and then activate mysql and replicate the DB using normal replication methods.

[dspaan]

Thanks, i will checkout the standard replication documentation. Any idea about the NTP setup for vicibox 9?

[williamconley]

modify the settings in yast. put in a google ntp server IP. Search for that IP in the /etc/ folder and all sub-folders beneath. (and report back ...)

[dspaan]

NEVER use express. use Standard.

Why?

Can you confirm that this is the most practical procedure:

1.Install all machines in the cluster with all role (so all servers have mysql, web and asterisk)
2.Configure timesync
3.Define keepalives in astguiclient.conf for each server
4.Define correct crontjobs for each server
5.Add servers from GUI
6.Add conferences
7.Setup phones and carriers



Alright so just run vicibox-express script on all servers and then tie them together by modifying the astguiclient file and crontjobs and adding servers and conferences (like in the poundteam manual)?

[williamconley]

No. Use standard install. You'll only have to configure NTP. The Q/A of the standard install replaces the manual we wrote a decade ago. Kumba put a LOT of work in to make that standard installer do everything you NEED for clustering. Technically, NTP on each server to "the web" works, but having one as master and the other as slaves works ... better. So NTP is the only thing you'd still want to set up differently than the standard installer.

Express is for newbies who are setting up a single server and have no idea what they are doing yet.

You do not fit that description.

There's a reason the manual hasn't been updated in a decade. It's not really needed any more (although, technically, it's still valid, lol)

[dspaan]

ok but i thought you meant to say install all roles on all servers so you can change their role later on, for instance add the telephony role to a webserver because one of the dialers has crashed or upgrade the slave webserver to master because the main db server has crashed or diconnected.

[williamconley]

I'd say that's fairly obvious. It doesn't hurt to install dialer and web on all servers. It can be a lifesaver, in fact, if a server goes down or even just overloads. In our case, it's often handy to have dialer available for development on a server with no agents (like ... the replication server) so we can test features without impacting agents on live campaigns.

Plus it's pretty simple to disable dialer functions if you do have enterprise requirements and need the Full Power of each dialer/web server to be dedicated to that role ... and yet you can reactivate those features pretty quickly in time of need. The trick is learning which table record to delete to stop a server from being "red" after it has been a dialer but is no longer one. lol

[dspaan]

But if you answer the Y/N questions from the vicibox-install method logically and for instance on a dialer answer the question "Will this server be used as the Database" with N, won't this make it more difficult later on to set this up as the database if you want to? Because only if you answer N to that question the wizard will ask me what the IP of the database server is and make the needed configurations. So that's why i figured using vicibox-express on each server and then manually connecting them. Or maybe it's not clear to me which features are or are not installed when choosing Y/N for any of the options.

Anyway, i figured out the NTP procedure for Vicibox 9. Here it is for anyone to enjoy.

On the database/NTP server:

Setup server to use chrony as ntp client to sync with internet timeservers and at the same time configure chrony as NTP server.

Code: Select all

nano /etc/chrony.conf

Uncomment #allow 192.168.0.0/16 and change to:

Code: Select all

allow all
service restart chronyd
systemctl enable chronyd

Open up the NTP service for the firewall trusted zone and add the IP's from the servers in the cluster to the trusted zone:

Code: Select all

firewall-cmd --permanent --zone=trusted --add-service=ntp
firewall-cmd --permanent --zone=trusted --add-source=x.x.x.x/32
firewall-cmd --permanent --zone=trusted --add-source=x.x.x.x/32
firewall-cmd --permanent --zone=trusted --add-source=x.x.x.x/32
firewall-cmd --permanent --zone=trusted --add-source=x.x.x.x/32
firewall-cmd --reload

Check if chrony is active:

Code: Select all

timedatectl

On each of the servers enable chrony and point to the NTP server:

Code: Select all

nano /etc/chrony.conf
Comment out pools and servers and add:
server ip.of.your.ntp iburst
service restart chronyd

Check if the client is syncing with the server:

Code: Select all

chronyc sources


Sources:
https://www.golinuxcloud.com/configure- ... orce-sync/
https://doc.opensuse.org/documentation/ ... a-ntp.html

[williamconley]

won't this make it more difficult later on to set this up as the database if you want to?

Um, no. Each cluster can have only ONE database server. Changing the DB later will ALWAYS involve the same steps. MySQL is going to be installed, but not active at startup. Plus, if you set the server up as replication, it will set up everything very well for DB and even load the DB in for you so you could (if necessary) turn this server into the live DB server with some fairly quick changes.

Warning: The MySQL version for a "live" and "backup" (via replication) server must be identical. If one or the other is more recent, the newer one MUST be the replication server as there is no way to reverse the roles and replicate from a newer master to an older slave. That would require a mysqldump and reload of the db rather than what some might like to do, which is switch over to the "new master" and reverse the roles completely. For that, they must be identical (or damn close). It does make sense, though. Since the slave is just doing as it's told and would be able to understand everything the master says (and the master is the controlling server). But if those roles are reversed, there will be newer commands the "newer' master would emit that the "older" slave simply wouldn't understand. Crash and burn.