Callcentre System Design

[Arffeh]

Hi all, longtime forum lurker here.

After finding out I can buy an enormous amount of the following server spec:

Code: Select all

IBM Server xSeries X3850
4x Intel Xeon Dual Core CPUs 3.0Ghz
8GB RAM
2x 73GB SAS

(10 or more), as well as my boss explaining we might be getting another (huge) office, I figure I'll leave my current drafted idea open to critique from the experts.

All servers are the above spec, vicibox 5.0.3 (Mainly because I'm thrilled that MariaDB is implemented now)

Server 1: MariaDB Dedicated Master
Server 2: MariaDB Dedicated Slave
Server 3: Webserver Dedicated
Server 4: Asterisk Dedicated, 4:1 Outbound
Server 5: Asterisk Dedicated, 4:1 Outbound

Keep in mind we don't have to handle call recordings, our carrier gives us dedicated hardware to bridge to their termination server, recording whatever goes outbound from our offices, taking a huge load from the Asterisk servers

I also want to say thanks to Will and Matt for being godly in both their work and community roles. I dare say a lot of people would be pretty screwed without those two.

[williamconley]

you're going to want 4 cores (or 8!!) on your DB server unless you have no intention of growing. replacing the DB server after it's been running for a while can be easy, but scary nonetheless. Especially if you have a replication system set up and have links to worry about. Also beware of RAID in SAS configurations, they often argue during installation and mirroring can stress the DB server (and with only two cores, that could be fatal).

[Arffeh]

These systems all have 4 physical processors, 2 cores each, making 8 total.
I was thinking the DB master might need to be a bit beefier all-round though, just so later on I could slap in an extra couple of asterisk servers later if needed.

[geoff3dmg]

Best way to speed the DB up is to use SSDs.

[mflorell]

The BEST way to speed up a DB is more RAM, more RAM, more RAM. Second best would be SSDs on a RAID10.

[glenewhittenberg]

I have a client i built a single server for and used 4x 250GB SSD drives in raid10 and it is hella fast. The specs below in my signature is a for another server I run and did not use SSD in because of space needed and budget. Fast machine for sure, but not SSD fast!!!

[Arffeh]

glenewhittenberg, with that single server, how many agents have you managed on it?

[glenewhittenberg]

The one with the SSD drives it normally has about 20 agents on it, sometimes 25, and is barley being used (disk io, cpu, etc.). They do no call recording on this and 100% outbound at a ratio of 4:1. avg wait times are under 20 sec with less than 2% drop rate. This server is not an actual server class but has some nice pieces in it.

ViciBox Redux v.5.0.2-130821
Asterisk 1.8.25.0-vici
Version: 2.8b0.5
SVN Version: 2052
Asus mb
Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz (4 core/ 8 thread)
2x 8GB 1600 Patriot RAM
4x 250GB Samsung SSD RAID10

The non SSD server in signature has only 10 agents on it now. This is new install. They have another 10 starting in the next few days. We are hoping to get 30 agents on this single server. Full call recording. inbound and outbound campaigns. This a SuperMicro 1U server the 5710R-MTF i think.

Hope i answered your question. thanks!

[williamconley]

Note that RAM can be added with a quick reboot. SSD can be added as well, with a bit of "move the data files" magic. But the number of processors and the speed of those processors is not generally so easy to change "later". So ... beef it up with cpu count and speed during purchase and pay attention to the pricing of SSD and MegaRAID and RAM for the model you've chosen for later addition (growth).

[Arffeh]

Pretty neat Will.

If during my adventures I do a proper writeup on scaling vicidial systems, would it be worth handing over to you later on down the line for others?

My thought is you probably get system sizing/scaling questions all the time, so maybe even writing out a chart of features vs agents might be worthwhile, to make up a base of what's recommended.

For example, I'm currently running 25-35 agents on up to a 6:1 ratio on a single server setup, but hit a huge problem the moment any additional agents join the pool. Having a standard document to approach the problem with would be pretty handy as far as required hardware goes.

Just a thought.

[williamconley]

Better idea: Don't send it to me or anyone else. Post it HERE. It's not a secret, it'll help others who may have interest in your experience. And do remember that it'll still just be your experience, not a rule set. Vicidial expansion is personal to the call center in each instance. Differences can be subtle, and requirements (especially those of Budget) vary Widely.

But mostly: every installation has its own idiosyncrasies and minor modifications that can swing the experience in a totally unexpected direction. Sometimes this is due to errors that are never found, sometimes it is simply a requirement due to a business model (sometimes both).

Having a discussion on the topic, with details of expansion, never hurts. Especially if the discussion is open for all to benefit. Ask anyone, you'll find that *I* want as many people to attend this party as possible. That's why I provide free support on this board instead of on another site (ahem "goautodial" LOL) which would fragment the support available to those using the system.

Here, there are actually lots of helpers who actually know what they are doing all condensed in one place. And many of them are actually new, but they will still Chip in and help the next guy. Open Source. (Hi Striker! Keep it up!)

[glenewhittenberg]

Just to share some number with every one. The server below in my signature (64k Chunks setup on RAID via Intel RST) gets these results

Code: Select all

vicidialer:~ # dd bs=1M count=2560 if=/dev/zero of=test conv=fdatasync
2560+0 records in
2560+0 records out
2684354560 bytes (2.7 GB) copied, 12.2163 s, 220 MB/s
vicidialer:~ #


While another server using a i7 CPU, 16GB 1600 non ECC Ram, 4x 250GB Samsung SSD, with 64k Chunks setup on RAID via Intel RST gets

Code: Select all

vici5:~ # dd bs=1M count=2560 if=/dev/zero of=test conv=fdatasync
2560+0 records in
2560+0 records out
2684354560 bytes (2.7 GB) copied, 4.27317 s, 628 MB/s
vici5:~ #


Mucho difference

[williamconley]

Now: What about random reads and writes instead of pure constant write? (Vicidial is not all "append only logs")

[Arffeh]

Holy mother of god, SSD's never cease to amaze me.

Would it be worth setting up a 2TB HDD as the mount point for the recordings, so the SSD's remain more available for the DB/OS, as well as the bonus of not wearing out the read/write cycles of the SSD's as quickly?

Or would that just be a waste of time?

[williamconley]

Absolutely, but put it on another physical machine and put it in a mirror (multi T drives DIE ...). This will remove Traffic (CPU and network) from the DB server and allow drive death without human rebuild time (drop in a new mirror drive and hit the rebuild button .. and walk away).

When you have 1.2 TB of audio files to recover after a drive death, waiting for that rebuild can be a killer ... unless it's on a machine unrelated to the cluster, just a generic web server, in which case you really don't care.