A 300 Seats Live CallCentre for testing & Analy. of VICI

[voip.art]

Hello,
A 300 Seats Live call centre for testing and analysis of VICDIAL…!!!
I hope this is a good news for all of you in ViciDial, I have a centre ready for going ahead with Vicidial+Asterisk for there real production , with predictive dial + 100% Voice logging, It was hard for me to convince the operational guys who are use to with Industry leading Dialers.. but I am through with it , but I have a time line to achieve..
July 10-2008 – trial run
July 20-2008 – 50 agents on live
Aug-5 -2008 – 100 agents on live
Sep- 1 – 2008 – 200 agents on live
Dec-15-2008 – 300 agents on live

Let me tell you about me, I am working on Voice over IP for last about 10 years, worked as technical consultant for some of the industry leaders (who makes voip gateways /trunking gateways/IPPBX) in some part of the world. Asterisk was every time fascinated me but when you represent the commercial IPPBX you are not allowed to price the goodies of freeware and more over my know how on Linux is also poor.

Now I am my own, going to venture into a real life scenario for the freeware world, and intern feed back the information to you developers to make the product much stronger.
Even though I have gone through the information available on web site I am still not clear about the capacity , Multi server / Load balancing architecture, How many server ? what server ? why too many server ? what config ?? so on…

Is there any experienced helping hand out there !!!

[mflorell]

This sounds like a great project. We have done several projects for companies with 200+ seats, and as the creators of VICIDIAL we can figure out just about any problem that may come up.

We have done similar setups in very short timeframes with great results.

I would recommend that you contact us off-list so that we can ask you more about your requirements to give you a better idea of what kind of hardware you will need.

http://www.vicidial.com

[voip.art]

How do I contact you...\

[mflorell]

here's our main email:
sales (at) vicidial.com

[voip.art]

Hello All,
I am almost completed the installation, every thing seems to be working fine on a single system environment, now I am splitting it to multiple servers
I like to know your commends on the server specification
To reduce the load on the server I am using a Digium card with echo cancellation ( 2E1PRI x2 x 2 servers=8 pri) and agents are on Voip Gateways with G711,

Planning the distribution as fallows:

Asterisk servers = 2 (Each with 4 PRI)
MySQL Server +Apache Web = 1 (the web here is only for Webform extended CRM)
Vicidial+Web = 1 ( the Web here is Vicidial Admin and Agent Login)


On such case what should be the server configuration ???
Also let me know if there is any document on easy config of asterisk and its extensions/users/conferance/ etc..

Waiting for your valuable inputs

[mflorell]

Is this specific setup for 50 agents or 100?

Will you be doing full recording?

[voip.art]

It is for 200 agents and expandable to 300 agents soon

[mflorell]

You will need more servers for 200 seats.

Do you want full recording of all calls?

If you only have 8 T1/E1 ports you will not have very many lines for those 200 agents.

[voip.art]

I have an STM Terminated at the location, and I can take as much E1 I want, for the first 100 seats I will be using 8 E1 and as soon as I cross 150 agents I will add another 4 E1 ( or if needed may be before)

Yes I need 100% recording

What you suggest on server end how many servers? and each servers for what applications and what configuration

[mflorell]

You will need at least a 2 x quad core Database server with at least 16GB RAM
2 x webservers load balanced
4 x vicidial servers for 100 seats or 8 x vicidial servers for 200 seats
1 x archive server to store recordings

[voip.art]

I am reading those other inputes in forum saing 40 agents all in single servers and so on, and max i look for 1:1.5 ratio dialing no answering machine detection and 100% recording..
I still fail to understand the qty of severs, products like Aheeva based on asterisk the purchase cost itself is lesser than the cost of servers???
Can some one give some inputs on this...

[mflorell]

I am reading those other inputes in forum saing 40 agents all in single servers and so on, and max i look for 1:1.5 ratio dialing no answering machine detection and 100% recording..
I still fail to understand the qty of severs, products like Aheeva based on asterisk the purchase cost itself is lesser than the cost of servers???
Can some one give some inputs on this...

The capacity of a single server is extremely variable depending on the specific hardware, the overall system capacity, and the features desired.

Sure, you can put 40 agents on a single server, but not if it is a single core Xeon with only 1GB RAM. We have put over 80 agents on a single quad-core server before with no issues, but the database and web server were on separate machines in that VICIDIAL setup.

I thought you wanted to build a framework to eventually handle 300 agents. To do that you need a high-end database server and at least two web servers.

You asked for a recommendation, so that's what I gave you, the ideal situation for a large VICIDIAL system where you will never run into load issues no matter your configuration. If you want to go as cheap as possible go ahead and put as many agents on the system as you can and see how it works for you, you might be fine with 50-70 agent per VICIDIAL server, but you might have intermittant problems too.

If you want to go cheap you certainly can, but I have no idea where you came up with the idea that Aheeva would be cheaper. Do you have any idea how much they would charge you for a 300 seat system? It is based on Asterisk just like VICIDIAL is and would require just as much server hardware for a 300 seat system.

[voip.art]

Thanks for your Clarification, I have the fallowing Three Servers, please sugest me what goes to what server (for 100 seats , as and whne we grow to 200/300 we can add more servers)


Server A: HP Proland DL180 Dual-Core Intel Xeon Processor 5110 with 4MB L3 cache 4GBRAM

Server B: HP Proland DL180 Dual-Core Intel Xeon Processor 5110 with 4MB L3 cache 4GBRAM

Server C: HP Proland ML350 Quadcore Intel Xeon Processor 5310 8MB L2 Cache, and 4GB RAM

All with 500 GB SATA hardisk

[mflorell]

You could try putting 50 agents on each of the dual core servers and the web server/database server would be on the quad core server.

You will still want another machine to hold the archived recordings after a couple weeks if you need to keep them.

[voip.art]

Thanks for that update, it is realy useful
More to this I am using a Dual E1 ard with Echo cancelation as well as no voice coder change ( end to end g711 Ulaw) no sip trunk

Is there any doccument other than LOAD_BALANCING.txt for a detailed optimse setup for what goes to what server and minimum enviornement

[mflorell]

The load balancing pretty much covers most of it.

There really isn't much to it, just make sure the astguiclient.conf file and servers table entries are configured properly along with your extensions.conf and iax.conf on the VICIDIAL machines.

[voip.art]

Now I have better servers, Three Servers with Dual Xeon Processor Quad-Core, 8MB L2 Cache, 8 GB RAM , 146x2 SAS HDD (10,000RPM) , 2 Giga Eth Ports.

Server 1 : asterisk+astguiclient +2 x TE220B digium cards
Server 2 : asterisk+astguiclient +2 x TE220B digium cards
Server 3 : web+mysql
it is For 100 seats, I will go for another ftp server for recordiing on later stage : what should be the minimum spec?, as it is 100% recording needed
On such server config what you think my max capaity (number of agents) for hassle free operation?
what all things I need to consider for optimum performance ???
Please advise..

[mflorell]

The FTP server doesn't need to be anything very high performance, it just needs to have a lot of hard drive space. A single core CPU machine with 1GB RAM should do the trick.

[voip.art]

I struck in one point, and it is delaying my project
I have installed one server with every thing in one server and it worked
then i moved the web and sql to another server that also worked perfectly
now i added the second server with asterisk and astguiclient only and my sip registration to the second server working perfectly call from one server to another server extension working also but Agent login for second server is not happening

Users in second server not get the ring on there phone and an error on the agent screen Noone in the session 860xxx (conferance number)
I don't know where i am making mistake, I tried calling to the conferance number directly from my sip phone and i can ( i get the message you are the only one in this conference) but the agent login is still showing the error???
can you pls help me

[mflorell]

aren't you using IAX trunks from one server to the other?

Have you set the 010*010*010*015 extensions to match each server IP address in the dialplan?

Have you read the LOAD_BALANCE doc?

[voip.art]

yes, i am doing as per LOAD_BALANCE.txt documentations
I have ext mapping and trunking between thats why i am able to call extensions of each others

[mflorell]

It would be useful to see some Asterisk CLI output of the problem areas like the phon login.

Of course you can always call the VICIDIAL Group(http://www.vicidial.com) and we can login to your servers to figure out what your problem is if you would like this fixed quickly.

[voip.art]

Dear Mflorell,
Every thing gone well except some small issues as fallows , I will be grateful if you could answer me point by point on the fallowing

Present Status as fallows:
Server 1: HP Proland ML350 Quadcore Intel Xeon (2) Processor 5310 8MB L2 Cache, and 8GB RAM (MYSQL +WEB)

Server 2: HP Proland DL180 Dual-Core Intel Xeon (2) Processor 5110 with 4MB L3 cache 4GBRAM (ASTERISK +ASTGUCLIENT)

Server 3: HP Proland DL180 Dual-Core Intel Xeon (2) Processor 5110 with 4MB L3 cache 4GBRAM (ASTERISK +ASTGUCLIENT)

Presently I am not using the Server 3 and I have 76 Agents on Server 2 Server 3 is for expansion ( LOAD BALANCING)

Q1:
In Server 1: I Could see that the CPU utlization is not going above 4 to 5% at any point of time in full load and memory utlization is about 70-80% even if i make 20 user log in or 50 user or even all 76 users loged in:
So Can I make up to 100 users in this server ???

Q2:
After Few Hours of Predicitive Dialing I could see number of Call geting connected go very less , but if I remove the E1 and put it back it works well again, On investigating more in to it I found the asterisk is ialing only on few Trunk channels (only on 7 or 8 channels) and all other channles it gives error "Cause 34" "Congesion" , It is like those chanels are getting blocked for out going call till I make the PRI physicaly down and up again or restart the server.
What Could be the problem and how to rectifiy..

Q3:
I have a SIP based trunking gateway with few E1s on it, Is there a way to use a SIP trunk when the local trunks are full ???

Q4:
Some times I am getting Agents on cross talk ( and found both agents are in same conferance) I am using astguiclient 2.0.4.1rc2

Q5:
Some time I am finding my agents web page get updated very delayed ( especialy on first login) , Where as my Server1 ( MysQl+Web) shows only 40% of Memory and less than 5% of CPU utlization. Also I have eaccelarator installed in this, but I could see too many httpd service running (as much as 150) What is the way out to resolve this.

I know it is too big a message but hope you will find time to replay this.
Thanks in advance
Regards

[mflorell]

Dear Mflorell,
Q1:
In Server 1: I Could see that the CPU utlization is not going above 4 to 5% at any point of time in full load and memory utlization is about 70-80% even if i make 20 user log in or 50 user or even all 76 users loged in:
So Can I make up to 100 users in this server ???

It is not recommended, but it may be possible. We have had over 72 agents on a single server before and the server had similar specs.

Q2:
After Few Hours of Predicitive Dialing I could see number of Call geting connected go very less , but if I remove the E1 and put it back it works well again, On investigating more in to it I found the asterisk is ialing only on few Trunk channels (only on 7 or 8 channels) and all other channles it gives error "Cause 34" "Congesion" , It is like those chanels are getting blocked for out going call till I make the PRI physicaly down and up again or restart the server.
What Could be the problem and how to rectifiy..

We have noticed this at many call centers, not just centers using VICIDIAL either. It is usually caused by carrier equipment not being able to handle the rapid dialing, or it can be caused by carrier throttling of the circuits, forcably setting call attempts to CONGESTION to lower the load on their network.

Q3:
I have a SIP based trunking gateway with few E1s on it, Is there a way to use a SIP trunk when the local trunks are full ???

Yes, just add another line to your dialplan after the first dial

Q4:
Some times I am getting Agents on cross talk ( and found both agents are in same conferance) I am using astguiclient 2.0.4.1rc2

This is usually either caused by improper logging out of the agent or high server load or load spikes.

Q5:
Some time I am finding my agents web page get updated very delayed ( especialy on first login) , Where as my Server1 ( MysQl+Web) shows only 40% of Memory and less than 5% of CPU utlization. Also I have eaccelarator installed in this, but I could see too many httpd service running (as much as 150) What is the way out to resolve this.

Move your web server to another physical machine, and enable Prefork in Apache.

[gschaller]

Can you please tell the status of your system? Are you running with 300 agents now?
I were asked to set up a center with 200 agents and full recording. Mixed inbound and outbound. My first thoughts are 4 Asterisk servers (each 50 agents), 2 Web-servers and 1 MySQL server.

[Op3r]

Here's my setup for a 200 seats vicidial.

10 HTTP/Asterisk servers - 20 seats per server - quadcore with 4gb memory. Using vicidialnow community edition.

1 Database server. - dual quad xeon processor with 16gb memory. using centos 64bit.

I never dared to go more than 25 seats per asterisk server. this might be a huge waste of hardware resources but I get to sleep more than 8 hours at night.

I never run into much trouble except when an asterisk suddenly stop if we forgot to reboot the server at least twice a week.

[gschaller]

Thanks for your answer. Can you tell me the load of your database server? Is it working on it's limit or is it possible to add some more agents with that specs?

[williamconley]

Why don't you reboot your server nightly like the rest of us? Are you special?

[Op3r]

the reason I dont reboot nightly is because I forgot to put in the reboot on the crontab and too lazy to do it considering there are 10 servers . so I just asked the techs to just reboot it at least twice weekly

[gardo]

I never reboot my servers unless it's needed. The uptime usually are as high as 100+ days.

[mflorell]

on a multi-server system we almost never reboot our database or web servers, but the asterisk/Vicidial servers are usually scheduled to reboot automatically. In some cases they reboot nightly, weekly or monthly depending on the customer application.