running multiple broadcast campaigns simultaneously

[sfligg]

Hi, is it possible to run multiple broadcast campaigns simultaneously?
We are running vicidialnow 1.1 on a new dual core system with plenty of gusto.
We are setting this up for strictly broadcast campaigns (dial out only) as a customer reminder system. I have multiple campaigns setup already but they are all staggered throughout the week so that only one campaign is running at any given time. This has worked out pretty good. Now we have a situation where I need to setup several other broadcast campaigns that will run throughout the day but with minimal load. I've setup everything, activated the new campaigns, lists, and remote agents. Tested the new campaigns and everything seems to be working fine. However, now I have noticed that my original campaigns are not working properly. Not all the leads will be called in the campaign and even at times the campaign will not even run. All campaigns have their own lists and remote agents. The original campaigns have their own call times with nothing really over lapping ... except when dealing with different time zones there will be some overlapping. The new campaigns are set to run during business hours during the week so they will run at the same time the original campaigns do. When I turned off the new campaigns, everything was back to normal and the original campaigns ran fine. This makes me think that I can not run multiple campaigns simultaneously, or I am missing something in the setup.
Any idea's?

[mflorell]

loadavg when running one campaign, and two?

[sfligg]

In Server Status and Reports, the Server Performance report reads:

Average/Peak load for server: 20.7860 / 105
Average USER process cpu percentage: 5.2817 %
Average SYSTEM process cpu percentage: 2.5981 %
Average IDLE process cpu percentage: 92.0092 %

It is a new PC dual core 2.2 GHz with 2 gigs of ram.

[sfligg]

Another question, what would keep a campaign from running that has previously run with no problem? Scenario is this: Tuesdays I have six campaigns scheduled to run, all at different times. These are all broadcast only campaigns. The first four ran fine with no problems. The fifth one did not run at all, the system did not even look at it (staus='NEW' and modify_date='0000-00-00 00:00:00'). Then the sixth campaign ran but did not call all the leads in the list (called only 218 leads out of 298 leads). All six campaigns have run fine before, this just happens periodically (as I also have six campaigns running Wednesday, Thursday, and Fridays). And sometimes it may be the second campaign that doesn’t run and one of the others (or all) that do not complete the calls in the lists. All campaigns have plenty of time and resources to run through. This is all run on a brand new dedicated PC dual core 2.2 GHz with 2 gigs of ram.
I mention this in this post thinking it might be related.

[mflorell]

How many concurrent calls are you placing for each of the campaigns?

[sfligg]

Only 5 concurrent calls each at a time.

[sfligg]

A point of interest/information ... regarding my post from yesterday (Dec 17th). Wednesday I had another six campaigns run throughout the day. There were at least twice as many calls (broadcast only) and all six campaigns ran fine. These are all different campaigns but setup the same except for the obvious changes like what extension it points to, list, call time, etc.

[mflorell]

When a campaign won't run, are there leads in the hopper at that time?

[sfligg]

I haven't logged in to watch the hopper, so I can't say yet. I will try and monitor the 3 daytime campaigns but the last 3 run during the evening.
If the leads were loaded in the hopper, would that not modify the record changing the modify_date field in the vicidial_list table? At one point a while back I noticed the modify_date field would be updated but the status field still had NEW as a value. That is when we got the new PC with more horsepower.

[sfligg]

Wednesday and Thursday's campaigns (6 each day) all ran fine, however Fridays six campaigns did not complete all their calls. Friday is a light day for calls (broadcast only), not so many to make. When I logged in to look at the hopper while it was running, the hopper was empty but I had figured that was due to timezones. I have logged in other days when there were still calls to make and the hopper was empty but the campaign eventually finished them. At least on Friday the campaigns ran, they just did not complete.

[williamconley]

Check to see why your hopper runs out. Watch a campaign run. You may find that your connection to your VOIP provider is not good or that your server is losing internet. If that happens, the system will quite often eat through the calls rather quickly and drain the hopper. The hopper should never really be empty.

So watch while it is running once and see if it is really making all the calls it thinks it it.

Are these campaigns the same from day to day, or are Friday's campaigns different from Thursdays?

What method are you using to start/stop the campaigns on the days they run?

[sfligg]

Is there any logging I can setup to monitor the hopper? I have serveral other duties so sitting and watching the hopper as the campaigns run isn't a solution can do, especially if the campaings happen to run okay for that day. The Vici server connects to out Trixbox (for out office system), so we can setup logging there to see if vici is getting through. But I suspect (as I have seen in the past) that there will be no records in trixbox for the calls that do not run. Each champaign is different (6 in a day), so the campaign that runs on Tuesday from 1:00 to 5:00pm, that is the only day and time is runs. All the campaigns were copied from a basic template and then altered accordingly. I have created "call times" to control when the campaigns run.

[williamconley]

I was not so much suggesting monitoring the hopper, but being available at the moment that the hopper runs empty so you can figure out why. Logging the hopper level would be interesting, and if you look in /var/log/astguiclient you will find a multitude of logs which may suit your purpose. Unfortunately, the time savings in not sitting down to look at the campaign as it runs a few time to try to catch the hopper empty ... will likely be eaten up by poring over logs to match up times with the occurrence of the empty hopper.

What I expect to be happening is that the system tries to make calls and fails. But Vicidial only knows that the call failed, not why. So it makes another call ... if they are all failing this can eat your hopper rather quickly and in fact can eat all your leads if you're not careful.

Why is this system calling out through Trixbox instead of straight to your VOIP provider? Unless you have hardware for hard lines in your Trixbox that are not available any other way, you should consider bypassing the trixbox entirely to see if this resolves your issue. You are wasting processing power in your trixbox by passing calls through it and using a system commonly known as incompatible (i won't say unstable, but do your research) with vicidial.

[sfligg]

okay the problem is not with the hopper filling up, but rather vicidial is not taking the leads in the hopper. the campaign starts, the hopper fills, and then nothing ... the leads just sit there. after a while vicidal will grab the first group of leads from the hopper, dial them and then leave the rest. then after a while it may grab the next group (group meaning # of lines to dial by the remote user), dial them and again back to the waiting game. the asterisk output when it is not calling is just a whole bunch of:

vici*CLI>
== Manager 'sendcron' logged on from 127.0.0.1
vici*CLI>
== Manager 'sendcron' logged off from 127.0.0.1
vici*CLI>
== Parsing '/etc/asterisk/manager.conf': Found
vici*CLI>
== Manager 'sendcron' logged on from 127.0.0.1
vici*CLI>
== Manager 'sendcron' logged off from 127.0.0.1
vici*CLI>
== Parsing '/etc/asterisk/manager.conf': Found
vici*CLI>

there are no errors that I can see.
so now I realize that when there are some leads not called in a list, it probably just ran out of time according to the campaign.
So now my question is, why is vici not running through the hopper continuously?

[williamconley]

Check your Vicidial Manager Real Time Interface and see how many calls Vicidial believes are "ringing" or "connected" vs how many agents the campaign has. Compare this to your dial ratio and see if Vicidial is at least dialing correctly according to its own rules.

* If you have 3 agents available and a 2:1 dialing ratio and no agents online, you should have 6 calls ringing according to the Real Time interface. If you do not, we go in one direction. If you do, we go in another.

Then check your asterisk CLI and see if it agrees with the number of active calls (logged in agents would be active calls, ringing or connected prospects would be active calls ...). If this agrees with Vicidial, we go one way, if not, we go another.

Also, at the moment that Vicidial "makes that group of calls" check for errors in the CLI output regarding how those calls were made (chan_unavail would be considered an error).

[sfligg]

Okay, what keeps vicidial from dialing the leads in the hopper? The campaign is running, hopper is full, and the remote agents are waiting. The leads in the hopper are good and should be called, but nothing is happening. I have narrowed this down to a few campaigns, consistently. So I have deleted the campaigns and recreated them comparing with the campaigns that run fine. I have moved the campaigns so they run on different days and different times. Sometimes they work and sometimes they don't. Most times the hopper fills but the leads are never called. There no errors in my CLI output ... I am initiating it by typing "asterisk -vvvvvr" at the command prompt. Is there something else I should be using to debug this?

Here is my real-time screen data:

DIAL LEVEL: 1
DIALABLE LEADS: 13
HOPPER LEVEL: 10
LEADS IN HOPPER: 10
TRUNK SHORT/FILL: 5 / 0
CALLS TODAY: 0
DROPPED / ANSWERED: 0 / 0
DROPPED PERCENT: 0%
FILTER: NONE
AVG AGENTS: 5
DL DIFF: 5
DIFF: 100.00%
TIME: 2009-01-30 11:04:52
DIAL METHOD: RATIO
STATUSES: NEW
ORDER: DOWN PHONE

5 agents logged in
0 agents in calls
5 agents waiting
0 paused agents

+-----------+------+-----------+--------+
| STATION | USER | SESSIONID | STATUS |
+-----------+------+-----------+--------+
| R/854 | 854 | 8335 | READY |
| R/853 | 853 | 8335 | READY |
| R/852 | 852 | 8335 | READY |
| R/851 | 851 | 8335 | READY |
| R/850 | 850 | 8335 | READY |
+---------+------+-----------+--------+
5 agents logged in on all servers
System Load Average: 0.27
- Agent waiting for call


CLI output .... continuous ...

== Manager 'sendcron' logged on from 127.0.0.1
== Manager 'sendcron' logged off from 127.0.0.1
== Manager 'sendcron' logged off from 127.0.0.1
== Parsing '/etc/asterisk/manager.conf': Found
== Manager 'sendcron' logged on from 127.0.0.1
== Parsing '/etc/asterisk/manager.conf': Found
== Manager 'sendcron' logged on from 127.0.0.1
== Manager 'sendcron' logged off from 127.0.0.1
== Manager 'sendcron' logged off from 127.0.0.1
== Refreshing DNS lookups.
== Parsing '/etc/asterisk/manager.conf': Found
== Manager 'sendcron' logged on from 127.0.0.1
== Manager 'sendcron' logged off from 127.0.0.1
== Parsing '/etc/asterisk/manager.conf': Found
== Manager 'sendcron' logged on from 127.0.0.1
== Manager 'sendcron' logged off from 127.0.0.1

[williamconley]

/var/log/astguiclient logs are generally very useful for this diagnosis.

if they are not on, they are turned on via "ADMIN" -> "Servers"

Modify your server and locate "server logs" and "AGI Output"