Optimium system settings.

[Nefariousparity]

We have 11-15 agents on average and are running an outbound Predictive campaign.

These are our current settings:
Max FILL Calls per Second: 40
Max VICIDIAL Trunks: 200
Max Calls per Second: 100

Other interesting specs.
System Load: 184 - 13%
Live Channels: 287

I see congestion a lot, and some times agents have trouble making out bound manual calls, as well as three way transfer outbound. Our sip trunk has 400 channels, and our dialer is usually on ratio 18-25.

-Kind Regards.

Tl;DR
Help with optimum settings.

[omarrodriguezt]

Reduce the Max calls per second. Maximum 20.
Also, try another carrier or use blended carriers

[Nefariousparity]

Thank you for the response Omar. Testing right now. Could you please explain to me, what the math behind all of this is?

[omarrodriguezt]

Well, there are not maths regarding congestions. If you carrier only allow you 50 simultaneous calls, you can not send 100 calls at the same time
First, ask your carrier hat is the maximum simultaneous calls you can send.

[Nefariousparity]

Ok, because I just changed it down to twenty, and it took three times of trying to call to get manual to dial out. With dial level at 16 and Seven agents logged in. They got to make call, and you hear initial da dah, and then ba doop instant drop. Frustrating because we are losing leads which is losing money.

[omarrodriguezt]

Dial level at 16? That is too high.
Can you please post the asterisk cli output here (dont forget to remove any sensitive data)?

[sadikhov]

how many agents dialing?

[williamconley]

Congestion has several basic causes, eliminate them one at a time ...:

1) You are dialing too many overall channels, and when you exceed them the calls over your simultaneous call limit are rejected via congestion. This setting is in Admin->Servers under max calls. Set it to 10 and see if congestion disappears. If it does, raise it by tens until you "congest" and then call your carrier to find out why your limit is set to that number. Wait at least a couple minutes at each level to be sure you get an accurate reading.

2) You are dialing "too hard" ie: too many calls per second. If you do not have an agreement with your carrier regarding calls per second and you do not have more than 100 channels, you should try 1 call per second to see if that reduces your congestion. Then discuss Calls Per Second with your carrier and see if you are over your limit. Great to test and easy to set (also in admin->servers).

3) You are dialing the same numbers in the same list that were all "NA" last time and are "NA" again, most of which are actually disconnected numbers. These can often be returned as "congestion" which is a Feature, not a Bug. Stop dialing the same disconnected numbers and you'll stop getting congestion! LOL

4) Your internet connection to your carrier is poor and exceeds the "qualify=500" setting (500=.5 second round trip). If your settings is qualify=yes, that is 1000 or 1 second ... since you did not choose a number it was chosen for you. In either case if the round trip for a packet exceeds the qualify setting, the call will "auto-congest" because your carrier is "unreachable" and your system is configured to congest the call so it can failover to a more reliable carrier. To avoid this you can set qualify=NO, but that's not recommended as your connection to the carrier may not actually exist any more and those calls will still fail. Better to find out why the packets are taking too long to return or increase to 2000 or even 3000. At that level, if they do not return ... you have no connection to your carrier. Fix your connection.

5) My personal favorite: Pay your phone bill. Many carriers will congest for failure to pay.

[omarrodriguezt]

Also, take a look of http://wiki.vicidial.org/

[Nefariousparity]

Ok, thank you so much. I am trying various things that you are suggesting. Sorry for the delay.

I think this is safe...

[May 6 10:32:31] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0
[May 6 10:32:31] NOTICE[13515]: rtp.c:1378 ast_rtp_read: Unknown RTP codec 126 received from '*****'
[May 6 10:32:33] -- Got SIP response 503 "Call could not be completed" back from *****
[May 6 10:32:33] -- SIP/Connecium-00002cba is circuit-busy
[May 6 10:32:33] == Everyone is busy/congested at this time (1:0/1/0)
[May 6 10:32:33] -- Executing [919098630617@default:3] Hangup("Local/919098630617@default-68ae,2", "") in new stack
[May 6 10:32:33] == Spawn extension (default, 919098630617, 3) exited non-zero on 'Local/919098630617@default-68ae,2'
[May 6 10:32:33] -- Executing [h@default:1] DeadAGI("Local/919098630617@default-68ae,2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----34-----CONGESTION----------") in new stack
[May 6 10:32:33] -- Got SIP response 503 "Call could not be completed" back from *****
[May 6 10:32:33] -- SIP/Connecium-00002cbd is circuit-busy
[May 6 10:32:33] == Everyone is busy/congested at this time (1:0/1/0)
[May 6 10:32:33] -- Executing [919098640174@default:3] Hangup("Local/919098640174@default-1c38,2", "") in new stack
[May 6 10:32:33] == Spawn extension (default, 919098640174, 3) exited non-zero on 'Local/919098640174@default-1c38,2'
[May 6 10:32:33] -- Executing [h@default:1] DeadAGI("Local/919098640174@default-1c38,2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----34-----CONGESTION----------") in new stack
[May 6 10:32:33] -- Got SIP response 503 "Call could not be completed" back from *****
[May 6 10:32:33] -- SIP/Connecium-00002cb9 is circuit-busy
[May 6 10:32:33] == Everyone is busy/congested at this time (1:0/1/0)
[May 6 10:32:33] -- Executing [919098623937@default:3] Hangup("Local/919098623937@default-bb3d,2", "") in new stack
[May 6 10:32:33] == Spawn extension (default, 919098623937, 3) exited non-zero on 'Local/919098623937@default-bb3d,2'
[May 6 10:32:33] -- Executing [h@default:1] DeadAGI("Local/919098623937@default-bb3d,2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----34-----CONGESTION----------") in new stack
[May 6 10:32:34] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0
[May 6 10:32:34] == Manager 'sendcron' logged off from *****
[May 6 10:32:34] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0
[May 6 10:32:34] == Manager 'sendcron' logged off from *****
[May 6 10:32:34] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0
[May 6 10:32:34] == Manager 'sendcron' logged off from *****
[May 6 10:32:36] -- Got SIP response 503 "Call could not be completed" back from *****
[May 6 10:32:36] -- SIP/Connecium-00002cc0 is circuit-busy
[May 6 10:32:36] == Everyone is busy/congested at this time (1:0/1/0)
[May 6 10:32:36] -- Executing [919098642415@default:3] Hangup("Local/919098642415@default-d499,2", "") in new stack
[May 6 10:32:36] == Spawn extension (default, 919098642415, 3) exited non-zero on 'Local/919098642415@default-d499,2'
[May 6 10:32:36] -- Executing [h@default:1] DeadAGI("Local/919098642415@default-d499,2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----34-----CONGESTION----------") in new stack
[May 6 10:32:37] -- Got SIP response 503 "Call could not be completed" back from *****
[May 6 10:32:37] -- SIP/Connecium-00002cc1 is circuit-busy
[May 6 10:32:37] == Everyone is busy/congested at this time (1:0/1/0)
[May 6 10:32:37] -- Executing [919098642415@default:3] Hangup("Local/919098642415@default-e488,2", "") in new stack
[May 6 10:32:37] == Spawn extension (default, 919098642415, 3) exited non-zero on 'Local/919098642415@default-e488,2'
[May 6 10:32:37] -- Executing [h@default:1] DeadAGI("Local/919098642415@default-e488,2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----34-----CONGESTION----------") in new stack
[May 6 10:32:37] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0
[May 6 10:32:37] == Manager 'sendcron' logged off from *****
[May 6 10:32:38] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0
[May 6 10:32:38] == Manager 'sendcron' logged off from *****
[May 6 10:32:42] == Manager 'sendcron' logged off from *****
[May 6 10:32:42] == Spawn extension (default, 919098643870, 2) exited non-zero on 'Local/919098643870@default-cee1,2'
[May 6 10:32:42] -- Executing [h@default:1] DeadAGI("Local/919098643870@default-cee1,2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----0-----CANCEL----------") in new stack
[May 6 10:32:42] NOTICE[13515]: rtp.c:1378 ast_rtp_read: Unknown RTP codec 126 received from '*****'
[May 6 10:32:43] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0
[May 6 10:32:44] ERROR[19042]: utils.c:967 ast_carefulwrite: write() returned error: Broken pipe
[May 6 10:32:44] ERROR[19042]: utils.c:967 ast_carefulwrite: write() returned error: Broken pipe
[May 6 10:32:44] ERROR[19042]: utils.c:967 ast_carefulwrite: write() returned error: Broken pipe
[May 6 10:32:44] == Manager 'sendcron' logged off from *****
[May 6 10:32:44] == Spawn extension (default, 919098644993, 2) exited non-zero on 'Local/919098644993@default-fe9b,2'
[May 6 10:32:44] -- Executing [h@default:1] DeadAGI("Local/919098644993@default-fe9b,2", "agi://127.0.0.1:4577/call_log--HVcauses--PRI-----NODEBUG-----0-----CANCEL----------") in new stack
[May 6 10:32:45] -- AGI Script agi://127.0.0.1:4577/call_log--HVcauses ... ---------- completed, returning 0

So right now these are my settings::

Dial Method: Ratio
Auto Dial Level: 5 (0 = off) HELP ADAPT OVERRIDE
Available Only Tally: Y
Drop Percentage Limit: 9 %
Maximum Adapt Dial Level: 5 number only
Latest Server Time: 2100 4 digits only
Adapt Intensity Modifier: 34- More Intense
Dial Level Difference Target: 3 --- 3 Calls waiting For Agents
Concurrent Transfers: Auto
Queue Priority: 76 - Higher
Multiple Campaign Drop Rate Group: Disabled
Inbound Queue No Dial: Disabled

These are our current settings:
Max FILL Calls per Second: 40
Max VICIDIAL Trunks: 200
Max Calls per Second: 20

Our internet connection is 50mbpsx25mbps. We currently have 10 agents logged in. That is about all of this offices internet usage. We barely spike above 10mbps in either direction. And that is a heavy day! I will have to check my RRD graphs on my FreeBSD router.

Dial ratio of 5 seems to work but them my average agent wait time goes up to 25 seconds.


EDIT *** Carrier Setup ***

[Connecium]
type=peer
host=*******
port=5060
dtmfmode=rfc2833
disallow=all
allow=gsm
allow=ulaw
qualify=yes
canreinvite=no
context=trunkoutbound



[Xcast_OutBound]
type=friend
host=*****
port=5060
dtmfmode=rfc2833
disallow=all
allow=ulaw
allow=gsm
qualify=yes
canreinvite=no
context=trunkoutbound

[Xcast_Inbound]
type=friend
host=*****
port=5060
dtmfmode=rfc2833
disallow=all
allow=ulaw
qualify=yes
canreinvite=no
context=trunkinbound

So just thought I should also include, that I have setup previous to your guys suggestions having all manual and 3 way transfers go out on a different carrier. So for instance, Predictive outbound campaign is using Connecium, while manual and three way transfer using Xcast_OutBound.

exten => _91NXXNXXXXXX,1,AGI(agi://127.0.0.1:4577/call_log)
exten => _91NXXNXXXXXX,2,Dial(SIP/${EXTEN:1}@Connecium,,tTo)
exten => _91NXXNXXXXXX,3,Hangup

exten => _81NXXNXXXXXX,1,AGI(agi://127.0.0.1:4577/call_log)
exten => _81NXXNXXXXXX,2,Dial(SIP/${EXTEN:1}@Xcast_OutBound,,tTo)
exten => _81NXXNXXXXXX,3,Hangup

Once again thank you all for the support.

[Nefariousparity]

***EDit****So I just edited my manual calls, and three way to use the same carrier as our AutoDialer. In short, everything uses the same trunk now. As I noticed with our Xcast_Outbound trunk, I was able to catch a transfer and it said busy congested. My carrier also responded saying that our max CPS is 150, and they don't restrict the channels on their customers.

[williamconley]

you posted all the call CLI information after they failed. which definitely shows they failed. but you did not post any of the information that may be helpful to show WHY they failed.

Code: Select all

Executing [919098630617@default:3]

you have a 3 line dialplan for each carrier. your cli output should show all three lines, not just the hangup portion of the line. 919098630617@default:3 ... where is :1 and :2?

[Nefariousparity]

What is the best way to capture CLI? I typically am logged into asterisk cli via asterisk -rvvvvvvvvv. And try to catch it, it flies through the screen pretty fast.

Still having this issue where three way transfers are dropping.

[omarrodriguezt]

put the dial ratio to 1 and logged in only 1 agent. It will be slow.

[Nefariousparity]

So just really quick, I did what you suggested at the dialer being really slow, I grabbed this right out of the log.

[May 7 10:43:27] -- Got SIP response 503 "Carrier Service Unavailable" back from *.*.*.*
[May 7 10:43:27] -- SIP/Xcast_OutBound-000085f5 is circuit-busy

Does that mean my carrier? Or can I blame the the client? They did recently change their DID.

[omarrodriguezt]

Before you blame your carrier, try to dial manually the same pattern that you are seeing in the CLI.

[Nefariousparity]

WE finally got the customer to admit that it was on them. Sometimes this tech job is so frustrating, because if something goes wrong even when we know the client changed something on their end I get blamed.

[omarrodriguezt]

This is very familiar for us

[callcrazy]

Dial level on our system never goes above 2.5. 16 is absolutely insane unless your leads are from 1982 and no one answers.

On RATIO dialing with a level of 1 there will be 1 outgoing call for each available agent. At 16 there are 16 calls for each available agent. If you have 10 agents available your system will instantly start dialing 160 calls at once. The max fill calls per second setting at 20 means that every second the dialer script loops it will only make 20 calls. So to get the 160 calls going it will take 8 seconds.

On our system we set max fill calls very low like 2-3. This made a much smoother outbound flow of calls for us and lowered our drop % considerably.

[Nefariousparity]

That is quite interesting. I was unfortunately laid off as a admin of that last job. However I am friends with the owner. He recently left for Florida to inspect large sailing vessels. I noticed two things when the floor manager came up to me pulling out all her hair. One, he had the dial level at "32-ratio" and then he also had the status of "NA" on. I think he wanted to guarantee people were on the phone making calls. He also likes to include this small margin of error with his data. So maybe 1 in every 500 "NA" numbers dials, he turns it on. Because he fails to admit that in a list of 20-30K numbers about 10K get the "NA" status.