Delays loading the leads to hopper

[jessiekidfernando]

Hello Everyone,

I've been seeing delays loading the leads to hopper every time this script "/usr/share/astguiclient/AST_VDhopper.pl" runs via cron every minute. When I tried to use --debug, it appears that there are a lot of "Doing DNC Check: XXXXXXXX -Y" and took almost more than 5 minutes to finish the script.

Do you know the reason and cause of this?

Thanks.

[alo]

How Many records to you have in your vicidial_dnc table?

[jessiekidfernando]

5million records. The weird thing here is it is happening only on a specific time.

[williamconley]

DNC should never be that large. These are all records of people you've called and they have said "do not call me!"? or did you download a DNC record from somewhere and toss it in your dNC table?

[jessiekidfernando]

did you download a DNC record from somewhere and toss it in your dNC table? - Yes

[williamconley]

That's why it is overloading. DNC lists from a government should be used to remove leads without loading them into Vicidial at all. Or to (nightly, weekly or monthly) compare to the lists in Vicidial and remove them.

The Vicidial "Company" or "Internal" DNC list is meant ONLY for people who have told your company to never call them again. It needs to be up to the second. This way if someone tells you at 2PM to NEVER F***ing call them again, at 4PM your dialer will NOT load them into the hopper and call them again.

This is not the same relationship as someone who went to a government site and put their name no a list for all call centers to not call them. The government gives you some leeway to enable that list and stop calling those members of the public. Usually 30 days.

If you have 30 days to remove someone from your lists, you can certainly perform a nightly check to get rid of them, not nearly as "intense" as "Yes, ma'am, we'll stop calling you ... yes, ma'am we understand that if we call you again you'll sue us, no ma'am we will definitly not be calling you again ... sorry again, ma'am." This relationship is special and has significant danger to your livelihood if not handled To The Second, which is why Vicidial has an Internal DNC list.

To meet this need, and stop call centers from being shut down due to this ongoing danger, the Vicidial Internal DNC system checks leads when are about to be dialed and REFUSES to dial them if the campaign is configured to adhere to the list. This means that each time Vicidial is about to dial a number, it checks against the internal DNC list. Every Time It Dials. The entire reason Vicidial has a "hopper" is to avoid the "entire" vicidial list each time it has to dial. The hopper is rarely more than a couple thousand numbers, and often has a system max capacity near 10k. Comparing 10k numbers to 5k DNS entries is a bit intense, but not too horrible. But when you must compare even 2K numbers against 5M ... that's gonna take some time. Every Time It Dials.

IMHO: You are trying to use a moped on the autobahn to carry several hundred passengers from A to B, when you SHOULD be putting those passengers on a Train instead.

Trying to load 5M leads into the DNC is an overload situation. If you MUST have 5M records in the Internal DNC, you will need a Very Powerful Database Server just to dial normally. The cost of that server will be much more than paying for a script to be built to remove National Do Not Call numbers from your system nightly.

[nikolaos]

And what is a good (i.e. automated) way to delete entries from a list already uploaded to vicidial? I could query and update the database directly but I would like to know what table(s) to work on and if there are precautions I should take (e.g. stop some services first do my job then start them)

[jessiekidfernando]

Any idea why it is taking too long to finish the hopper script? This is the last line that I got when I tried to execute it using the argument --debug.

Please see image below.

https://snag.gy/xGm9TK.jpg

We are executing this script every minute.

/usr/share/astguiclient/AST_VDhopper.pl -q --run-check

[williamconley]

run each of the queries yourself to see how long they take. they should take no more than one or two seconds.

note that a lot of callbacks and seriously slow down the system. call 'em or delete 'em.

[nikolaos]

DNC lists from a government should be used to remove leads without loading them into Vicidial at all. Or to (nightly, weekly or monthly) compare to the lists in Vicidial and remove them.

I would like to do both but I'm not sure how to do the later (remove entries from list already loaded to Vicidial).

As far as I can tell from a quick check all phone numbers end up in mysql's table vicidial_list. My first thought is to delete entries from vicidial_list and reboot. Is this OK or am I going to mess up vicidials internals?

[williamconley]

DNC lists from a government should be used to remove leads without loading them into Vicidial at all. Or to (nightly, weekly or monthly) compare to the lists in Vicidial and remove them.

I would like to do both but I'm not sure how to do the later (remove entries from list already loaded to Vicidial).

As far as I can tell from a quick check all phone numbers end up in mysql's table vicidial_list. My first thought is to delete entries from vicidial_list and reboot. Is this OK or am I going to mess up vicidials internals?

reboot is not needed. but remove them from the hopper table as well.

poundteam has a nightly ndnc checker, and that's what it does. but no deletion: merely changes the status from whatever it was to NDNC. Don't put NDNC in your dial status list and you're good to go. Also, it will not allow a fresh copy of the same lead to be re-uploaded if you're using dupe check on upload.

[nikolaos]

no deletion [is required]: merely change the status from whatever it was to NDNC. Don't put NDNC in your dial status list and you're good to go. Also, it will not allow a fresh copy of the same lead to be re-uploaded if you're using dupe check on upload.

Thanks for the guidance williamconley! Just to make sure I'm following you:

NDNC is a custom status and by default leads with this status will not be called. If one is foolish enough to go to Campaigns > {some campaign} > Add A Dial Status to Call: > [select NDNC] and click ADD it would override this sane default.

So basically all I have to do to clean up all my lists from a number that is in the National DNC list is a

Code: Select all

UPDATE vicidial_list SET status='NDNC' WHERE status<>'NDNC' AND phone_number="1234567890"

Am I correct?

[williamconley]

Code: Select all

UPDATE vicidial_list SET status='NDNC' WHERE phone_number="1234567890"

Code: Select all

delete from vicidial_hopper where lead_id IN (select lead_id from vicidial_list where status='NDNC')

[nikolaos]

Thanks williamconley!