Mysql limits or list size

[chasejordan1]

GoAutodial 2.2.1 latest and greatest.

How can I increase the sql limits so that I can increase the size of the lists i load?

Thanks Thad

[williamconley]

the limits are not sql limits. they would be php time and file upload limits.

and yes, you can increase the timeout for php and the file upload size. php has LOTS of help available for it, the values are in the php.ini file for web (not the CLI version of the php.ini, that is not being used when you are on a web page, there are two copies of the php.ini file).

Also, there is a command-line lead loader that will allow virtually unlimited upload capacity. You need to scp or find some other method to upload the file, then you execute the command line lead loader and it will run for as long as it takes to load the leads.

Please don't say latest - specify your Vicidial Version and BUILD (GoAutoDial is the Installer, but you have not listed the version which is likely 2.0CE, 2.2.1 is the Vicidial Version, but you left off the BUILD). It's required to post at least Vicidial Version and Build to submit requests (after all, the board is FREE , it's the least you can do ...)

[chasejordan1]

So what is the "command-line lead loader" command and how would I find out how to upload or import leads via scp. No secrets please hehe.

[chasejordan1]

Ok so I increase php.ini file please see below and I was able to load 1million records took forever but worked, if there is a faster way please let me know but it did load using GUI.

/etc/php.ini

;;;;;;;;;;;;;;;;;;;
; Resource Limits ;
;;;;;;;;;;;;;;;;;;;

max_execution_time = 60000000 ; Maximum execution time of each script, in seconds
max_input_time = 180000000 ; Maximum amount of time each script may spend parsing request data
memory_limit = 768M ; Maximum amount of memory a script may consume
upload_max_filesize = 120M
post_max_size = 120M

some of these are in different areas so you have to look for them. Then to be safe I rebooted server and all is fine, slow but fine. Thanks

[williamconley]

LOL

have you looked at /usr/share/astguiclient? that's where ALL the scripts are. try VICIDIAL_IN_new_leads_file.pl

Code: Select all

perl VICIDIAL_IN_new_leads_file.pl --help

[chasejordan1]

whats so funny, I did what I had too. anyway what you gave me doesn't help. But it's ok. Thanks. But hey can you help me with the VM to EM issue I am having. Thanks Thad

[williamconley]

the instructions for the use of that script are IN that script. all you need to do to use it is use WinSCP (or any other file transfer utility you feel like using) to transfer the file to the vicidial server, directly into the appropriate folder ... then fire off the script. But the file must be formatted according to the notes in the script (or you must modify the script to have a ready-made template for your format).

VM to EM is an asterisk function. We use scripted email in linux servers all over the place. but not in vicidial. LOL

in fact: look into Zimbra and you'll find a full replacement for Microsoft Exchange that runs on Linux and is FREE.

[VFRDavid]

I had a very large suppression file to import (just phone numbers, so that I wouldn't import these in any lists I purchase in the future - PLUS - vs just adding to our DNC table - I KNOW that they're NOT getting imported - so - I know the actual net count - and not have to rely on a future "DNCC" or "DNCL" to update my actual net count, if that makes sense). It was 3.5M+ records - so - rather than split it up into many 75k files - I tried to use this utility - and the output is QUITE different from the GUI lead loader.

Before I go further, I am running VICIdial VERSION: 2.14-670a; BUILD: 180424-1521 (from the bottom of all of the menus); and Version: 2.14b0.5 SVN 2973; DB Schema 1542 (from the Admin>System Settings screen). I did an svn up - and this utility's version is exactly the same as for the SVN I am running - so - the fact that my SVN is a bit out of date, doesn't appear to be material to my issue / question...

I use the GUI's normal output that shows the DID and ListID duped against to produce reports - I maintain each lists' performance (values for OverAll, 30 Days, 7 Days and Today for each of the following: contacts, sales, contacts:sales ratio, calls, and contact percent) - so - if a newly imported list loses a significant percentage against an existing list, I can see how the duplicates performed (or at least how the list they came from did) - which allows me to assign some sort of potential value to the leads netted, etc.

Anyway - the utility mentioned in this thread doesn't seem to show the same thing - at least, not the way(s) that I attempted to run it (with a 100 record file, just to see how it worked).

The import layout is documented for the most part, but - so that I don't have to try a bunch of different combinations - is there any command line options that would give me something closer to what the GUI importer does?

Thanks for any feedback / help.

[williamconley]

1) Lots of text, but no clear details.

2) Using a utility that's nine years old, does that utility have any updates in the last few years?

3) For SVN Clarity:

svn up ... does not actually update any LIVE code. It merely updates the code in the Local Repository

There is no "this is definitely the SVN revision level of code you are presently using" command. Since the SVN repository is not the LIVE code.

To determine the Very Likely SVN Revision Level: diff against the admin.php file in the svn repository and the live vicidial/admin.php in the web root. When you get a match (by upgrading/downgrading your svn level in the repo), you have the same version in the repo compared to the LIVE code. That's your SVN revision level. If you're not sure whether to upgrade or downgrade, you can have a look at the "build" value in the admin.php file.

Beware, however, that any customizations in the admin.php will always cause at least some differences. You could use a different file, but always finalize with the admin.php since that will have a matching version/build line even if you have custom code. There are also SVN browsers available that can make the search a little simpler.

[VFRDavid]

The utility that was downloaded via SVN up and the one already on my system both had the same version (in the version history listed at the top of each file) - they both appear to be under five years old - from October 10, 2016, based on the version number found in the beginning of each file (161010-1616), so not quite nine years old, but, regardless - it doesn't look there has been any updates since then, and the --help doesn't show anything that gives options for what the output looks like.

What I am looking for is a way to both import a very large file AND generate output similar to the web GUI, due to some post-processes I perform internally that rely on parsing the current output.

Thanks for your answer - I really do appreciate it - BTW - great point on "3)" - I don't think that I ever really considered that - unless I know for sure which, if any, files have been modified since installation, I cannot really count on the SVN version showing in my system - especially for "inherited" systems...

[williamconley]

The utility that was downloaded ...

What is the filename/path of this utility? Is it a secret?

What I am looking for is a way to both import a very large file AND generate output similar to the web GUI, due to some post-processes I perform internally that rely on parsing the current output.

I'm a little lost on how these are connected. I'll start here: Can you just use wget, file_get_contents or curl to request the Web GUI output and then your output isn't "similar" to the gui output, it *is* the gui output?

[VFRDavid]

If I may...you answer with "command line lead loader" then comment "have you tried looking in usr/share/astguiclient" when they ask for the name of it and help running it, yet then also comment that I didn't tell you the name of the command line lead loader I used, after I said I used the one you specified...shouldn't you follow your own rules?



And the output from VICIDIAL_IN_new_leads_file.pl is very different from admin_listloader_fourth_gen.php - so I'm not sure what you mean by using wget to obtain the file contents - are you saying that if I run the .pl lead loader and have the output somehow show up on a web page, that it will match the output of the .php page?

Anyway - just busting your chops a bit - my VICIdial knowledge certainly doesn't hold a candle to yours - which is why I usually post questions - and you usually post answers - but - before you scold me - re-read the thread - I think it's possible you'd admit that you might have complained about your answers, had they been written to you by someone else

[carpenox]

:::grabs the popcorn:::