SILLY Leads Upload Issues !!!

[gmcust3]

When I am uploading Leads :

Tab Seperated got : 9000 leads

When Uploading : its shows 7579 uploaded

In DB, I see only 1831 when I run count(*)

In Admin page, it shows 1831.

In vicidial_temp_file.txt, all 9000 records are there

listloader_stmts contains only 1831 or approx Insert Statement

Where can be the issue ?

[williamconley]

did you have deduping on, and how many did it say "good" and/or "bad" during the upload?

also, did you use the list_id override to be sure they all landed in the same list, just to be sure.

[gmcust3]

Uploaded another :

IST ID OVERRIDE FOR THIS FILE: 900038


record 0 BAD- PHONE: ROW: |0| DUP: 0 900038
record 368 BAD- PHONE: 9122326156 ROW: |0| DUP: 1 900038
record 460 BAD- PHONE: 7199643395 ROW: |1| DUP: 1 900038
record 490 BAD- PHONE: 8452593428 ROW: |0| DUP: 1 900038
record 524 BAD- PHONE: 2126951008 ROW: |0| DUP: 1 900038
record 614 BAD- PHONE: 6502259060 ROW: |0| DUP: 1 900038
record 634 BAD- PHONE: 8582742300 ROW: |0| DUP: 1 900038
record 699 BAD- PHONE: 3188659272 ROW: |0| DUP: 1 900038
record 762 BAD- PHONE: 3202675808 ROW: |0| DUP: 1 900038
record 1088 BAD- PHONE: 7039941336 ROW: |0| DUP: 1 900038
record 1223 BAD- PHONE: 2145632103 ROW: |0| DUP: 1 900038
record 1234 BAD- PHONE: 8478008799 ROW: |0| DUP: 1 900038
record 1256 BAD- PHONE: 8284410214 ROW: |0| DUP: 1 900038
record 1261 BAD- PHONE: 2128334966 ROW: |1| DUP: 1 900038
record 1276 BAD- PHONE: 4408125924 ROW: |0| DUP: 1 900038
record 1294 BAD- PHONE: 9703764057 ROW: |0| DUP: 1 900038
record 1440 BAD- PHONE: 2147393645 ROW: |0| DUP: 1 900038
record 1510 BAD- PHONE: 8475255282 ROW: |0| DUP: 1 900038
record 1544 BAD- PHONE: 6414371432 ROW: |0| DUP: 1 900038
record 1560 BAD- PHONE: 2033580066 ROW: |0| DUP: 1 900038
record 1587 BAD- PHONE: 9176849131 ROW: |0| DUP: 1 900038
record 1617 BAD- PHONE: 4408886044 ROW: |0| DUP: 1 900038
record 1691 BAD- PHONE: 6142054316 ROW: |0| DUP: 1 900038

Done GOOD: 1671 BAD: 23 TOTAL: 1694

STATUSES WITHIN THIS LIST:
STATUS STATUS NAME CALLED NOT CALLED
NEW New Lead 0 168
SUBTOTALS 0 168
TOTAL 168

ADMIN CHANGE LOG: Record Detail - 1612

ID: 1612
DATE TIME: 2009-12-09 04:55:55
USER: admin - Admin
IP: 192.168.0.17
SECTION: LISTS
TYPE: LOAD
RECORD ID: 900038
DESCRIPTION: ADMIN LOAD LIST
NOTES: File Name: Propecia.txt
SQL:

Output from SQL command select count(*) from vicidial_list where list_id='900038' ..

count(*)
168

Above are the stats !!

[williamconley]

Interesting.

Upload a "test" list with 1000 of the same record (and do not choose dedupe during the upload)

Obviously something is misfiring. Let's find out if it is misfiring from something IN the list or if it's the system.

Only phone, phone_code and country in the list you upload. makes it unlikely to have a flaw in the record.

Possibilities:

* Drive Full

* Record 167-169 has a *FLAW* of some sort, look at 'em in the download file -- use excel to identify the line #s, then open in notepad or nano or some such program that will not "interpret" anything for you, just show it.

[gmcust3]

Processing tab-delimited file...

LIST ID OVERRIDE FOR THIS FILE: 909090

Done GOOD: 1000 BAD: 0 TOTAL: 1000

STATUSES WITHIN THIS LIST:
STATUS STATUS NAME CALLED NOT CALLED
NEW New Lead 0 100
SUBTOTALS 0 100
TOTAL 100

ADMIN CHANGE LOG: Record Detail - 1614

ID: 1614
DATE TIME: 2009-12-09 05:19:12
USER: admin - Admin
IP: 192.168.0.17
SECTION: LISTS
TYPE: LOAD
RECORD ID: 909090
DESCRIPTION: ADMIN LOAD LIST
NOTES: File Name: 1000 TEST.txt
SQL:

Sample of 1 Record, copies 1000 times. { I took this from another Server A using export feature of webmin and uploading in Server B}

208609 11/19/2009 2:27 11/19/2009 23:02 AA VDAD 900025 -8 N 1 8185158XXX Richard Mertz 0000-00-00 mangz@hotmail.com Propecia 1 11/19/2009 1:21

* Drive Full

193.69 GB FREE

[williamconley]

ok, that shows 100

STATUSES WITHIN THIS LIST:
STATUS STATUS NAME CALLED NOT CALLED
NEW New Lead 0 100
SUBTOTALS 0 100
TOTAL 100

instead of 1000

what software do you have running on this box in addition to vicidial? have you done any upgrades/updates on the OS?

[gmcust3]

VERSION: 2.0.5-173
BUILD: 90320-0424

I have Voicetime USB but I did it long back and there was NO issues.

No other Extra Software installed in 7 days !!

Facing this problem since yesterday !

[williamconley]

but do you have other software on the box other than vicidial? (well, other than what came with vicidialnow ...)

not saying it's the issue. just hoping that there's a conflict someone will recognize. otherwise you have a "fluke" which i would have to look at personally or get deeper into than i am prepared to do today with my schedule on the forum. times up.

[gmcust3]

No Extra software.

Long back I tried CDR Analyser but then didn't install it.

[williamconley]

well, if it uploaded 100 out of the 1000 with a "test" list, you may want to look at reloading your system. or get into the script piece by piece and find out what is going awry.

[gmcust3]

Anyway to log it to find where its going wrong ?

[williamconley]

try sql logging find the queries that were run on the system. after all, the adding of the leads is a query ... did it ask for the import of all and some failed, or did it only ask for import of the ones that actually imported, in which case the script failed before that point.

if mysql shows only the 100 leads instead of the 1000 leads in the "insert" request, then run the script manually one piece at a time to see where if "falls through" and loses those other 900 leads.

[gmcust3]

Didn't understand !!

did it ask for the import of all and some failed, or did it only ask for import of the ones that actually imported, in which case the script failed before that point.

Added “log=/var/log/mysqld.log” under the [mysqld] section of the file “/etc/my.cnf“.

Lets see now !!

[williamconley]

i was referring to whether the problem was in vicidial script or mysql engine (or the actual mysql query itself). mysql log will break it down. did mysql create all the records it was asked to create? if so, the problem was before the mysql request, thus, in the vicidial script before that point. if not, the mysql query itself may be flawed or the script dropped a brain cell somewhere before that point

[gmcust3]

I did

Added “log=/var/log/mysqld.log” under the [mysqld] section of the file “/etc/my.cnf“.

and reboot the mysql.

But, alas, I don't see any entry in the log file.

Did I make any mistake in logging ?

[williamconley]

i do not believe mysql logs debug entries merely because it has been given a log file in which to store the data.

[gmcust3]

If you want to know what happens within mysqld, you should start it with --log[=file]. This will log all connections and queries to the log file (by default named `'hostname'.log'). This log can be very useful when you suspect an error in a client and want to know exactly what mysqld thought the client sent to it.

By default, the mysql.server script starts the MySQL server with the -l option. If you need better performance when you start using MySQL in a production environment, you can remove the -l option from mysql.server or change it to --log-binary.

The entries in this log are written as mysqld receives the questions. This may be different than the order in which the statements are executed. This is in contrast to the update log and the binary log which are written after the query is executed, but before any locks are released.

Thats why i added

log=/var/log/mysqld.log

[okli]

Does this help?

http://www.electrictoolbox.com/logging- ... ith-mysql/

In short- move log file location to /tmp and set the appropriate permissions and ownership.

[gmcust3]

091210 22:20:22 mysqld started
091210 22:20:23 InnoDB: Started; log sequence number 0 2928006
091210 22:20:23 [Warning] 'user' entry 'root@localhost.localdomain' ignored in --skip-name-resolve mode.
091210 22:20:23 [Warning] 'user' entry '(null)@localhost.localdomain' ignored in --skip-name-resolve mode.
091210 22:20:23 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.0.22' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
091210 22:25:55 [Note] /usr/libexec/mysqld: Normal shutdown

091210 22:25:55 InnoDB: Starting shutdown...
091210 22:25:57 InnoDB: Shutdown completed; log sequence number 0 2928006
091210 22:25:57 [Note] /usr/libexec/mysqld: Shutdown complete

091210 22:25:57 mysqld ended

091210 22:25:57 mysqld started
091210 22:25:57 InnoDB: Started; log sequence number 0 2928006
091210 22:25:57 [Warning] 'user' entry 'root@localhost.localdomain' ignored in --skip-name-resolve mode.
091210 22:25:57 [Warning] 'user' entry '(null)@localhost.localdomain' ignored in --skip-name-resolve mode.
091210 22:25:57 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.0.22' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
091210 22:29:19 [Note] /usr/libexec/mysqld: Normal shutdown

091210 22:29:21 InnoDB: Starting shutdown...
091210 22:29:23 InnoDB: Shutdown completed; log sequence number 0 2928006
091210 22:29:23 [Note] /usr/libexec/mysqld: Shutdown complete

091210 22:29:23 mysqld ended

091210 22:29:24 mysqld started
091210 22:29:24 InnoDB: Started; log sequence number 0 2928006
091210 22:29:24 [Warning] 'user' entry 'root@localhost.localdomain' ignored in --skip-name-resolve mode.
091210 22:29:24 [Warning] 'user' entry '(null)@localhost.localdomain' ignored in --skip-name-resolve mode.
091210 22:29:24 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.0.22' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution

mysqld.log :

Thats under tmp file.

But I don't see any query written...

I changed the permission to rwxrwxrwx !!