mflorell wrote:Bad should be that the phone number is less than 6 digits. This also could mean a formatting error for the record.
Matt,
Found this to be incorrect. The logic in the lead loader does not increment correctly for duplicates, and duplicates get marked as bad. The duplicate count appears to be off as well. I took a sample file and ran some bash commands against it to see what was going on while Kevan looked at the lead loading code. Here is what we found:
Kevan noticed the duplicate incrementing in the listloader was part of a loop. So if you increase the printout of bad errors from 10 to all, you will never see duplicate go over 1 in the last collumn, even when you find a duplicate 20 times in a file. So it gets 20 individual lines with the correct row, phone number, and duplicate as 1 in each error.
Also when finishing this file we would get a list of 117 Duplicates, and 1847 bad (when inserted into a new list). When we inserted into an existing list we would get 117 Duplicates and 2903 bad.
I then ran
- Code: Select all
cat TEST.TXT |cut -d'|' -f5 |sort -n | uniq | wc -l
This came out with 8153, matching the 1847 "bad" records when inserted into a new list. So the lead loader appears to increment the total BAD when they find a duplicate record. When examining the code above without the word count, i find that this file has about 275-325 duplicate leads, repeated for a total of 1847 times. I assumed 117 duplicates would be total leads that had duplicate records 1 or more times, however its obvious I am incorrect.
Finally when inserting into an existing list with prior leads we got 2903 Bad's, further look showed an additional 1056 were being marked bad as we had them from other vendors already existing in the file, however it still showed 117 duplicates even though we found another 1056 unique records as duplicates.
