Importing lists extremely slow on fast db server

[Infinity_ISF]

Hello,

I am writing because i downloaded and installed ViciBox.x86_64-4.0.1 on my database server and when i attempt to load lists they take forever. The installation has seemed to go great including installing the telephony servers. When i built this server it was to get the maximum read/write performance to eventually accommodate 10+ telephony servers.

Here is my configuration

Database server

Supermicro H8QG6-F motherboard
(4) Amd Opetron 6168 12 core processors
64gb ram
LSI 9265-8I Raid ( Direct read and write - tried 64 and 256k striping) no cacheing
(6) Samsung pro 840 SSD Hard drivers in raid 10

I have done simple benchmarks and get well over 300mb per second.

the purpose of this post is just to see if there is something out of the norm of a standard install i should be doing in order for vici/mysql to handle data at its full potential. All i can do is compare this database server above to HP G4 and HP G5 servers and they handle lists at about the same speeds.

Again, i am installing from the 64bit iso as per the instructions on vicibox.com/server - nothing special.


Any advice would be greatly appreciated. Thank you

[williamconley]

Speed of import is dependent upon the CPU speed mostly and also requires that the HD be able to keep up with the processors (not generally a problem, but sometimes). You have not mentioned the speed of the processors on the new system. On the other hand, since your previous servers had 3.6Ghz processors, I'd say it is very unlikely to be faster during import at all since you will not likely get a 12 core system faster than 3.6Ghz.

So ... it's not surprising that the speed does not increase assuming the processor speed does not increase.

More processors is only faster if multiple threads are used to accomplish the task. Which is is not, unless you load multiple lists simultaneously. That could (theoretically) allow faster loading.

[Infinity_ISF]

I see what you mean. The processors are 1.9ghz each. I understand that the "ghz" are lower but these processors are a much faster chip with higher bus speeds.

I just thought i was missing something seeing that i was going from 2 scsi drives to 6 raid 10 sata 3 SSD drives, not to mention a much better raid card.

So in the end, when you load a list you are only using one thread of one core of one processor?

[williamconley]

I understand that the "ghz" are lower but these processors are a much faster chip with higher bus speeds.

Um. No, they are not faster. 1.9Ghz is not faster than 3.6 Ghz. One processor is all that works on this task at a time, so "grouping" the processors is like saying my car goes faster than your motorcycle because it has four tires. LOL

As for more drives on a RAID, and a better RAID card, that will help when your system is under heavy load, but will not help during loading of leads as the i/o speed is apparently not your bottleneck. And if it were, you'd still only be likely to gain somewhere in the realm of 5% by increasing that capacity (unless you changed from 3G/sec to 6G/sec during this change ...).

One thread per uploading list, yes. So log on to two web browser sessions and load two lists at once.

Or my personal favorite: load with the command line lead loader. Much faster.

[chasejordan1]

Quick question; for a stand alone server which is faster or can handle more agents, 4 cores at 3.3ghz or 12 cores at 2.5ghz. I understand there is more involved, but hoping for a simple answer.

Thank you.

[williamconley]

12 cores at 2.5ghz

[chasejordan1]

Thank you very much