ScaleMP (finally) glues together 128 Opteron servers

Many have tried. Newisys, Liquid Computing, Fabric7, 3Leaf Systems, and NUMAscale all took very serious runs at it, and in doing so far, four out of five of them have gone the way of all flesh. It is not a coincidence that these companies fail because they require clients to invest in expensive software that turned many Opteron nodes into a big, often virtualized, single system image.

Since its founding in 2003, ScaleMP has tried a different approach. Instead of using special ASICs and interconnection protocols to lash at the same time multiple server modes at the same time into a shared memory system, ScaleMP cooked up a special hypervisor layer, called vSMP, that rides atop the x64 processors, memory controllers, and I/O controllers in multiple server nodes. To put it more exactly than carve up a single system image into multiple virtual machines, vSMP takes multiple physical servers and - using InfiniBand as a backplane interconnect - makes them look like a giant virtual SMP server with a shared memory space. vSMP has its limits. It only runs on Linux and doesn't do Windows. And up until today, it was only supported on Intel's Xeon processors, not Opterons.

The before releases of vSMP could scale across 16 nodes and up to 4TB of aggregate main memory, nevertheless with vSMP Foundation 3.0, launched in May 2010, the company expanded the underlying hypervisor to support up to 128 nodes and 64TB of memory in a single image.

The vSMP hypervisor that glues systems at the same time is not for every workload, however on workloads where there is a lot of message passing between server nodes - financial modeling, supercomputing, data analytics, and similar parallel workloads. Shai Fultheim, the company's founder and chief executive officer, says ScaleMP has over 300 clients now. "We focused on HPC as the low-hanging fruit," Fultheim tells El Reg, "yet these days we are doing business analytics and virtualization consolidation."

ScaleMP needs to get Windows supported on vSMP just in case to adding Opteron support, which quite frankly would have been more useful two years ago when the Opteron 6100s came out. You could as well make the case that vSMP would be useful on skinnier Opteron 4100 nodes for certain kinds of workloads, like those that are sensitive to clock speed as so then as memory capacity.

On open industry network performance and power test for private and public data center clouds ethernet fabrics evaluating 10GbE switches.