10,000-core Linux supercomputer built in Amazon cloud

High-performance computing expert Jason Stowe recently asked two of his engineers a simple question: Can you build a 10,000-core cluster in the cloud?

In effect nice round number

"It's a in effect nice round number," says Stowe, the CEO and founder of Cycle Computing, a vendor that helps clients gain fast and efficient access to the kind of supercomputing power taking everything into consideration reserved for universities and large technology organizations.

Cycle Computing had already built a few clusters on Amazon's Elastic Compute Cloud that scaled up to several thousand cores. However Stowe wanted to take it to the straightway level. Provisioning 10,000 cores on Amazon has probably been done numerous times, yet Stowe says he's not aware of anyone else achieving that number in an HPC cluster, meaning one that uses a batch scheduling innovation and runs an HPC-optimized application.

The customer that opted for the 10,000-core cloud cluster was biotech company Genentech in San Francisco, where scientist Jacob Corn needed computing power to examine how proteins bind to each other, in innovation that might in the long run lead to medical treatments. Compared to the 10,000-core cluster, "we're a tenth the size internally," Corn says.

Cycle Computing and Genentech spun up the cluster on March 1 a little afterwards midnight, based on Amazon's advice regarding the optimal time to request 10,000 cores. During Amazon offers virtual machine instances optimized for high-performance computing, Cycle and Genentech instead opted for a "standard vanilla CentOS" Linux cluster to save money, according to Stowe. CentOS is a version of Linux based on Red Hat's Linux.

The cluster ran for eight hours at a cost of $8,500

The cluster ran for eight hours at a cost of $8,500, including all the fees to Amazon and Cycle Computing.

For Genentech, this was cheap and easy compared to the alternative of buying 10,000 cores for its own data center and having them idle away with no work for most of their lives, Corn says. Using Genentech's existing resources to perform the simulations would take weeks or months instead of the eight hours it took on Amazon, he says. Genentech benefited from the high number of cores because its calculations were "embarrassingly parallel," with no communication between nodes, so performance stats "scaled linearly with the number of cores," Corn said.

The cluster

Cycle Computing boasted that the cluster was in broad outline equivalent to the 114th fastest supercomputer in the world on the Top 500 list, which hit about 66 teraflops. In reality, they didn't run the speed benchmark required to submit a cluster to the Top 500 list, nevertheless nearly all of the systems listed below No. 114 in the ranking contain fewer than 10,000 cores.

Genentech is after all waiting to see whether the simulations lead to anything useful in the real world, nevertheless Corn says the data "looks fantastic." He says Genentech is "very open" to building out more Amazon clusters, and Cycle Computing is looking ahead as so then.