Dell floats Cloud Hadoop clusters

"There are hundreds of settings for a Hadoop configuration, and this makes specific setting recommendations based on the hardware and software that clients are deploying on," Ed Albanese, head of business development at Cloudera tells El Reg. The settings for deploying on a small Hadoop cluster can be radically different from those needed on a mid-sized or large cluster, and SCM Express makes it possible for companies to get the right settings without having to be Hadoop experts like Yahoo!

Dell is as well tossing in its Crowbar tool, which it created for OpenStack-based private clouds running on its PowerEdge-C servers, which were announced last week. Crowbar works by common consent with SCM Express to do BIOS configuration, RAID array setup, network setup, operating system deployment, and manage the provisioning of Hadoop software on the bare metal iron from Dell.

Dell's reference architecture recommends that clients choose the company's PowerEdge-C 2100 energy efficient rack servers, which are optimized to cram the most computing performance in the smallest space with the least amount of extraneous hardware. This is the same iron that Dell used for its Ubuntu Enterprise Cloud pre-fabbed private clouds that came out in March, nevertheless is different from the PowerEdge C6100 cookie-sheet servers that are being used in the preconfigured OpenStack clouds launched last week.

The Dell-Cloudera stack is built in accordance with the assumption that companies will start out small however grow fast. It consists of six PowerEdge C2100 server nodes and six PowerConnect 6248 48-port Gigabit Ethernet switches. There are two Hadoop master nodes that manage the Hadoop Distributed File System and MapReduce task distribution across the cluster. These master nodes can as well run ZooKeeper. The edge node, of which there is only one in the base configuration, runs Pig and Hive and is the interface between users and the cluster. The slave nodes, of which there are three, run MapReduce and HDFS store data on local disk drives and chews on it as instructed by the master nodes.