Google Shaman Explains Mysteries Of 'Compute Engine'

Magnusson is the director of engineering for Compute Engine’s sister service, Google App Engine, and over the past eighteen months, as he spoke at various conferences and chatted with various software developers about Google’s place in the world of cloud computing, he couldn’t quite explain how serious the company is about competing with Amazon’s massively popular Elastic Compute Cloud and other commercial services that seek to reinvent the way online applications are built and operated.

The cloud computing game back in 2008

Google entered the cloud computing game back in 2008, when it unveiled Google App Engine, a service that lets outside software developers build and host applications atop the same sweeping infrastructure that runs Google’s own web services, just as Google Search and Gmail. Like Amazon’s cloud, this is a way of running online applications without setting up your own data center infrastructure. However it was difficult to tell whether the service was just one of those half-hearted Google experiments that would one day fall by the wayside. Although the service let you automatically accommodate an infinite amount of traffic — or thereabouts — it put tight restrictions on what programmers could and couldn’t do, and this seemed to limit its appeal.

What this means is that developers and businesses can grab a vast amount of processing power and apply it to nearly any task they want. Google is not only offering App Engine — a service that lets you build applications without having to worry about raw storage and processing power — it’s as well giving you, so then, raw storage and processing power. That is, it’s going to head-to-head with Amazon, the undisputed king of commercial cloud services that has long offered such raw resources as so then as “higher level” services for building and running massive applications.

The art of the “cloud” infrastructure

Google pioneered the art of the “cloud” infrastructure. However Amazon beat it to the idea of sharing such an infrastructure with the rest of the world. Six years afterwards Amazon first offered its web services to outside developers and businesses, Google is after all playing catchup. Yet it’s intent on making up that lost ground.

Google showed just how much it believes in Compute Engine, Magnusson says, when it tapped Hölzle to introduce the thing at its annual developer conference in San Francisco. Hölzle is the former UC Santa Barbara computer science professor who joined Google in early 1999 to oversee the growth on its internal network. At that time, the company had less than ten employees, nevertheless he ended up building a worldwide network of data centers that are among the most advanced on earth. Google vice president Sundar Pichai calls Hölzle “the person — more than anyone — responsible for building all of Google’s infrastructure.”

Hölzle and company describe the Google infrastructure as “warehouse-scale computing.” The idea is that each data center — running a common software platform — behaves like a single machine, running massive online applications and providing these applications with additional resources as needed. Google Compute Engine was built atop its existing software platform, taking advantage all the work that came previously.

Google has openly discussed part of its overarching software platform however not others. Hölzle declines to go into much detail, yet he does say that Compute Engines runs atop Google’s existing “server cluster management” service, which has long allowed Google internal engineers to rope at the same time CPU power and memory from across its network of servers and apply it to the task at hand. According to M.C. Srivas — a former Google engineer — this service is known as Borg.

Request for comment

Amazon did not respond to a request for comment. However Jason Hoffman — the chief innovation officer at Joyent, a cloud computing outfit that as well uses the KVM hypervisor to serve up virtual machines across the net — disputes Hölzle’s cost-per-dollar claims, saying that the Google’s price list indicates that Compute Engine in fact more expensive than Amazon or Joyent. “I just don’t get it,” he says.

However Google Compute Engine compares to the competition, Google is intent on making up lost ground against Amazon, whose services now run as much as one percent of the internet, according to one estimate. Compute Engine won’t replace App Engine. It will compliment App Engine. “You can use one or the other or both,” says Greg D'alesandre, who oversees App Engine. “We offered App Engine for a during, and what we realized is that every from time to time, there are going to be things that are simpler and more straightforward to do with VMs than to do with App Engine.”