Autoscaling Strategies for Windows Azure

The strategies discussed in this article can be applied to any cloud platform that has an ability to dynamically provision compute resources, although I rely on examples from AzureWatch auto scaling and monitoring service for Windows Azure

The topic of auto scaling is an in the extreme important one when it comes to architecting cloud-based systems.  The major premise of cloud computing is its utility based approach to on-demand provisioning and de-provisioning of resources during paying only for what has been consumed.  It only makes sense to give the matter of dynamic provisioning and auto scaling a great deal of thought when designing your system to live in the cloud.  Implementing a cloud-based application without auto scaling is like installing an air-conditioner without a thermostat: one either needs to constantly monitor and manually adjust the needed cooling power or pray that outside conditions never change.

Many cloud platforms just as Amazon's EC2 or Windows Azure do not automatically adjust compute power dedicated to applications running on their platforms.  Instead, they rely upon various tools and services to provide dynamic auto scaling.  For applications running in Amazon cloud, auto-scaling is offered by Amazon itself via a service CloudWatch as so then as third party vendors just as RightScale.  Windows Azure does not have its own auto scaling engine however third party vendors just as AzureWatch can provide auto scaling and monitoring.

The goal of this article is to describe scaling strategies that gracefully handle unplanned and planned spikes in demand.  I'll use AzureWatch to demonstrate specific examples of how these strategies can be implemented in Windows Azure environment.  Important note: although this article will taking everything into consideration talk about scale up techniques, do not forget to think about matching scale down techniques. In part, it may help to think about building an auto scaling strategy after a fashion similar to building a thermostat.

Strategy 3: Predictable demand based just in time of dayThere are frequently situations when load on the application is known ahead of time.  Maybe it is between 7am and 7pm when a line-of-business application is accessed by employees of a company, or like as not it is while lunch and dinner times for an application that processes restaurant orders.  Whichever it may be, the more you know at what times while the day the demand will spike, the better off your scaling strategy will be.  AzureWatch handles this by allowing to specify scheduling aspects into execution of scaling rules.

Strategy 4: Predictable demand based on amount of work left to doWhile schedule-based demand predictions are great if they exist, not all applications have consistent times of day when demand changes.  If your application utilizes some sort of a job-scheduling approach where the load on the application can be determined by the amount of jobs waiting to be processed, setting up scaling rules based on such metric may work best.  Benefits of asynchronous or batch job execution where heavy-duty processing is off-loaded to back-end servers can not only provide responsiveness and scalability to your application however also the amount of waiting-to-be-processed jobs can serve as an important leading metric in the ability to scale with better precision.  In Windows Azure, the preferred supported job-scheduling mechanism is via Queues based on Azure Storage.  AzureWatch provides an ability to create scaling rules based on the amount of messages waiting to be processed in such a queue.  For those not using Azure Queues, AzureWatch can as well read custom metrics through a special XML-based interface.

Veteran software architect

Igor Papirov is a veteran software architect and developer, an MCSD since 1996, and a passionate promoter of cloud concepts. In 2010, Igor Papirov founded Paraleap Technologies, an emerging Chicago-based startup, focused on providing tools and services for cloud computing technologies. AzureWatch is Paraleap's flagship product, designed to add dynamic scalability and monitoring to applications running on Microsoft Windows Azure cloud platform.