Load Balancing in a Cloud
How should auto-scaling work, and why doesn’t it?
Although “rapid elasticity” is part of NIST’s definition of cloud computing, it may be interesting to note that many cloud computing environments don’t include this capability at all – or charge you extra for it. Many providers offer the means by which you can configure a load balancing service and manually add or remove instances, but there may not be a way to automate that process. If it’s manual, it’s certain “rapid” in the sense that’s it’s probably faster than you can do it (because you’d have to acquire hardware and deploy the application, as opposed to simply hitting a button and “cloning” the environment ‘out there’) but it’s not necessarily as fast as it could be (because it’s manual) nor is it automated. There’s a number of reasons for that – but we’ll get to that later. First, let’s look at how auto-scaling is supposed to work, at least theoretically.
1. Some external entity that is monitoring capacity for “Application A” triggers an event that indicates a new instance is required to maintain availability. This external entity could be an APM solution, the cloud management console, or a custom developed application. How it determines capacity limitations is variable and might be based solely on VM status (via VMware APIs), data received from the load balancer, or a combination thereof.
2. A new instance is launched. This is accomplished via the cloud management console or an external API as part of a larger workflow/orchestration.
3. The external entity grabs the IP address of the newly launched instance and instructs the load balancer to add it to the pool for resources for “Application A.” This is accomplished via the standards-based API which presents the configuration and management control plane of the load balancer to external consumers as services.
4. The load balancer adds the new application instance to the appropriate pool and as soon as it has confirmation that the instance is available and responding to requests, begins to direct traffic to the instance.
This process is easily reversed upon termination of an instance. Note: there are other infrastructure components that are involved in this process that must also be notified on launch and decommission, but for this discussion we’re just looking at the load balancing piece as it’s critical to the concept of auto-scaling.
The important thing to note here is that the process for adding – and subsequently using – the new application instance to the load balancer should be automatic. Once the instance is launched, all that remains is to inform the load balancer that a new instance is available. The load balancer should take care of the rest, and it should do so transparently. That means no interruption of service, no reboot, no reload the configuration, nothing. Similarly the termination and removal of the application instance from the pool of resources should be as seamless and non-disruptive. It does not matter whether the load balancing solution is hardware or a VNA (Virtual Network Appliance) as long as it has an API through which available resources can be added and removed on-demand.
This process is relatively straightforward and can even be accomplished without the “external entity” by directly integrating the application with the load balancing service. By hooking the appropriate setup and tear-down routines in the application, the load balancer can be automatically instructed to perform the appropriate actions whenever an instance of that application is launched or terminated.
1. Application is initialized and instructs load balancer to add it to the appropriate resource pool.
2. Load balancer adds the application instance to the right pool of resources and begins directing traffic to it as configured.
3. Application is terminating, instructs load balancer to remove it from the available resource pool.
4. Load balancer removes the application instance from pool of resources.
This requires tighter coupling of the load balancer with the application, however, which may be less than ideal unless you are very tied to your load balancing solution. The advantage of this solution is that regardless of whether the application is “physical” or “virtual”, the same process occurs, which lets you mix and match and be a lot more flexible in your architecture.