Why you still need layer 7 persistence
Tony Bourke of the Load Balancing Digest points out that mega proxies are largely dead. Very true. He then wonders whether layer 7 persistence is really all that important today, as it was largely implemented to solve the problems associated with mega-proxies - that is, large numbers of users coming from the same IP address. Layer 7 persistence is still applicable to situations where you may have multiple users coming from a single IP address (such as a small client base coming from a handful of offices, with each office using on public IP address), but I wonder what doing Layer 4 persistence would do to a major site these days. I’m thinking, not much. I'm going to say that layer 4 persistence would likely break a major site today. Layer 7 persistence is even more relevant today than it has been in the past for one very good reason: session coherence. Session coherence may not have the performance and availability ramifications of the mega-proxy problem, but it is essential to ensure that applications in a load-balanced environment work correctly. Where's F5? VMWorld Sept 15-18 in Las Vegas Storage Decisions Sept 23-24 in New York Networld IT Roadmap Sept 23 in Dallas Oracle Open World Sept 21-25 in San Francisco Storage Networking World Oct 13-16 in Dallas Storage Expo 2008 UK Oct 15-16 in London Storage Networking World Oct 27-29 in Frankfurt SESSION COHERENCE Layer 7 persistence is still heavily used in applications that are session sensitive. The most common example is shopping carts stored in the application server session, but it also increasingly important to Web 2.0 and interactive applications where state is important. Sessions are used to store that state and therefore Layer 7 persistence becomes important to maintaining that state in a load-balanced environment. It's common to see layer 7 persistence driven by JSESSIONID or PHPSESSIONID header variables today. It's a question we see in the forums here on DevCentral quite often. Many applications are rolled out, and then inserted into a load balanced environment, and subsequently break because sessions aren't shared across web application servers and the client isn't always routed to the same server or they come back "later" (after the connections have timed out) and expect the application to continue where they left it. If they aren't load balanced back to the same server, the session data isn't accessible and the application breaks. Application server sessions generally persist for hours as opposed to the minutes or seconds allowed for a TCP connection. Layer 4 (TCP) persistence can't adequately address this problem. Source port and IP address aren't always enough to ensure routing to the correct server because it doesn't persist once the connection is closed, and multiple requests coming from the same browser use multiple connections now, each with a different source port. That means two requests on the same page may not be load balanced to the same server, even though they both may require access to the application session data. These sites and applications are used for hours, often with long periods of time between requests, which means connections have often long timed out. Could layer 4 persistence work? Probably, but only if the time-out on these connections were set unreasonably high, which would consume a lot more resources on the load balancer and reduce its capacity significantly. And let's not forget SaaS (Software as a Service) sites like salesforce.com, where rather than mega-proxy issues cropping up we'd have lots-of-little-proxy issues cropping up as businesses still (thanks to IPv4 and the need to monitor Internet use) employ forward proxies. And SSL, too, is highly dependent upon header data to ensure persistence today. I agree with Tony's assessment that the mega proxy problem is largely a non-issue today, but session coherence is taking its place a one of the best reasons to implement layer 7 persistence over layer 4 persistence.Layer 7 Switching + Load Balancing = Layer 7 Load Balancing
Modern load balancers (application delivery controllers) blend traditional load-balancing capabilities with advanced, application aware layer 7 switching to support the design of a highly scalable, optimized application delivery network. Here's the difference between the two technologies, and the benefits of combining the two into a single application delivery controller. LOAD BALANCING Load balancing is the process of balancing load (application requests) across a number of servers. The load balancer presents to the outside world a "virtual server" that accepts requests on behalf of a pool (also called a cluster or farm) of servers and distributes those requests across all servers based on a load-balancing algorithm. All servers in the pool must contain the same content. Load balancers generally use one of several industry standard algorithms to distribute request. Some of the most common standard load balancing algorithms are: round-robin weighted round-robin least connections weighted least connections Load balancers are used to increase the capacity of a web site or application, ensure availability through failover capabilities, and to improve application performance. LAYER 7 SWITCHING Layer 7 switching takes its name from the OSI model, indicating that the device switches requests based on layer 7 (application) data. Layer 7 switching is also known as "request switching", "application switching", and "content based routing". A layer 7 switch presents to the outside world a "virtual server" that accepts requests on behalf of a number of servers and distributes those requests based on policies that use application data to determine which server should service which request. This allows for the application infrastructure to be specifically tuned/optimized to serve specific types of content. For example, one server can be tuned to serve only images, another for execution of server-side scripting languages like PHP and ASP, and another for static content such as HTML , CSS , and JavaScript. Unlike load balancing, layer 7 switching does not require that all servers in the pool (farm/cluster) have the same content. In fact, layer 7 switching expects that servers will have different content, thus the need to more deeply inspect requests before determining where they should be directed. Layer 7 switches are capable of directing requests based on URI, host, HTTP headers, and anything in the application message. The latter capability is what gives layer 7 switches the ability to perform content based routing for ESBs and XML/SOAP services. LAYER 7 LOAD BALANCING By combining load balancing with layer 7 switching, we arrive at layer 7 load balancing, a core capability of all modern load balancers (a.k.a. application delivery controllers). Layer 7 load balancing combines the standard load balancing features of a load balancing to provide failover and improved capacity for specific types of content. This allows the architect to design an application delivery network that is highly optimized to serve specific types of content but is also highly available. Layer 7 load balancing allows for additional features offered by application delivery controllers to be applied based on content type, which further improves performance by executing only those policies that are applicable to the content. For example, data security in the form of data scrubbing is likely not necessary on JPG or GIF images, so it need only be applied to HTML and PHP. Layer 7 load balancing also allows for increased efficiency of the application infrastructure. For example, only two highly tuned image servers may be required to meet application performance and user concurrency needs, while three or four optimized servers may be necessary to meet the same requirements for PHP or ASP scripting services. Being able to separate out content based on type, URI, or data allows for better allocation of physical resources in the application infrastructure.The Real News is Not that Facebook Serves Up 1 Trillion Pages a Month…
It’s how much load that really generates and how it scales to meet the challenge. There’s some amount of debate whether Facebook really crossed over the one trillion page view per month threshold. While one report says it did, another respected firm says it did not; that its monthly page views are a mere 467 billion per month. In the big scheme of things, the discrepancy is somewhat irrelevant, as neither show the true load on Facebook’s infrastructure – which is far more impressive a set of numbers than its externally measured “page view” metric. Mashable reported in “Facebook Surpasses 1 Trillion Pageviews per Month” that the social networking giant saw “approximately 870 million unique visitors in June and 860 million in July” and followed up with some per visitor statistics, indicating “each visitor averaged approximately 1,160 page views in July and 40 per visit — enormous by any standard. Time spent on the site was around 25 minutes per user.” From an architectural standpoint it’s not just about the page views. It’s about requests and responses, many of which occur under the radar from metrics and measurements typically gathered by external services like Google. Much of Facebook’s interactive features are powered by AJAX, which is hidden “in” the page and thus obscured from external view and a “page view” doesn’t necessarily include a count of all the external objects (scripts, images, etc…) that comprises a “page”. So while 1 trillion (or 467 billion, whichever you prefer) is impressive, consider that this is likely only a fraction of the actual requests and responses handled by Facebook’s massive infrastructure on any given day. Let’s examine what the actual requests and responses might mean in terms of load on Facebook’s infrastructure, shall we? SOME QUICK MATH Loading up Facebook yields 125 requests to load various scripts, images, and content. That’s a “page view”. Sitting on the page for a few minutes and watching Firebug’s console, you’ll note a request to update content occurs approximately every minute you are on a page. If we do the math – based on approximate page views per visitor, each of which incurs 125 GET requests – we can math that up to an approximation of 19,468 RPS (Requests per Second). That’s only an approximation, mind you, and doesn’t take into consideration the time factor, which also incurs AJAX-based requests to update content occurring on a fairly regular basis. These also add to the overall load on Facebook’s massive infrastructure. And that’s before we start considering the impact from “unseen” integrated traffic via Facebook’s API which, according to the most recently available data (2009) was adding 5 billion requests a day to that load. If you’re wondering, that’s an additional 57,870 requests per second, which gives us a more complete number of 77,338 requests per second. SOURCE: 2009 Interop F5 Keynote Let’s take a moment to digest that, because that’s a lot of load on a site – and I’m sure it still isn’t taking into consideration everything. We also have to remember that the load at any given time could be higher – or lower – based on usage patterns. Averaging totals over a month and distilling down to a per second average is just that – a mathematical average. It doesn’t take into consideration that peaks and valleys occur in usage throughout the day and that Facebook may be averaging only a fraction of that load with spikes two and three times as high throughout the day. That realization should be a bit sobering, as we’ve seen recent DDoS attacks that have crippled and even toppled sites with less traffic than Facebook handles in any given minute of the day. The question is, how do they do it? How do they manage to keep the service up and available despite the overwhelming load and certainty of traffic spikes? IT’S the ARCHITECTURE Facebook itself does a great job of discussing exactly how it manages to sustain such load over time while simultaneously managing growth, and its secret generally revolves around architectural choices. Not just the “Facebook” application architecture, but its use of infrastructure architecture as well. That may not always be apparent from Facebook’s engineering blog, which generally focuses on application and software architecture topics, but it is inherent in those architectural decisions. Take, for example, an engineer’s discussion on Facebook’s secrets to scaling to over 500 million users and beyond. The very first point made is to “scale horizontally”. This isn't at all novel but it's really important. If something is increasing exponentially, the only sensible way to deal with it is to get it spread across arbitrarily many machines. Remember, there are only three numbers in computer science: 0, 1, and n. (Scaling Facebook to 500 Million Users and Beyond (Facebook Engineering Blog)) Horizontal scalability is, of course, enabled via load balancing which generally (but not always) implies infrastructure components that are critical to an overall growth and scalability strategy. The abstraction afforded by the use of load balancing services also has the added benefit of enabling agile operations as it becomes cost and time effective to add and remove (provision and decommission) compute resources as a means to meet scaling challenges on-demand, which is a key component of cloud computing models. In other words, in addition to Facebook’s attention to application architecture as a means to enable scalability, it also takes advantage of infrastructure components providing load balancing services to ensure that its massive load is distributed not just geographically but efficiently across its various clusters of application functionality. It’s a collaborative architecture that spans infrastructure and application tiers, taking advantage of the speed and scalability benefits afforded by both approaches simultaneously. Yet Facebook is not shy about revealing its use of infrastructure as a means to scale and implement its architecture; you just have to dig around to find it. Consider as an example of a collaborative architecture the solution to some of the challenges Facebook has faced trying to scale out its database, particularly in the area of synchronization across data centers. This is a typical enterprise challenge made even more difficult by Facebook’s decision to separate “write” databases from “read” to enhance the scalability of its application architecture. The solution is found in something Facebook engineers call “Page Routing” but most of us in the industry call “Layer 7 Switching” or “Application Switching”: The problem thus boiled down to, when a user makes a request for a page, how do we decide if it is "safe" to send to Virginia or if it must be routed to California? This question turned out to have a relatively straightforward answer. One of the first servers a user request to Facebook hits is called a Load balancer; this machine's primary responsibility is picking a web server to handle the request but it also serves a number of other purposes: protecting against denial of service attacks and multiplexing user connections to name a few. This load balancer has the capability to run in Layer 7 mode where it can examine the URI a user is requesting and make routing decisions based on that information. This feature meant it was easy to tell the load balancer about our "safe" pages and it could decide whether to send the request to Virginia or California based on the page name and the user's location. (Scaling Out (Facebook Engineering Blog)) That’s the hallmark of the modern, agile data center and the core of cloud computing models: collaborative, dynamic infrastructure and applications leveraging technology to enable a cost-efficient, scalable architectures able to maintain growth along with the business. SCALABILITY TODAY REQUIRES a COMPREHENSIVE ARCHITECTURAL STRATEGY Today’s architectures – both application and infrastructure – are growing necessarily complex to meet the explosive growth of a variety of media and consumers. Applications alone cannot scale themselves out – there simply aren’t physical machines large enough to support the massive number of users and load on applications created by the nearly insatiable demand consumers have for online games, shopping, interaction, and news. Modern applications must be deployed and delivered collaboratively with infrastructure if they are to scale and support growth in an operationally and financially efficient manner. Facebook’s ability to grow and scale along with demand is enabled by its holistic, architectural approach that leverages both modern application scalability patterns as well as infrastructure scalability patterns. Together, infrastructure and applications are enabling the social networking giant to continue to grow steadily with very few hiccups along the way. Its approach is one that is well-suited for any organization wishing to scale efficiently over time with the least amount of disruption and with the speed of deployment required of today’s demanding business environments. Facebook Hits One Trillion Page Views? Nope. Facebook Surpasses 1 Trillion Pageviews per Month Scaling Out (Facebook Engineering Blog) Scaling Facebook to 500 Million Users and Beyond (Facebook Engineering Blog) WILS: Content (Application) Switching is like VLANs for HTTP Layer 7 Switching + Load Balancing = Layer 7 Load Balancing Infrastructure Scalability Pattern: Partition by Function or Type Infrastructure Scalability Pattern: Sharding Sessions Architecturally, Is There Such A Thing As Too Scalable? Forget Hyper-Scale. Think Hyper-Local Scale.I do not think that word means what you think it means
Greg Ferro over at My Etherealmind has a, for lack of a better word, interesting entry in his Network Dictionary on the term "Application Delivery Controller." He says: Application Delivery Controller (ADC) - Historically known as a “load balancer”, until someone put a shiny chrome exhaust and new buttons on it and so it needed a new marketing name. However, the Web Application Firewall and Application Acceleration / Optimisation that are in most ADC are not really load balancing so maybe its alright. Feel free to call it a load balancer when the sales rep is on the ground, guaranteed to upset them. I take issue with this definition primarily because an application delivery controller (ADC) is different from a load-balancer in many ways, and most of them aren't just "shiny chrome exhaust and new buttons". He's right that web application firewalls and web application acceleration/optimization features are also included, but application delivery controllers do more than just load-balancing these days. Application delivery controller is not just a "new marketing name", it's a new name because "load balancing" doesn't properly describe the functionality of the products that fall under the ADC moniker today. First, load-balancing is not the same as layer 7 switching. The former is focused on distribution of requests across a farm or pool of servers whilst the latter is about directing requests based on application layer data such as HTTP headers or application messages. An application delivery controller is capable of performing layer 7 switching, something a simple load-balancer is not. When the two are combined you get layer 7 load-balancing which is a very different beast than the simple load-balancing offered in the past and often offered today by application server clustering technologies, ESB (enterprise service bus) products, and solutions designed primarily for load-balancing. Layer 7 load balancing is the purvey of application delivery controllers, not load-balancers, because it requires application fluency and run-time inspection of application messages - not packets, mind you, but messages. That's an important distinction, but one best left for another day. The core functionality of an application delivery controller is load-balancing, as this is the primary mechanism through which high-availability and failover is provided. But a simple load-balancer does little more than take requests and distribute them based on simple algorithms; they do not augment the delivery of applications by offering additional features such as L7 rate shaping, application security, acceleration, message security, and dynamic inspection and manipulation of application data. Second, a load balancer isn't a platform; an application delivery controller is. It's a platform to which tasks generally left to the application can be offloaded such as cookie encryption and decryption, input validation, transformation of application messages, and exception handling. A load balancer can't dynamically determine the client link speed and then determine whether compression would improve or degrade performance, and either apply it or not based on that decision. A simple load balancer can't inspect application messages and determine whether it's a SOAP fault or not, and then once it's determined it is execute logic that handles that exception. An application delivery controller is the evolution of load balancing to something more; to application delivery. If you really believe that an application delivery controller is just a marketing name for a load-balancer then you haven't looked into the differences or how an ADC can be an integral part of a secure, fast, and available application infrastructure in a way that load-balancers never could. Let me 'splain. No, there is too much. Let me sum up. A load balancer is a paper map. An ADC is a Garmin or a TomTom.Making the most of your IP address space with layer 7 switching
Organizations trying to make their presence known on the Internet today run into an interesting dilemma - there's just not enough IP addresses to go around. Long gone are the days when any old organization could nab a huge chunk of a Class A or even Class B network. Today they're relegated to a small piece of a Class C, which is often barely enough to run their business. This is especially true for smaller businesses who are lucky if they can get a /29 at a reasonable rate. While we wait for IPv6 to be fully adopted and solve most of this problem (a solution that seems to always be on the horizon but never fully realized) there is something you can do to resolve this situation, right now. That something is layer 7 - or URI - switching, which is the topic on which a reader wrote for help this morning. A reader asks... Using the iRule we can choose the pool based on the URI, but how to choose the pool based on URL. It's a great question! Choosing pools based on URI, i.e. URI switching, is something we talk a lot about, but we don't always talk about the other, less exciting HTTP headers upon which you can base your request routing decisions. Basically, we're talking about hosting support.example.com and sales.example.com on the same IP address (as far as the outside world is concerned) but physically deploying them on separate servers inside the organization/data center. Because both hosts appear in DNS entries to be the same IP address, we can use layer 7 switching to get the requests to the right host inside the organization. (On a side note this is a function made possible by "server virtualization", one of the umpteen types of virtualization out there today and supported by application delivery controllers and load balancers since, oh, the mid 1990s.) Using iRules you can route requests based on any HTTP header. You can also route requests based on anything in the payload, i.e. the application message/request, but right now we're just going to look at the HTTP header options, as there are more than enough to fill up this post today. What's cool about iRules is that you can switch on any HTTP header, and that includes custom headers, cookies, and even the HTTP version. If it's a header, you can choose a pool based on the value of the header. Here's a quick iRule solution to the problem of switching based on the host portion of a URL. The general flow of this iRule is: when HTTP_REQUEST { switch [string tolower [HTTP::host]] { "support" { pool pool_1 } "sales" { pool pool_2 } }} If you'd like to switch on, say, the HTTP request method, you could just replace the HTTP::host portion with HTTP::method and adjust the values upon which you are switching to "get" and "post" and "delete". iRules includes an HTTP class that makes it easy to retrieve the value of the most commonly accessed HTTP headers, such as host, path, method, and version. But you can use the HTTP::header method to extract any HTTP header you'd like. HTTP::host - Returns the value of the HTTP Host header. HTTP::cookie - Queries for or manipulates cookies in HTTP requests and responses. HTTP::is_keepalive - Returns a true value if this is a Keep-Alive connection. HTTP::is_redirect - Returns a true value if the response is a redirect. HTTP::method - Returns the type of HTTP request method. HTTP::password - Returns the password part of HTTP basic authentication. HTTP::path - Returns or sets the path part of the HTTP request. HTTP::payload - Queries for or manipulates HTTP payload information. HTTP::query - Returns the query part of the HTTP request. HTTP::uri - Returns or sets the URI part of the HTTP request. HTTP::username - Returns the username part of HTTP basic authentication. HTTP::version - Returns or sets the HTTP version of the request or response. Even if you have a plethora of IP addresses available, the ability to architect your application infrastructure is made even easier if you have the capability to perform layer 7 switching on HTTP requests. It allows you to make better use of resources and to optimize servers for specific type of content. A server serving up only images can be specifically configured for binary image content, while other servers can be better optimized to serve up HTML and other types of content. Whether you have enough IP addresses or not, there's something to be gained in the areas of efficiency and simplification of your application infrastructure using layer 7 switching. For a deeper dive into HTTP headers (and HTTP in general) check out the HTTP RFC specification Imbibing: Coffee
