How to Set up F5 Application Connector
Last week we covered the basic overview of Application Connector and this week we’ll look at how to set it up. Settle in, this is detailed. F5 Application Connector is made up of two components: The Proxy and the Service Center. Step One is to set up the Service Center on BIG-IP. A brief overview of the Service Center steps: Download Service Center template (rpm) file Provision iRules LX Enable iApps LX Install and deploy the Service Center First, let's go to the F5.Downloads.com and grab the template that we'll use to deploy the Service Center. It's an RPM file. Now we’re going to log into the BIG-IP and under System Resource Provisioning>Provision, set iRules LX to at least nominal. Now we're going to connect to the BIG-IP using SSH - in this example we’re using putty - and you're going to run this command to enable iApps LX. Now back to the config utility, we're going to click iApps>Package Management LX and if you don't see this menu you're going to need to restart the BIG-IP and then you'll see it. Now import the RPM file that you downloaded and then upload it. When it's done you go to Application Services>Applications LX. Now we're going to select the Application Connector Template... ...and here is the Service Center. We’re going to scroll to the bottom and add an application name and then save it. Now we're going to select the application and click Deploy. The ball next to the name should turn green. Now on to Step 2 - Setting up the Proxy. You can do this on a small Linux instance that's running in the cloud in the same virtual network as your application servers. Here are the steps for The Proxy: Download and deploy the Docker container file Create virtual server for Proxy traffic Add virtual server in the Service Center Add virtual server in the Proxy Authorize the Proxy in the Service Center Start by downloading the Docker container from downloads.f5.com. It's the one with the .tgz file extension and copy this tgz file to your proxy instance. We’re running Windows and using WinSCP so we’ll just copy it from our local machine over to the proxy instance. Now back on the proxy instance on the Linux instance, we’re going to load the file and run a command to deploy the Docker container. If you look at the command a little more closely you'll see that we need to tell it apart, which in this case we’re using port 8090 and we’ll give it a username and password. Again, in the setup guide you'll find all the details on all the parameters that you can use in this command. Now we can see that the deployment was successful and it's running. We go back to the BIG-IP and create a Virtual Server so that BIG-IP can accept incoming traffic from the proxy. This has to be on port 443 and for testing we're going to use the default client SSL profile. In the Service Center, we're going to add the Virtual Server like you're going to select it. Click Config Proxy Virtual Server and then pick the virtual server and Save. If we go back and look at the Virtual Server, you can see that has an iRule associated with it. That's how you know it was successful. Now we’ll going to log into the Proxy with the port we specified and if your Proxy is in the cloud, it is make sure that you have the security rules so that this port is open. Again, in this case we used port 8090. We login with the username and password that we gave it and then in the Service Center connections area we’re going to add the Proxy virtual servers’ public IP address. One last step is going to go back into the Service Center to authorize the Proxy and now you can see the Proxy in here. Now on to the Final Step of adding your Cloud Nodes. Here are the steps for The Cloud Nodes: Create pool and virtual server for application traffic Add the virtual server in the Service Center Create AWS IAM role Add node to the pool On the BIG-IP, we’re going to create a pool and select one of these application connector monitors. For now, the pool is empty and we create a virtual server for the application traffic, pointing to that pool. Now we go into the Service Center and we tell it. ‘hey this is my virtual server for application traffic.’ To automatically add notes to the Proxy - in the AWS example – we’re going to create an IAM role... ...and then associate it with the Proxy instance. Then we’re going to need to restart the Proxy and now we can go into the Proxy and see that I was authenticated by AWS. And there are the nodes! The list is showing both the Proxy instance and the application servers but they're all automatically published at BIG-IP. If we go back to BIG-IP, we can see the nodes in the Service Center. Then we can go to the pool and we can choose them from a list. They're displayed here but it's important to know that these nodes are not exposed to the Internet and it's as if the nodes are local to the BIG-IP. Congrats! You’ve configured and deployed F5’s Application Connector. You can watch the step through video here. ps Related: Application Connector OverviewX-Forwarded-For Log Filter for Windows Servers
For those that don't know what X-Forwarded-For is, then you might as well close your browser because this post likely will mean nothing to you… A Little Background Now, if you are still reading this, then you likely are having issues with determining the origin client connections to your web servers. When web requests are passed through proxies, load balancers, application delivery controllers, etc, the client no longer has a direct connection with the destination server and all traffic looks like it's coming from the last server in the chain. In the following diagram, Proxy2 is the last hop in the chain before the request hits the destination server. Relying on connection information alone, the server thinks that all connections come from Proxy2, not from the Client that initiated the connection. The only one in the chain here who knows who the client really is (as determined by it's client IP Address, is Proxy1. The problem is that application owners rely on source client information for many reasons ranging from analyzing client demographics to targeting Denial of Service attacks. That's where the X-Forwarded-For header comes in. It is non-RFC standard HTTP request header that is used for identifying the originating IP address of a client connecting to a web server through a proxy. The format of the header is: X-Forwarded-For: client, proxy1, proxy, … X-Forwarded-For header logging is supported in Apache (with mod_proxy) but Microsoft IIS does not have a direct way to support the translation of the X-Forwarded-For value into the client ip (c-ip) header value used in its webserver logging. Back in September, 2005 I wrote an ISAPI filter that can be installed within IIS to perform this transition. This was primarily for F5 customers but I figured that I might as well release it into the wild as others would find value out of it. Recently folks have asked for 64 bit versions (especially with the release of Windows 2008 Server). This gave me the opportunity to brush up on my C skills. In addition to building targets for 64 bit windows, I went ahead and added a few new features that have been asked for. Proxy Chain Support The original implementation did not correctly parse the "client, proxy1, proxy2,…" format and assumed that there was a single IP address following the X-Forwarded-For header. I've added code to tokenize the values and strip out all but the first token in the comma delimited chain for inclusion in the logs. Header Name Override Others have asked to be able to change the header name that the filter looked for from "X-Forwarded-For" to some customized value. In some cases they were using the X-Forwarded-For header for another reason and wanted to use iRules to create a new header that was to be used in the logs. I implemented this by adding a configuration file option for the filter. The filter will look for a file named F5XForwardedFor.ini in the same directory as the filter with the following format: [SETTINGS] HEADER=Alternate-Header-Name The value of "Alternate-Header-Name" can be changed to whatever header you would like to use. Download I've updated the original distribution file so that folks hitting my previous blog post would get the updates. The following zip file includes 32 and 64 bit release versions of the F5XForwardedFor.dll that you can install under IIS6 or IIS7. Installation Follow these steps to install the filter. Download and unzip the F5XForwardedFor.zip distribution. Copy the F5XForwardedFor.dll file from the x86\Release or x64\Release directory (depending on your platform) into a target directory on your system. Let's say C:\ISAPIFilters. Ensure that the containing directory and the F5XForwardedFor.dll file have read permissions by the IIS process. It's easiest to just give full read access to everyone. Open the IIS Admin utility and navigate to the web server you would like to apply it to. For IIS6, Right click on your web server and select Properties. Then select the "ISAPI Filters" tab. From there click the "Add" button and enter "F5XForwardedFor" for the Name and the path to the file "c:\ISAPIFilters\F5XForwardedFor.dll" to the Executable field and click OK enough times to exit the property dialogs. At this point the filter should be working for you. You can go back into the property dialog to determine whether the filter is active or an error occurred. For II7, you'll want to select your website and then double click on the "ISAPI Filters" icon that shows up in the Features View. In the Actions Pane on the right select the "Add" link and enter "F5XForwardedFor" for the name and "C:\ISAPIFilters\F5XForwardedFor.dll" for the Executable. Click OK and you are set to go. I'd love to hear feedback on this and if there are any other feature request, I'm wide open to suggestions. The source code is included in the download distribution so if you make any changes yourself, let me know! Good luck and happy filtering! -Joe
Three things your proxy can’t do unless it’s a full-proxy
Proxies are one of the more interesting (in my no-doubt biased opinion) “devices” in the network. They’re the basis for caching, load balancing, app security, and even app acceleration services. They’re also a bridge between dev and ops and the network, being commonplace to all three groups and environments in most data center architectures. But not all proxies are built on the same architectural principles, which means not all proxies are created equal. A large number of proxies are half-proxies while others are full-proxies, and the differences between them are what mean the difference between what you can and cannot do with them. In fact, there are three very important things you can do with a full-proxy that you can’t do with a regular old proxy. Before we jump into those three things, let’s review the differences between them, shall we? Half-Proxy Half-proxy is a description of the way in which a proxy, reverse or forward, handles connections. Basically it’s describing the notion that the proxy only mediates connections on the client side. So it only proxies half the communication between the client and the app. The most important thing to recognize about a half-proxy is that it has only one network stack that it shares across both client and server. Full-Proxy By contrast, a full-proxy maintains two distinct network stacks – one on the client side, one of the app side – and fully proxies both sides, hence the name. While a full-proxy can be configured to act like a half-proxy, its value is in its typical configuration, which is to maintain discrete connections to both the client and the server. It is this dual-stack approach that enables a full-proxy to provide capabilities that a half-proxy with its single network stack simply cannot. The Three Things A full-proxy completely understands the protocols for which it proxies and is itself both an endpoint and an originator for those protocols and connections. This also means the full-proxy can have its own TCP connection behavior for each network stack such as buffering, retransmits, and TCP options. With a full-proxy each connection is unique; each can have its own TCP connection behavior. This means that a client connecting to the full-proxy device would likely have different connection behavior than the full-proxy might use for communicating with servers. Full-proxies can look at incoming requests and outbound responses and can manipulate both if the solution allows it. #1 Optimize client side and server side Because it can maintain separate network stacks and characteristics, a full-proxy can optimize each side for its unique needs. The TCP options needed to optimize for performance on the client side’s lower-speed, higher-latency network connection – particularly when mobile devices are being served – are almost certainly very different than those needed to optimize for performance on the server side’s high-speed, low latency data center network connection. A full-proxy can optimize both at the same time and thus provide the best performance possible in all situations. A half-proxy, with its single network stack, is forced to optimize for the average of its connections, which certainly means one side or the other is left with less than optimal performance. #2 Act as a protocol gateway Protocol gateways are an important tool in the architect’s toolbox particularly when transitioning from one version of an application protocol to another, like HTTP/1 to HTTP/2 or SPDY. Because a full proxy maintains those two unique connections, it can accept HTTP/2 on the client side, for example, but speak HTTP/1 to the server (app). That’s because a full-proxy terminates the client connection (the proxy is the server) and initiatives a different connection to the server (the proxy is the client). The protocol used on the client side doesn’t restrict the choice of protocols on the server side. Realistically, any protocol transition that makes sense (and even those that don’t) can be managed with a full-proxy. A programmable full-proxy ensures that even if its an uncommon (and thus not universally supported) that you can code up a gateway yourself without expending effort on reinventing the proxy-wheel. #3 Terminate SSL/TLS Technically this is a specialized case of a protocol gateway but the ascendancy of HTTP/S (and the urgency with which we are encouraged to deploy SSL Everywhere and Encrypt All The Things) makes me treat this as its own case. Basically terminating SSL/TLS is a critical capability in modern and emerging architectures because of the need to inspect and direct HTTP-based traffic (like REST API calls) based on information within the HTTP protocol that would otherwise be invisible thanks to encryption. The ability to terminate SSL/TLS means the proxy becomes the secure endpoint to which clients connect (and ultimately trust). Termination means the proxy is responsible for decrypting requests and encrypting responses and is thus able to “see” into the messages and use the data therein to make routing and load balancing decisions. So the next time you’re looking at a proxy, don’t forget to find out whether it’s a full proxy or not. Because without a full-proxy, you’re limiting your ability to really take advantage of its capabilities and reaping the benefits it can offer modern and emerging application architectures.Lightboard Lessons: Microsoft AD FS Web Application Proxy Using F5 BIG-IP
Many users and organizations want the flexibility and convenience of identity federation and Single Sign-On (SSO) from the corporate network to intranet, extranet, and cloud applications. Users want touse their corporate login information to access applications outside of the organization...for example, login from the Microsoft/Windows environment and have seamless access to Office 365 and external applications like Salesforce and Concur. Microsoft utlilizes a WebApplication Proxy (WAP) that acts as agateway product to allow external users to access internal applications (behind the firewall), like Active Directory Federation Services (AD FS) for example. The F5 BIG-IP can be used to providehigh availability, performance, and scalability for both AD FS and AD FS Proxy servers using theLocal Traffic Manager (LTM) module. The same BIG-IP can also be usedto secure AD FS traffic without the need for AD FS Proxy servers by using the Access Policy Manager (APM) module. Check out the video below to learn more! Related Resources: Deploying F5 with Microsoft Active Directory Federation Services Active Directory Federation Services Content MapWhat is a Proxy?
The term ‘Proxy’ is a contraction that comes from the middle English word procuracy, a legal term meaning to act on behalf of another. You may have heard of a proxy vote. Where you submit your choice and someone else votes the ballot on your behalf. In networking and web traffic, a proxy is a device or server that acts on behalf of other devices. It sits between two entities and performs a service. Proxies are hardware or software solutions that sit between the client and the server and does something to requests and sometimes responses. The first kind of proxy we’ll discuss is a half proxy. With a Half-Proxy, a client will connect to the proxy and the proxy will establish the session with the servers. The proxy will then respond back to the client with the information. After that initial connection is set up, the rest of the traffic with go right through the proxy to the back-end resources. The proxy may do things like L4 port switching, routing or NAT’ing but at this point it is not doing anything intelligent other than passing traffic. Basically, the half-proxy sets up a call and then the client and server does their thing. Half-proxies are also good for Direct Server Return (DSR). For protocols like streaming protocols, you’ll have the initial set up but instead of going through the proxy for the rest of the connections, the server will bypass the proxy and go straight to the client. This is so you don’t waste resources on the proxy for something that can be done directly server to client. A Full Proxy on the other hand, handles all the traffic. A full proxy creates a client connection along with a separate server connection with a little gap in the middle. The client connects to the proxy on one end and the proxy establishes a separate, independent connection to the server. This is bi-directionally on both sides. There is never any blending of connections from the client side to the server side – the connections are independent. This is what we mean when we say BIG-IP is a full proxy architecture. The full proxy intelligence is in that OSI Gap. With a half-proxy, it is mostly client side traffic on the way in during a request and then does what it needs…with a full proxy you can manipulate, inspect, drop, do what you need to the traffic on both sides and in both directions. Whether a request or response, you can manipulate traffic on the client side request, the server side request, the server side response or client side response. You get a lot more power with a full proxy than you would with a half proxy. With BIG-IP (a full proxy) on the server side it can be used as a reverse proxy. When clients make a request from the internet, they terminate on the reverse proxy sitting in front of application servers. Reverse proxies are good for traditional load balancing, optimization, SSL offloading, server side caching, and security functionality. If you know certain clients or IP spaces are acceptable, you can whitelist them. Same with known malicious sources or bad ranges/clients, you can blacklist them. You can do it at the IP layer (L4) or you can go up the stack to Layer 7 and control an http/s request. Or add a BIG-IP ASM policy on there. As it inspects the protocol traffic if it sees some anomaly that is not native to the application like a SQL injection, you can block it. On the client side, BIG-IP can also be a forward proxy. In this case, the client connects to the BIG-IP on an outbound request and the proxy acts on behalf of the client to the outside world. This is perfect for things like client side caching (grabbing a video and storing locally), filtering (blocking certain time-wasting sites or malicious content) along with privacy (masking internal resources) along with security. You can also have a services layer, like an ICAP server, where you can pass traffic to an inspection engine prior to hitting the internet. You can manipulate client side traffic out to the internet, server side in from the internet, handle locally on the platform or or pass off to a third party services entity. A full proxy is your friend in an application delivery environment. If you'd like to learn more about Proxies, check out the resources below including the Lightboard Lesson: What is a Proxy? ps Related: Lightboard Lessons: What is a Proxy? Encrypted malware vs. F5's full proxy architecture The Concise Guide to Proxies The Full-Proxy Data Center Architecture Three things your proxy can't do unless it's a full-proxy Back to Basics: The Many Modes of ProxiesLightboard Lessons: What is a Proxy?
The term ‘Proxy’ is a contraction that comes from the middle English word procuracy, a legal term meaning to act on behalf of another. In networking and web traffic, a proxy is a device or server that acts on behalf of other devices. It sits between two entities and performs a service. Proxies are hardware or software solutions that sit between the client and the server and do something to requests and sometimes responses. In this Lightboard Lesson, I light up the various types of proxies. ps Related: Encrypted malware vs. F5's full proxy architecture The Concise Guide to Proxies The Full-Proxy Data Center Architecture Three things your proxy can't do unless it's a full-proxy Back to Basics: The Many Modes of ProxiesMicroservices – application fragments need application services
Microservices are a cool way to build applications – cool in terms of the attention they get in blogs, marketing materials and conference speaking slots, but also cool in terms of genuinely offering a way to build applications that are more agile and robust than the monolithic architectures we have become used to. But really what are microservices? They are just fragments of an application. All the things we need from our monolithic applications we need from our fragments. Security, availability, performance. Each microservice needs to be up and running, have capacity and not be compromised if the integrity of the whole is to be preserved. Sure, there may be some services that are less critical than others, but in general you want all your services secure, fast, and available. How can you achieve this? Well there are number of ways, but the easiest is to reuse a common design pattern from the monolithic stack – an application proxy. Application proxies allow you to insert a layer of logic and control between the microservice client and the server. Pretty basic stuff. But by de-coupling the client from the server, you essentially virtualize the service. That means you can scale, protect and manage the service without disruption to the clients. You can provide capacity without complexity and security without slowing the application. Application traffic flow in microservices can be complex – multiple services need to be able to communicate to create the application. These complex flows can make application monitoring and security more complex to deliver – plus you need to code in client side discovery patterns if you need to scale out the number of instances supporting the service. Adding an application proxy – like a BIG-IP allow you to shrink or grow the processing power for a particular microservice – and add security, availability and optimization services inline. This can help you with segmentation, reporting and performance challenges. With a programmable, API-driven application proxy you can design a responsive system that is able to respond to changes in the environment and add additional application layer logic easily and without changing application code. With a strategic point of control like an application proxy - you can provide application services to your microservices, adding security and scalability while keeping the agility and performance you need.8 things you can do with a proxy
After having recently discussed all the different kinds of proxies that exist, it occurred to me that it might be nice to provide some examples of what you can do with proxies besides the obvious web filtering scenario. This is by no means an exhaustive list, but is provided to show some of the more common (and cool, I think) uses of proxies. What's really awesome is that while some of these uses are available with only one type of proxy (reverse or forward), a full proxy can provide all these uses, and more, in a single, unified application delivery platform. 1. DATA SCRUBBING Data scrubbing is the process of removing sensitive information like credit card and social security numbers from web application responses. This is particularly useful in preventing data leaks, especially if you're subject to regulations like SOX, HIPPA, and PCI DSS where the penalties for divulging personally identifiable information can be harsh fines - or worse. Data scrubbing is is an implementation of a reverse proxy. 2. URL REWRITING Rewriting URLs is something everyone has likely had to do at one time or another if they've developed a web application. URL rewriting is used to refer web requests to new resources instead of sending out a redirect response in cases where resources have moved, renamed, or migrated to a new version. URL rewriting is an implementation of a reverse proxy. 3. LAYER 7 SWITCHING Layer 7 switching provides an organization with the ability to maximize their IP address space as well as architect a more efficient, better performing application architecture. Layer 7 switching routes specific web requests to different servers based on information in the application layer, like HTTP headers or application data. Layer 7 switching is an implementation of a reverse proxy. 4. CONTENT FILTERING The most common use of proxies is content filtering. Generally, content filtering allows or rejects requests for content based on organizational policies regarding content type, the existence of specific keywords, or based on the site itself. Content filtering is an implementation of a forward proxy. 5. REDIRECTION Redirection is the process of, well, redirecting a browser to a new resource. This could be a new instance of a requested resource or as part of application logic such as redirecting a failed login to the proper page. Redirection is generally implemented by a reverse proxy, but can also be implemented by a forward proxy as a means of redirecting rejected requests to an explanation page. 6. LOAD BALANCING Load balancing is one of the most common uses of a reverse proxy. Load balancing distributes requests for resources across a number of servers in order to provide scalability and availability services. Load balancing is an implementation of a reverse proxy. 7. APPLICATION FIREWALL An application firewall provides a number of functions including some in this list (data scrubbing and redirection). An application firewall sits in front of web applications and inspects requests for malicious content and attempts to circumvent security. An application firewall is an implementation of a reverse proxy. 8. PROTOCOL SECURITY Protocol security is the ability of a proxy to enforce protocol specifications on requests and responses in order to provide additional security at all layers of the OSI stack. Protocol security provides an additional layer of security atop traditional security mechanisms that focus on data. Protocol security is an implementation of a reverse proxy.Whiteboard Wednesday: SSL Proxy Solutions
In this episode of Whiteboard Wednesday, Jason continues the proxy discussion from last episode with a breakdown of the ProxySSL and SSL Forward Proxy solutions available on the BIG-IP. Resources Proxy SSL Overview Implementing Proxy SSL Implementing SSL Forward Proxy SSL Forward Proxy Bypass F5 SSL Options Clarified! (External Article)
