EmployeeIIS 7 Web Farms and authentication
Problem we faced - Users were intermittently being asked for their credentials to access a site that was listed in the Local Intranet IE Zone, and many times authentication was being required for resources that were not required to have authentication. Neither behaviors were desirable for the end user on a windows/IE system.
What we were trying to do - take a single windows 2000 IIS server/intranet site and migrate it to an IIS 7 web farm behind a LTM.
Some requirements - All access to the content had to be authenticated using a fully qualified domain name(this allows external analytics to work and kerberos authentication to function properly).
Rough setup details
So we built a server for development that also passed through the LTM to test the configuration to be used in production. In IIS 7, I disabled the default site, removed all the non-Http bindings from the site, and bound the site to a higher port number, to avoid it turning on and causing conflicts. Sure I could remove it, but some applications have an affinity for installing themselves in the default web site, either the named default site or by the site ID, so I kept it, but in a state to keep everything happy. I created a new site, pointed it to the content directory root, made sure that the root had the appropriate local user group that IIS uses(IIS_IUSRS). Then I set up some other custom logging and some special ACLs for certain directories. I had the system generate static machine keys so that the viewstate and other inner workings would not conflict between the boxes if the users got passed from one to another server. I had IIS 7 export the configuration and used that in the shared configuration which I applied to the other server. While MS talks about using a UNC for shared config, I have found this problematic at best, so I use a local directory and then use DFS-R to maintain the files between the two servers.
Diagnostics
Tools used - Fiddler, Wireshark, setspn, and TCPDUMP
Within fiddler I could see multiple requests from the server to authenticate (401 responses). These requests had different server headers, one from IIS 7.5 and another from microsoft httpapi2.0. My friend Google led me down a dark alley suggesting that the non IIS server was a remnant from SQL Server Reporting Services, which was never installed explicitly. While on the phone with M$ support, they pointed out that the negotiate responses were all different in the 401s, which is a clear indication of an error. MS Support had me run SETSPN -x to look for duplicate SPNs in the domain. MS Support pointed out that the FQDN for my site was registered to two servers, and that was a large part of the problem I was facing.
The Fix
Removed the SPNs for the http/FQDN associated with the machine names.
Needs AD admin rights to run- SETSPN -D http/FQDN Machinename
In IIS, configure the app pool to run as a domain account(I would recommend a service account of some sorts).
Then at the web site level of you IIS console, go into the authentication feature, then click the windows auth, and in the right hand pane click the advanced settings. Then ignore the warning that you will be kiling puppies and performance, and uncheck the Use kernel-mode authentication.
Now repeat these steps on the config of the other servers if they are not using shared config.
Now go back to your AD admin, and have them run the correct SETSPN command -
SETSPN -A http/fqdn domain\ServiceAccountName
If you are in a larger domain, wait for AD to replicate these changes, and test away.
What is different in IIS 6 to 7 that affects this:
When using the kernel mode authentication(IIS 7 only), the web server ignores the app pool domain credentials by default. This allows the https.sys process to handle the auth rather than at the worker process level. By disabling the Kernel-mode auth, it now becomes a user process task that is done within the worker process (w3wp.exe this is where the app pool runs). This then makes the IIS 7 system run in a manner similar to IIS 6, and allows the domain account that the app pool uses to be part of the Kerberos process.
**An alternate fix that allows Kernel-mode authentication is to do some modifications of the applicationhost.config file, but I decided not to do this at this time. One mistake in config files and the entire site or server will fail to respond.
Dead-Ends I tried and why they did not work:
Tried to remove the microsoft httpapi 2.0 from the server. Yeah I removed a lot of software and then finally removed and reinstalled the IIS role. When calling the site by the server name, I never saw the rogue http server, so I thought I had somehow removed it. WRONG, so yeah about the Microsoft HTTPAPI2.0 server.... that is actually how the http.sys process advertises itself and it will respond when there is confusion about Kerberos or when all sites are stopped, but at least one app pool is alive. So there is no way to remove that...
Tried to remove the 401 responses from the Microsoft HTTPAPI2.0 server with an Irule. ok so based on the info directly above this, you can see the futility here. I tested it and the site stopped responding altogether.
Tried to blame other systems for injecting these responses or caching them:
Was it the Wan accelerator? network traces from my system and then from the LTM to the VS, showed the responses were coming from the LTM system.
Was it the LTM and possibly something stuck in RamCache? Disabled Ramcache, the responses persisted, so then isolated the traffic to one backend server and listened to the network traffic. Then I was able to isolate the fault at the server layer and had to turn to the vendor for support...
So there you have it. Hopefully someone can glean some knowledge that will assist them in setting up their systems properly.
