Thursday, May 28, 2009

TLS blackhole?

So today I was called by one of my clients to assist with a problem they were having accessing a secured website operated by a major business data processing company (you know, those big companies that do payroll, that sort of thing). One of their accountants reported that he could access the site from home, but not from the office, and the behavior suggested to them that the issue was a firewall issue of some sort, and since I do their firewall consulting they called me.

Now, I strongly doubted that it was a firewall issue, but the facts did fit that pattern (as they had tested accessing the site from a laptop on the outside of the firewall and that worked), so I made a trip to the client site to investigate further.

Investigation determined that the laptop in question actually worked from inside the firewall as well as outside, which flatly ruled out firewall issues. I noted that the laptop was not in the domain while the computers being used to test were inside, and suggested that we remove one of the nonfunctioning computers from the domain. This did not help, and we pursued other options. I also noted that the functioning machine was running IE6, while the nonfunctioning one was running IE7. (The complaining user states that he was using the same computer at home as at work, a traveling laptop, but I believe he misrepresented that fact to us and was actually using a different home computer, which caused me to waste considerable time investigating whether there was a domain authentication issue going on). Further investigation showed that the site could be accessed with Firefox, Opera, and IE6, but not with IE7, Google's Chrome, or Apple's Safari. (No machine with IE8 was available.) After using Process Monitor (which is a most excellent tool, although it does generate great gobs of output that takes some effort to sort through) to understand what was causing the hang. This having led to the determination that the web server not responding for some reason, I decided to sniff packets using Wireshark, freshly installed on the laptop. The network traces for sessions using IE6, Firefox, and Chrome were quite illuminating.

The site in question was hanging when being presented with an https session using TLS 1.0. TLS 1.0 is enabled by default and used preferentially by IE 7, Chrome, and Safari, but is not enabled by default in IE 6. Since the remote server's response to an attempt to negotiate a TLS 1.0 session is to hang (it sends a TCP ack to the TLS "hello" packet, but no further traffic), these browsers will wait indefinitely on the stuck connection. Firefox (and apparently also Opera) apparently have programmers who have run into sites like this and have coded defensively around them: Firefox also tries to use TLS 1.0, but when the TLS session times out, it retries the session with SSL 2.0/3.0, which the web server was happy to accept. IE 6, on the other hand, works merely because TLS 1.0 isn't enabled by default in this browser.

The "solution" for my client was to disable TLS 1.0 on the affected user's computer; not ideal, but it works. It would be nice if Microsoft made IE smarter (like Firefox), or if they gave me a way to disable TLS 1.0 only for this site; or if the Large Data Processing Company would just fix their server....