Cozi Tech Blog

Iframes: thinking outside the box

 

Iframes have their uses, but they are not easy to deal with.

I added some text advertisements to our product this week. The standard technique for including advertising is to use an iframe. This works well for banner ads which come in well-known sizes.

I immediately ran into a problem with text ads in an iframe: there's no easy way to apply CSS to the contents of the iframe. Styles do not cascade through the iframe barrier. Normally, this is what you want, a self-contained unit on the page. It's fine for a banner ad, which requires no styling, but Times Roman text is jarring in a page of Arial.

It's difficult, perhaps outright impossible, to inject styles into an iframe coming from another domain.

Another problem is knowing how big to make the iframe. They don't autosize and the ad text could be one or more lines long.

I needed another way.

Making an Ajax call to fetch just the raw data that I cared about (title, body copy, link) was the obvious answer. A little wad of JSON would be much easier to deal with than trying to style an iframe. Unfortunately, the XMLHTTPRequest object cannot make cross-domain calls. But I read up on JSONP last week, so I knew that I could inject a script tag into my HTML DOM and set the src attribute to the adserver.

jQuery makes this easy: jQuery.getScript injects the script tag and removes it after the script has loaded.

We uploaded a custom template to the adserver:

setAdText({
  "adTitle": "%%TITLE%%",
  "bodyCopy": "%%BODYCOPY%%",
  "clickUrl": "%%CLICKURL%%"
});

I put this call to invoke the adserver's template in my page:

$.getScript(adServerUrl + cache_busting_random_token());

And the setAdText handler in my HTML page looks like this:

function setAdText(data)
{
  console.log("setAdText: adTitle=[%s], bodyCopy=[%s], clickUrl=%s."
    data.adTitle, data.bodyCopy, data.clickUrl);
  // add the ad to the DOM
}

Problem solved.

Preload Ajax Data as JSON

[Posted by George V. Reilly]

Preloading Ajax data as JSON has helped improve the load time and perceived performance of our family software application. Most of the pages in our Web client are dynamically generated in the browser from a complex set of JavaScript and CSS, so we're always looking out for ways to make them appear more quickly.

Our Combiner Control has been a big win: it coalesces a large number of small files together, reducing the latency of loading the page.

A few months ago, our Home Page would make eight Ajax calls as it loaded, to fetch data to populate different parts of the page, such as the family calendar and shopping list panes. That behavior fell out of the modular design, but the two-connection limit forced these calls to be serialized, magnifying the latency of the page. Panes were initially rendered blank, with an all-too noticeable pause before the data appeared.

My first change was to aggregate the eight calls into one ‘mondo’ call. That helped.

// pseudo code
function BuildHomePage()
{
    CreateCalendarPane();
    CreateShoppingPane();
    // other panes
    GetAjaxData( "/Ajax/GetMondoData.ashx", RenderMondoData );
}

// Callback function, executed several hundred milliseconds later
function RenderMondoData(ajaxData)
{
    CalendarPane.Fill( ajaxData.calendarData );
    ShoppingPane.Fill( ajaxData.shoppingData );
    // other panes
}

Later, I had the insight that I should send all that Ajax data down in the initial payload of the page. Now, the page can be rendered immediately: we've eliminated the latency of a roundtrip. Not only does this reduce the overall load time for the page, it significantly improves the perceived performance of the page, as panes are rendered sooner and with full data.

It's simple to make this work in ASP.NET:

<script runat="server">
static string HomePageDataJson()
{
    DateTime startDate = DateTime.Now;
    DateTime endDate = startDate.AddDays(9);
    return AjaxPro.JavaScriptSerializer.Serialize(
        new HomePageData( startDate, endDate ) );
}
</script>

<script type="text/javascript">
var homepagedata = <%= HomePageDataJson() %>;
var homepage = new HomePage( );
RenderMondoData( homepagedata );
</script>

The HomePageDataJson() function in the server script block creates the same object that the Ajax endpoint would have, then serializes it into a JSON string, using AjaxPro. This function could also be declared in a CodeBehind .cs page.

In the client script block, this JSON string is used to initialize a var with a JavaScript literal, which is then passed down into the JavaScript code.

In principle, this could make the client-side JavaScript a little simpler as it no longer needs to break the processing into asynchronous steps. In practice, we still need those handlers when the user navigates through the dataset.

This JSON preloading technique could easily be adapted to other Ajax libraries and other Web servers.

A Way To Unit Test ASP.NET IHttpHandler Implementations

[Posted by Pavel Repin]

If you find yourself writing simple HTTP handler code that produces and consumes structured data (for instance, some RESTful application), you may wonder how to test it without fiddling with IIS or configuration files. Here's a trick to write pure unit tests that verify your IHttpHandler implementation does what you expect. By "pure unit tests", I mean test code that:

• works without configuration files (like web.config);
• needs no servers (like Cassini, IIS, or your own mock HTTP server with full ASP.NET pipeline that you were about to write and debug just before you stumbled upon this article);
• doesn't access file system (like ashx files);
• avoids globals (like HttpContext.Current).

The Unit Tests In Action

Pretend we have an IHttpHandler implementation, TagHandler, that lets browsers retrieve a list of tags (using GET requests) and post new tags (using POST requests). The handler responds to GET requests with an array of tag objects serialized as JSON. It also parses submitted data (JSON) in POST requests and creates tags based on that.

Here are the unit tests that verify the POST behavior. Note the use of a helper class, HttpHandlerTest, that does all the tricky bits such as setting up HttpContext instance, creating a fake request body, the session state, and more.

[TestFixture]
public class TagHandlerTestFixture {
    private Dictionary<String, Tag> tags;

    [SetUp] public void SetUp() {
        // the tags object is always empty at the start of each test case!
        tags = new Dictionary<String, Tag>();
    }

    [Test, ExpectedException(typeof (ArgumentException))]
    public void PostTagWithInvalidRequestContentType() {
        HttpHandlerTest http = HttpHandlerTest.ImitatePost(
            "tags.ashx", "{}", Encoding.UTF8);
        // Won't like text/xml, it wants application/json!
        http.Context.Request.ContentType = "text/xml";
        http.Execute(new TagHandler(tags));
    }

    [Test] public void PostTag() {
        HttpHandlerTest http = HttpHandlerTest.ImitatePost("tags.ashx", 
            "{\"name\":\"foo\",\"description\":\"a a a\"}", Encoding.UTF8);
        http.Context.Request.ContentType = "application/json";
        http.Execute(new TagHandler(tags));
        Assert.IsTrue(tags.ContainsKey("foo"));
        Assert.AreEqual("a a a", tags["foo"].Description);
    }
    
    // ... more tests
}

You start off with setting up either a fake GET or POST request with HttpHandlerTest.Imitate[Get/Post]() method, then you set up additional preconditions in the initialized HttpContext object accessible as a property on HttpHandlerTest instance. Then you pass the IHttpHandler instance to the Execute() method. Your handler does its work. And then you can verify your post-conditions using NUnit assertions.

Here are some more tests from the same fixture that exercise the GET behavior:

    [Test] public void GetTagsResponseIsApplicationJson() {
        HttpHandlerTest http = HttpHandlerTest.ImitateGet("tags.ashx");
        http.Execute(new TagHandler(tags));
        Assert.AreEqual("application/json",
                        http.Context.Response.ContentType);
    }

    [Test] public void GetNoTags() {
        HttpHandlerTest http = HttpHandlerTest.ImitateGet("tags.ashx");
        http.Execute(new TagHandler(tags));
        Assert.AreEqual("{}", http.Output);
        Assert.AreEqual(Encoding.UTF8.GetType(),
            http.Context.Response.ContentEncoding.GetType());
    }

HttpHandlerTest Internals

Under the hood, HttpHandlerTest creates an instance of HttpContext initialized with a custom version of SimpleWorkerRequest. ImitateGet gives you an instance of HttpHandlerTest geared for testing GET requests, which do not have any content in the request body. ImitatePost is for simulating POST requests that do have content in the request. It should be easy to add support for other HTTP methods like DELETE, HEAD, and PUT.

public class HttpHandlerTest
{
    // Methods
    protected HttpHandlerTest(string requestMethod, string requestBody,
        Encoding requestEncoding, string page, string query);
    public virtual void Execute(IHttpHandler testSubject);
    public static HttpHandlerTest ImitateGet(string virtualPath);
    public static HttpHandlerTest ImitatePost(string virtualPath,
        string requestBody, Encoding requestEncoding);

    // Properties
    public virtual HttpContext Context { get; }
    public virtual string Output { get; }
}

Links

1. Full source code for HttpHandlerTest with the example web app. Note: the example web app uses AjaxPro.JSON.2.dll and nunit.framework.dll which are bundled in the "lib" directory. You can get everything by: svn co http://codebackpack.googlecode.com/svn/tags/how-to-test-httphandlers
2. Of course once I checked that nobody else found solution just as easy as the one presented here, and wrote this post, I was immediately shown ("Thank you" GeorgeR) that there was already a post about this by Phil Haack and my solution looks very similar :(  Oh well, at least this proves that this is not such a bad idea after all.
3. Lutz Roeder's Reflector has proven very valuable in figuring out which methods of SimpleWorkerRequest to override.

Combining JavaScript and CSS files for Improved Performance

[Posted by Mark Atherton]

Summary

Cozi has developed an ASP.NET control to combine multiple JavaScript or CSS files into a single reference, thus improving performance. The control handles problems of cache expiry dates and versioning, is easy to use, and can be switched on and off for debugging and testing.

Background

We’d like to organize our JavaScript and CSS files logically, without having to worry about performance. For instance, as we write more and more object-oriented JavaScript, we might want to put each JavaScript class into its own file. However, doing so would increase the number of round trips between the browser and the Web server.

This performance penalty is especially bad for JavaScript files since each one may contain a document.write which could change the sense of everything after it—so the browser can only download one at a time.

Unfortunately each extra JavaScript file adds about 200ms to page load time—the exact time varies by a dizzying array of factors (DSL vs Cable, how far it is from the browser to the server, etc, etc).

CSS files are not quite as bad as page parsing does not have to stop whilst they are being downloaded; however, the page cannot be correctly rendered until they have all been received.

Caching can help mitigate this delay, although it has its own problems.

Caching

To get maximum end-user performance, a Web site needs to set expiration headers on as many files as possible, especially static web content like images, CSS, JavaScript, etc. These headers tell the browser that it does not need to check back with the server for an updated version of a given file for some defined period.

Once that period has expired (or if no expiration is set), the browser will typically send a 'conditional get' giving the 'last-modified-time' of the file. If the file has not changed, the Web server can respond with a '304 Not Modified' response and not send down the data again.

Unfortunately the browser still needs to wait for the response before continuing. Therefore, it is best if the browser has a copy of the file AND it knows that it can be used without referring back to the Web server.

Versioning

Caching can also interfere with file versioning.

The easiest way to enabling caching is to tell your Web server that a whole directory of files can be cached on the client for a week (or day, or whatever). Unfortunately when you edit one of the cached files and deploy the new version, you have a problem because existing clients will run with their locally cached copy of the file until they reach the end of the expiration period.

How we solved these problems at Cozi

At Cozi we are developing an increasingly rich Web-based tool for making life simpler for families, including a shared family calendar, shopping list management, and more.

Our engineers need to have quite a few JavaScript and CSS files so that several of us can work on the site without trampling on each other’s code. However, this created the performance problems described above.

Part of our solution was to combine CSS and JS files into groups that get downloaded as one blob, saving many round trips.

Our requirements for the combining tool were:

1. It had to be easy to use.

2. You must be able to switch it on and off for testing and debugging scenarios (on a page-view-by-page-view basis).

3. It must support long cache expiration dates on these combined files.

4. It must solve the versioning problem.

To implement the solution, we wrote an ASP.NET control that wraps around our <link> and <script> HTML statements.

For example here’s a simplified version what the CSS references for our home page looks like:

<client:CombinerControl runat="server">

<link rel="stylesheet" href="/styles/sIFR-screen.css" type="text/css"/>

<link rel="stylesheet" href="/styles/Cozi.css" type="text/css"/>

<link rel="stylesheet" href="/styles/FooterToolbar.css" type="text/css" />

...

</client:CombinerControl>

The combiner outputs a reference to an Http Handler which will serve the combined file (I've put in some line breaks to make this a little more readable):

<link rel="stylesheet" type="text/css"  ←
href="../Combiner/Combiner.ashx?ext=css ←
&ver=9b0d6e4e ←
&type=text%2fcss ←
&files=!styles*sIFR-screen*Cozi*FooterToolbar*" />

As far as the browser is concerned, this is just a reference to a CSS file. It neither knows nor cares that it is programmatically generated.

A few parameters are passed to the handler (Combiner.ashx). They are:

Parameter	Meaning
ext	The file extension of the files, this is to save having to pass the file extension for each file separately.
ver	This is how we handle versioning the CSS files. The combiner calculates the ‘version’ of the combined file by mushing together the last-modified-time of all the constituent files. If any one of the files changes, this parameter will change and the browser will request a new combined file.
type	This is the Content-Type the handler should return.
files	This is the list of files to be combined. It is somewhat compressed to reduce the length of the URL generated. ! means the following token is a new directory that applies until the next change of directory is seen. * means a file to be combined. ' replaces / for legibility, as / would have to be encoded as %2f

Here’s a similar code snippet combining JavaScript code:

<WebClientCode:CombinerControl ID="CombineScript" runat="server">

<script src="script/third-party/jquery.js" type="text/javascript"></script>

<script src="script/third-party/sifr.js" type="text/javascript"></script>

<script src="script/third-party/soundmanager.js" type="text/javascript"></script>

<script src="script/cozi_date.js" type="text/javascript"></script>

</WebClientCode:CombinerControl>

The combiner outputs a reference to an Http Handler which will serve the combined file:

<script src="../Combiner/Combiner.ashx?ext=js ←
&ver=59169b00 ←
&type=text%2fjavascript ←
&files=!script'third-party*jquery*sifr*soundmanager*!script*cozi_date*" ←
type="text/javascript"></script>

The combiner control meets our requirements thus:

1. It had to be easy to use

It is, you just wrap the files you want: <client:CombinerControl runat="server">

2. You must be able to switch it on and off for testing and debugging scenarios

By passing a parameter on the Query String you can turn this behaviour off for testing and the control will merely output the list of files as though the control itself didn’t exist.

3. It must support setting long cache expiration dates on these combined files

The handler that serves the combined file automatically sets a very long expiration on the combined file, no configuration is needed.

4. It must solve the versioning problem

Because browsers cache based on the full, case-sensitive URL of the resource, you can set ‘infinite’ expiration, as if any file changes, a different reference will be written out and the browser will request a new file.

Gotchas

As always, we ran into a few gotchas:

1. You can’t combine CSS files with different media tags; all the CSS files must have the same media type.
2. You must preserve the order of the files; both for script and CSS files, order can matter.
3. You must remove UTF-8 BOM marks.  It would be OK to have a BOM mark at the very start but many of our JS files had BOM marks at the start and when we combined them, these BOMs ended up in the middle of the combined output and caused the browsers problems as they were interpreted as strange characters.
4. It’s very hard to combine script files that take query string parameters; luckily it is quite unusual for static script files to take parameters so this wasn’t much of a limitation. 

Further work

Our combiner also manipulates the contents of the CSS and JS files for other reasons, unrelated to main work of combining the files. For example, we might blog about how we use the combiner to load balance and edge-cache our images in another post.

Related web links

http://www.thinkvitamin.com/features/webapps/serving-javascript-fast
Great article on combiner-type issues, albeit rather PHP- and Apache-centric.

http://developer.yahoo.com/performance/index.html#rules
Rules for high performance Web sites.

Multiple Firefox Profiles: Run Firefox 2 and 3 Side-By-Side, and More

[Posted by George V. Reilly]

I find it useful to have multiple Firefox profiles for developing and testing. A clean profile for testing allows you to replicate most users' environments, who don't install extensions. Running a development profile in a separate profile lets you restart the browser without messing with your default environment. You can also run Firefox 2 and Firefox 3 side-by-side in separate profiles.

I currently have the following profiles:

• default: I browse the web in this profile. Outlook opens links in this profile.

• dev: I do most of my development here. Firebug gets a heavy workout.

• test: A clean profile. I only install the ResizeIT extension here.

• Firefox3: The latest beta of Firefox 3. The other profiles are Firefox 2-based.

Creating a new profile

Follow the instructions here to create test and dev profiles. Essentially, run:

 "%ProgramFiles%\Mozilla Firefox\firefox.exe" -profilemanager -no-remote

and use Create Profile to create profiles called dev and test

Your profile.ini should now look something like this:

 [General]
 StartWithLastProfile=1

 [Profile0]
 Name=default
 IsRelative=1
 Path=Profiles/y1mds144.default
 Default=1

 [Profile1]
 Name=dev
 IsRelative=1
 Path=Profiles/k034naef.dev

 [Profile2]
 Name=test
 IsRelative=1
 Path=Profiles/f2akbryu.test

Running the new profiles

If you want to run two profiles side-by-side, you need to run firefox.exe with the -no-remote and -P <profile> options. This launches each profile in a separate process.

Either create a batch file that looks like this:

 @start "Firefox Test" "%ProgramFiles%\Mozilla Firefox\firefox.exe" -no-remote -P "test"

Or create a Desktop Shortcut where the Location looks like:

 "C:\Program Files\Mozilla Firefox\firefox.exe" -no-remote -P "dev"

More information: Complete list of Firefox command-line options.

Running Firefox 2 and Firefox 3 side by side

By default, Firefox 2 installs into %ProgramFiles%\Mozilla Firefox, while Firefox 3 Beta 4 installs into %ProgramFiles%\Mozilla Firefox 3 Beta 4.

They can be installed on the same computer, but you must take care to use different profiles. Not all Firefox extensions have been updated yet for Firefox 3.

When you install Firefox 3, make sure that you do not launch it automatically at the end of the installation. Instead, run:

 "%ProgramFiles%\Mozilla Firefox 3 Beta 4\firefox.exe" -profilemanager -no-remote

and use Create Profile to create a profile called Firefox3. Then, follow the instructions above to create a batch file or a desktop shortcut, adjusting the path to firefox.exe.

Transparent PNGs Can Deadlock IE6

[Posted by George V. Reilly]

Summary: Internet Explorer 6 does not support transparency in PNG images. The best-known solution is to use the DirectX AlphaImageLoader CSS filter. It's less well known that using AlphaImageLoader sometimes leads to a deadlock in IE6. There are two workarounds. Either wait until after the image has been downloaded to apply the filter to the image's style, or use the little-known transparent PNG8 format instead of the filter.

The First Set of Hangs

Back in September 2007, some of my colleagues at Cozi complained that IE6 would occasionally hang while loading the homepage of our web application. I took a look at some of these IE6 processes in Process Explorer and I could see that thread 0 (the UI thread) was stuck in MsgWaitForMultipleObjects.

I attached WinDbg to these hung IE6 processes and found a callstack like this:

 0:020> .symfix+
 No downstream store given, using C:\Program Files\Debugging Tools for Windows\sym
 0:020> .reload
 ................................................................................
 Loading unloaded module list
 .............
 0:020> ~0k
 ChildEBP RetAddr  
 0013bfec 7c90e9ab ntdll!KiFastSystemCallRet
 0013bff0 7c8094e2 ntdll!ZwWaitForMultipleObjects+0xc
 0013c08c 7e4195f9 kernel32!WaitForMultipleObjectsEx+0x12c
 0013c0e8 7e4196a8 user32!RealMsgWaitForMultipleObjectsEx+0x13e
 0013c104 7e211e76 user32!MsgWaitForMultipleObjects+0x1f
 0013c168 7e200b6b urlmon!CTransaction::CompleteOperation+0x140
 0013c1a4 7e1ef557 urlmon!CTransaction::Start+0x52c
 0013c228 7e1ef1b4 urlmon!CBinding::StartBinding+0x4d8     <<<<---
 0013d2c0 7e1ef07e urlmon!CUrlMon::StartBinding+0x1d8
 0013d2f8 6bdd9abe urlmon!CUrlMon::BindToStorage+0x67
 0013d55c 6be43a78 dxtrans!CDXTransformFactory::LoadImageWrapW+0xdc
 0013e600 77135d81 dxtmsft!CDXTAlphaImageLoader::put_Src+0x1b5
 0013e61c 77136390 oleaut32!DispCallFunc+0x16a
 0013e6ac 6be29b0b oleaut32!CTypeInfo2::Invoke+0x234
 0013e6dc 6be29c9a dxtmsft!ATL::CComTypeInfoHolder::Invoke+0x45
 0013e724 6be16448 dxtmsft!ATL::CComDispatchDriver::PutProperty+0x78
 0013e75c 6be210eb dxtmsft!CComPropBase::IPersistPropertyBag_Load+0x9b
 0013e770 6bde9513 dxtmsft!ATL::IPersistPropertyBagImpl<CDXTAlphaImageLoader>::Load+0x2a
 0013e788 6bde51a3 dxtrans!CDXTFilter::Load+0x1f
 0013e834 6bdea876 dxtrans!CDXTFilterBehavior::AddFilterFromBSTR+0x424
 0:020> du (0013c228 + 48)
 0013c270  "http://cozicentral.cozi.com/imag"
 0013c2b0  "es/clock/clock_2.png"

(0013c228 is the ChildEBP address beside urlmon!CBinding::StartBinding+0x4d8. I was eventually to learn that du (<StartBinding's ChildEBP> + 48) would give me the URL that was being requested. It took probing with dds to figure this out. Spending 10 years in the Windows Division at Microsoft left me with advanced skills in debugging weird failures.)

This happened intermittently, but it was more likely when the browser's cache had been flushed. The exact URL would vary, but it was always one of the PNG images used to draw the digital clock. We had a lot of transparent PNGs, but it always seemed to happen to the clock digits, which are loaded on demand. This image was always available: it was not a server problem. The hang would eventually time out, after 20 or 30 minutes. Until it did, you could do nothing with IE. It wouldn't repaint. You couldn't close the app. All you could do was kill it in Task Manager. When the UrlMon call finally did time out, the requested image would not have shown up, but otherwise the browser was fine. This happened only with IE6, never with IE7 or Firefox. This was very strange, and we really didn't know what to make of it.

It recurred sporadically, and not having any better ideas, we decided to replace the PNGs for the clock digits with transparent GIFs in the IE6-specific CSS. The PNGs use alpha transparency, to get antialiasing and the translucent reflection below the baseline. We were using filter:progid:DXImageTransform.Microsoft.AlphaImageLoader in IE6-specific CSS to get alpha transparency. Unlike modern browsers, IE6 does not support alpha transparency. Fully transparent pixels are rendered as an opaque blueish-gray. Hence, our use of the well-known AlphaImageLoader hack.

We couldn't bake the background into the clock digit images: we have cobranded editions with different backgrounds, so the background underlying the digits can vary with both cobrand and window size. GIFs may be transparent, but they have no notion of alpha transparency, so we had to remove the translucent reflection in the GIFs. Not as pretty, but acceptable.

Hangs, Redux

The problem went away for several weeks, but in late October, it came back with a vengeance, recurring multiple times a day. Some machines seemed immune, some could repro it with almost 100% success. There was no obvious commonality; it reproed on several builds of IE6. It had to be fixed: we could not ship a web application that sometimes hung IE6 hard.

I formed the hypothesis that the two-connection limit was contributing to the problem, since it looked like DirectX was bypassing WinInet and using UrlMon to fetch the image. At the time, we were making a lot of Ajax requests to fetch data for different portions of the homepage, and that somehow this was tying up WinInet.

Sure enough, raising IE's connection limit made the problem go away -- as far as we could tell. This wasn't an acceptable workaround. We could hardly tell our users, "By the way, our app tends to hang on Internet Explorer 6. Please make the following changes in your registry."

Commenting out the AlphaImageLoader filter from the CSS also made the problem go away, but the results looked horrible.

We weren't enthusiastic about converting all of our transparent PNGs to GIFs. Our design relies heavily on alpha blending and many of the softly blended border images, for example, would look bad without alpha transparency. GIFs are limited to a 256-color palette and a few of our images are very colorful. Moreover, there are a lot of images and maintaining two sets would be a pain. At that point, we were still trying to make our web application look just as good on IE6 as in IE7 and Firefox.

I had, of course, Googled extensively to see if anyone else knew what was going on. I didn't find anything that was of any use. I'm amazed that so few people seem to have encountered this. I don't have a really simple repro case, but what we were doing does not seem especially odd for a Web 2.0 application.

Enter Microsoft

So we contacted Microsoft.

Several of us are Microsoft veterans, so initially we made direct contact with the Internet Explorer team. They were able to reproduce the problem intermittently and asked us to open an official support incident.

Fairly quickly, they came up with a workaround of loading the images first and then applying the filter. This ensures that the image is in the browser's cache:

 <img src="…" onload="iePNGLoader.loadThis(this);" alt="…" width="…" height="…" />

 var iePNGLoader =
 {
   loadThis: function(img)
   {
     if (navigator.userAgent.indexOf("MSIE") >-1 && parseInt(navigator.appVersion) <= 6)
     {
       var pSrc = img.src;
       img.onload = null;
       img.src = "/…/shim.gif";
       img.style.filter = "progid:DXImageTransform.Microsoft.AlphaImageLoader(" +
               "enabled=true, src='" + pSrc + "')";
     }
   }
 };

Unfortunately, that's not adequate for our web application. We make heavy use of background-images in CSS; onload events fire only for IMG tags. Also, I could never get img.onload events to fire after windows.onload, and we insert a number of images into the DOM as a result of Ajax calls.

We needed a JavaScript workaround, because even if Microsoft were to hotfix the bug, there's no way that we could ensure that all of our users installed it.

To make a long story slightly shorter, I spent a lot of time in November, with help from the IE CPR engineer, trying to preload images for IE6. To be completely safe, it was necessary to preload all of our images, even though most of them would never be used. I ended up loading them into 1x1 divs with a negative z-index obscured behind our toolbar. When windows.onload fired, I walked through all the transparent PNGs, setting their style.backgroundImage attributes (or src, if node.tagName == 'IMG') to a transparent 1x1 GIF (shim.gif, above), and applying the AlphaImageLoader filter. Inserting all of these images into the DOM added a very noticeable overhead, but we were prepared to live with it, if it gave us a reliable workaround.

Satzansatz has a comprehensive list of the shortcomings of AlphaImageLoader. For example, we use the background-repeat attribute to vertically or horizontally tile border images, allowing our bordered regions to resize themselves to the page's dimensions. Filtered images do not tile. At best, you can stretch them with sizingMethod='scale'. This happened to look okay with our border tiles, but it certainly wouldn't in general.

Anyway, preloading worked flawlessly on our intranet. Then we deployed to a test server in the datacenter, and it all fell apart. Apparently, the increased network latency exposed a bug in my preloading implementation. It turns out that if you dynamically insert images into the DOM, they are loaded after windows.onload fires -- even if the script itself executes before windows.onload. (Statically declared images are loaded before windows.onload fires, of course.) Hence, I was applying the filter before the image had necessarily loaded.

PNG8 to the Rescue

We were on the verge of switching over to GIF images, when one of my colleagues pointed out the PNG8 format to us, which we had never heard of before.

The SitePoint article on PNG8 contains a good explanation and examples. Briefly, however, typical PNG images contain arbitrary RGBA values, while PNG8 images contain an indexed palette of at most 256 colors, each of which can have 8-bit Red, Green, Blue, and Alpha values. This is similar to GIF's indexed palette, but PNG8 allows 256 degrees of opacity in each palette color, while GIF's transparency is all or nothing.

IE6, it turns out, does support transparency in PNG8 images, but it's a binary transparency, just like GIFs.

Here, then are three different logos, each rendered three different ways.

Here are the logos as PNG32s in IE6, without the AlphaImageLoader filter. Transparent pixels are opaque. Evidently unacceptable.

Here are the PNG8s in Firefox. Note the alpha transparency around the Cozi logo on the left. Look at the smooth edges in the logos. You can see the subtle alpha blending around the bordered box above the left Cozi logo and Mr Potato Head.

And here are the PNG8s in IE6. All alpha transparency is rendered as complete transparency. Note the jaggies in the logos, and the unsubtle borders around the box.

Clearly, PNG8 gives visually inferior results on IE6, but it looks great on modern browsers.

Kevin Freitas also covers the PNG8 technique. The comments in his article indicate that both the GIMP and Photoshop are capable of producing indexed PNG images.

The Aftermath

Googling again in early 2008, I did find an explanation from mid-2006. (I don't know why I never found this before. My google-fu is excellent.)

   

   

    To understand the problem, here is an explanation from the IE team     (thanks Peter Gurevich!):    

   

• Each IE 6 window is a UI thread.

• The HTML page and any script you wrote for the page run in the UI thread. Therefore filters in your page or in script will download on the UI thread

• IE’s implementation of the AlphaImageLoader filter downloads images synchronously

• Synchronous loading of an image or successive images on the UI thread has the potential to hang the browser and adversely affect the user experience.

   

They recommend using transparent GIFs as a workaround. We recommend using PNG8s.

I don't know why the AlphaImageLoader deadlock hasn't been a problem for more people. If they're out there, they haven't been vocal. The AlphaImageLoader hack is well known. The IE CPR engineer knew of only one other case like ours.

We have since decided that we will no longer make heroic efforts to get our web application looking just as good on IE6 as it does in modern browsers. Quite apart from the extraordinary amount of pain we endured in the Affair of the Transparent PNGs, fighting with IE6 has been a huge timesink for us, especially when it comes to working around its CSS bugs.

A significant fraction of our users are still using IE6, so we have no choice but to support it. However, we no longer aim to achieve full parity.

Alas, the recent announcement that Microsoft will be making IE7 an auto-update turns out not to be a panacea. IE7 will replace IE6 on intranets, at the discretion of IT organizations, but not on the Internet at large.

Acknowledgements. Despite my well-founded distaste for IE6, I'd like to thank the IE team members who helped us out. The amazing deadlock photo at the top of this article is taken from Ned Batchelder, who apparently got it from Reddit.