phpied.com

#5 This post is part of the Velocity countdown series. Stay tuned for the articles to come.

It's been over a year, since the launch of perfplanet.com. Looks good and useful for people so far. Sergey "ShowSlow" is doing also a great job of tweeting as @perfplanet about news from the perfplanet pipes as well as other interesting happenings in our perf community. Good stuff.

From the beginning I was sure I'm not including all blogs that deserve it. And new ones come up. So I said - send me an email, I'll look around the blog and add it to the planetarium. Problem with that is that the process was cumbersome. I don't always have the time (or am just being lazy because the process is kinda involved). Namely - update a yahoo pipe and update an html page with the list of blogs.

So I decided to take a few hours tonight to remedy the situation. I thought long and hard for what must have been a whole minute and the solution that came was - GitHub.

On GitHub

All the code is now updated, there's a bit of build process, minification and such and the code is now on GitHub, yeey. So if you send a pull request, I just accept it, run the build script and update the site.

Contributing

There is this "planetarium.json" file:
https://github.com/stoyan/perfplanet/blob/master/tools/planetarium.json

All you have to do is update this file, add your (or your favorite) blog for syndication and that's that. Or delete a spammy or irrelevant blog.

When adding a feed URL, try to find the "performance" related category. Because, sadly, not everyone is all that interested in other people's cats, as they are in other people's performance thoughts.

E.g. Ben Cherry's feed is:
http://feeds.feedburner.com/adequatelygood/
But we're only interested in the posts tagged "performance":
http://feeds.feedburner.com/adequatelygood/performance

There are exceptions, of course, some folks only talk about performance.

i18n

About internationalizations - talk to me. There is currently an fr.perfplanet.com, not necessarily maintained. But if you want to aggregate blogs in your language, you can just clone the github project, maintain your blogs and I'll setup your-lang.perfplanet.com and start pulling updates from github.

Build script

So first, I switched from Yahoo Pipes to YQL. Because the aggregation request can be generated from a list of URLs, no need to use a UI, login into Pipes, etc.

Other than that, I added a build script (in JavaScript, yes!) that does this:
- it takes the JSON list of blogs, an HTML template, CSS and JS
- generates index.html with inline minified CSS and JS using cssmin.js and jsmin.js. Gotta minify, gotta save requests
- also in the index.html there's a list of blogger names and URLs generated from the JSON
- generates an up.sh ("up" as in update) - this is a curl call with generated YQL query. This file is executed by a cron job every hour to read new blog posts and write data.js
- data.js is then used in index.html to display the content

Thanks!

So that's that. Hopefully this way the site will see much more updates with new fresh content and blogs.

Bugs, etc, welcome.

Contributions to the list of blogs (or anything else really) more than welcome.

Once again - this is the site and this is its GitHub.

#6 This post is part of the Velocity countdown series. Stay tuned for the articles to come.

Alrighty, this is something I've talked about last year at HighLoad++ and Fronteers, but never blogged. I came up with this thing while at Yahoo! Search and we used it there in production. So, it must be working. While probably not practical for many sites, the takeaway might very well be: optimizations are all around us, we just have to look around for them.

The summary: browsers have built-in search boxes which also give you search suggestions. IE8+ have added visual search suggestions, meaning images. The trick here is - when you don't have a visual suggestion, why not "suggest" an image for preloading, such as a sprite. Or two.

On to the details.

Chrome search

Every browser these days has them search boxes. Usually in the upper right. Like this:

(Aside rant: I call these browser search boxes "chrome search", but with the advent of the Chrome browser by Google and the fact that Google does search, this is now confusing, even doubly so. So next time you name a project or a company using a name that is already full of meaning such as Chrome or Closure, you're just being unimaginative and.. yeah, not cool. Unless of course it's something like Apple that's too far off from the actual product.)

Chrome search suggestions

Depending on how the search provider was set up, while you type the browser can send the partial string you typed to a URL of the search provider's choice. The search provider then can return a list of suggestions.

BTW, you don't have to be Google or Yahoo in order to be a search provider. This is "open search" and any site could (and frankly, should) be a search provider, here's an example how.

Visual chrome search suggestions

Starting with IE8, the provider can not only offer textual suggestions, but also visual ones, meaning images. So if you look up a stock symbol, it can give you a chart. Or a weather forecast.

The search provider sends back JSON or XML to the browser. The visual thing requires XML actually. Here's more from the horse's mouth. So in order to send an image back you use a syntax like so:

... 
<Item>
  <Text>Currently: Partly Cloudy, 67F</Text>
  <Description>High: 71F Low: 63F</Description>
  <Url>http://weather.yahoo.com/forecast/ USCA1024_f.html</Url>
  <Image source="http://path/to/my/image.png" 
         alt="Partly Cloudy" width="31" height="31"/>
</Item>
...

Preload in visual chrome search suggestions

So what if you don't have an appropriate image for a search term? Or maybe a partial term, say like for example "sch"? Why not load a sprite or some other image, hidden, 0x0 instead. Something that you can be reasonably confident the user will need once they get to the search results page. Easy-peasy:

... 
<Item>
  <Text>schmuck</Text>
  <Url>http://whatever.org</Url>
  <Image source="http://path/to/my/sprite.png" 
         width="0" height="0"/>
</Item>
...

This way you're in effect preloading something that will be needed later. I mean once the user starts typing, there's a pretty decent chance they'll end up on the results page. And they are likely to prefer a fast results page. It will be fast when pieces of it were pre-downloaded.

And if you inspect what's going on in HTTPWatch, you can see that as you type, the browser sends a request to the search provider, which then returns an XML document. And an image, although no image is seen.

What didn't work

I tried preloading CSS and JS too, but it didn't work. These show broken image icons, even with 0x0 size. I also tried 1x1 but then there's a 1x[LINE-HEIGHT] grey square/line.

And that's that

So here - a way to preload images for free, by simply using a browser feature.

You can also use this to preload 1x1 beacons for DNS resolution purposes. Say you serve search suggestions from suggest.example.org but the results are on www.example.org and the static stuff is on static1.example.org, static2.example.org, etc. There's an opportunity to pre-resolve all these domain names by requesting a tiny image from each.

Thanks for reading, this was the second post from the Velocity countdown. Only 5 more days to go! Hope you learned something, even if you don't intend to use it right away. Or maybe at least you've decided to offer your site as a search provider to all browsers. Free marketing, why not?

See you tomorrow.

#7 This post is part of the Velocity countdown series. Stay tuned for the articles to come.

Some time ago Google talked about using a sort of lazy JavaScript evaluation which especially helps mobile devices. The idea was to comment out a chunk of JavaScript you don't need right away and serve it this way. Later, when you need it, you get the content of the commented code and eval() it. More here and here.

At the last Fronteers conference I had the pleasure of chatting with Sergey Chikuyonok, who is so great and (among other things) is responsible for coming up with zen coding and writing a bunch of deep articles on image optimization for Smashing Magazine. So he told me he experimented with similar lazy HTML evaluation and it proved to be incredibly helpful for mobile devices. Not only the overall experience is faster but the initial rendering happens sooner and we all know how important that is.

Sergey is a busy person and chances of him writing about his experiment in English seemed pretty low at the time, so I decided to do an experiment on my own and see what happens. Meanwhile he did write about it so I forgot all about my findings, but here they are now.

Long document

I took one big HTML document - The adventures of Sherlock Holmes, which is half a megabyte or about 200K gzipped. Page A is the document as-is, plus some JS for measurements.

Page B (lazy) is the same page but with about 95% of its content commented out. The remaining 5% is a whole chapter so there's plenty of time to deal with the rest while the user is reading. After onload and a 0-timeout I take the commented markup (conveniently placed in <div id="lazy-daze">) and strip the comments. Then take the "unwrapped" time after another 0-timeout to let the browser repaint the DOM and regain control.

The overall skeleton of the lazy page is like so:

<!doctype html>
 
<html>
<body>
  <h1>THE ADVENTURES OF<br/>
  SHERLOCK HOLMES</h1>
  ...
  ... to chat this little matter over with you.</p>
 
  <div id="lazy-daze">
  <!--
    <p>II.</p>
    <p>
    At three o’clock precisely ... 
    ... she has met with considerable success.</p>
  -->
  </div>
 
<script>
 
 
window.onload = function () {
 
    setTimeout(function(){
 
        var daze = document.getElementById('lazy-daze'),
            inner = daze.innerHTML;
 
        daze.innerHTML = inner.substring(4, inner.length - 4);
    
        setTimeout(function(){
            // take end time... 
        }, 0);
                
    }, 0);
};
 
</script>
</body></html>

Experiment

All the files are here:
http://www.phpied.com/files/lazyhtml/

We have the plain normal document - http://www.phpied.com/files/lazyhtml/sherlock-plain.html
And the lazy one - http://www.phpied.com/files/lazyhtml/sherlock-lazy.html

In order to run the experiment you just go to
http://www.phpied.com/files/lazyhtml/start.html
And click "Go nuts". This will load each of the two documents 20 times and take a few time measurements. "Go nuts" again and you'll get 20 more data points.

The time measurements I take are:

• "plain" - unload to onload of the base version
• "lazy" - unload to onload of the lazy version NOT including unwrapping it. This should be quicker than the plain version
• "unwrapped" - unload to onload plus time to unwrap and rerender - this is expected to be bigger than "plain" because the browser has to render twice and is therefore doing more work
• DOM loaded "plain" - unload to DOMContentLoaded instead of onload
• DOM loaded "lazy"

Then I take the same 5 measurements but instead of starting at unload of the previous page, it starts at the top of the documents, as soon as a timestamp can be taken with JavaScript. This will exclude DNS, establishing connection, time to first byte...

Results

Here are the results from back when I did the experiment originally last year, using iPhone 2 (with iOS 3.2 or thereabouts)

I ran this experiment over Wifi and again over 3G.

First striking thing - it takes the about the same time to load the plain old page over Wifi and over 3G. For the smaller, "lazy" document, there is a difference, but there's virtually none for the plain base page. The guess here is that the rendering and its cost in terms of memory and CPU is far greater than the actual download time. In other words it takes longer to render than it does to download an HTML. At least in this class of phones. This guess is confirmed when you look at the time from the top of the documents, when the request overhead is removed:

With or without the request time - it's all pretty much the same.

The next striking thing - and how about that lazy document! It renders 3-4 times faster than the whole plain document. Not bad.

And one more surprise - lazy+unwrap time is less than the plain old document. Now that's interesting. It appears faster to split the task into two and do the whole double-rendering, which should've been slower because it's extra work. I guess that poor phone really chokes on the long document.

The same I found is true in Firefox, but almost the difference is negligible.

iPhone 4

I repeated the experiment tonight on iPhone 4 and wifi. And boy, is there a difference. What used to take 13 seconds is now under 3s.

The lazy + unwrap time is more than the plain time, which was to be expected.

Rendering that initial lazy document is still 2-3 times faster that waiting for the whole document.

The numbers:

• 2765 plain (2014 DOM)
• 1268 lazy
• 2995 lazy+unwrap

Ignoring the request overhead:

• 2200 plain (1421 DOM)
• 715 lazy
• 2423 lazy+unwrap

And one last run/observation - on the 3G and iPhone 4 there isn't much benefit of lazy-evaluation and empty cache. The request seems much more expensive. unload to onload 4.9s where document top to onload is 2.5. When the request overhead is out of the picture than lazy eval wins again - 1.7s compared to 2.5s

Parting words

• Lazy HTML FTW?
• Who the heck loads an entire book in a page?! Well it may happen. It may not be a whole book, but just a lot of markup. The entire book gzipped was 219K. A hefty document, but have you seen some of those news sites?
• Possible use case - blog comments. Lots and lots of blog comments. Or posts.
• If you're going to lazy-load something and get it with an ajax request, why not save yourself the request and ship with another chunk of html
• This was a simple layout task. Just a bunch of text. I'm guessing there could be much more complicated pages and layouts to render. And rendering is what takes the time it seems.
• Drawbacks a plenty because of the hidden content - accessibility, SEO.

Thoughts? Anyone want to run the test on Android or any other phone/device/tab/pad/whathaveyou? The guess is that the newer/powerful the device the smaller the difference. But it will be nice to know.

Let me try to set the mood for Velocity 2011 with an attempt to publish one performance-y post a day for the seven days between now and when the conference starts.

#7 - Lazy HTML evaluation
#6 - Preload in visual search suggestions
#5 - perfplanet.com is open
#4 - YSlow 2.0: the first sketches
#3 - Book of Speed
#2 - Sultans of Speed
#1 - Overlooked Optimizations: Images - guest post by Billy Hoffman

Tada! Velocity is here!

Sharding is the technique where you split static components across multiple domains so the browser can fetch more components in parallel? (Which browser? How many components per hostname? Ask browserscope.org)

So you have, say, image-a.png and image-b.png. You want to serve image-a from domain1.example.org and image-b.png from domain2.example.org

The thing is you don't want to randomize which image goes to which host in different page views. Otherwise you'll do like cnn.com where the same image (a 1x1 gif) is served from three different domains (in the same pageview even) causing three HTTP requests.

Here's a simple solution: use modulus to split components into buckets based on their URL (or path name).

Billy "Zoompf" Hoffman mentioned this as a simple strategy - if you want to split to two hostnames, take the length of the image path. If it divides by two - serve from host 2. Otherwise - host 1.

Same simple logic can apply for splitting into as many hostname buckets as you want. Need three buckets? % 3. Four buckets? % 4.

You can even do it per browser, for example if it's IE6, split to more buckets.

Here's the SSL (Simple Sharding Logic ) expressed in JavaScript:

function getBucket(url, numbuckets) {
  var number = url.length,
       group = number % numbuckets;
  return group;
}

If you think most of your paths will have same length, e.g. /images/top.png, /images/nav.png, /images/bot.png you can use some other number, e.g. the character code of the middle letter in the path.

var number = url.charCodeAt(parseInt(url.length/2));

Or the code of the letter in 1/4 of the path + the one in the middle + the on in 3/4 of the path. You get the point - all you need is a number that can be produced from the file path (or content or anything) and won't change from one page view to the next. You need stable file-to-hostname resolutions above all.

You can also pass an array of components and get a multi-array of the components, grouped into buckets. Here's what I tried on cnn.com and worked beautifully:

function toBuckets(stuff, numbuckets) {
  var numbuckets = parseInt(numbuckets, 10),
      url, group,
      buckets = Array(numbuckets),
      cache = {};
  for (var i = 0, max = stuff.length; i < max; i++) {
    url = stuff[i].src;
 
    if (typeof cache[url] === 'number') {
      continue;
    } 
    group = getBucket(url, numbuckets);
    if (!buckets[group]) {
      buckets[group] = [];
    }
    buckets[group].push(url);
    cache[url] = group;
  }
  return buckets;
}
 
console.log(toBuckets(document.images, 3));

This gave me (on cnn.com) a nice list of three buckets and the URLs in each:

[
  ["http://i2.cdn.turner.co...seals.02.cnn%5B1%5D.jpg", 
   "http://i2.cdn.turner.co...ves/tzvids.osama.gi.jpg", 
   "http://i.cdn.turner.com.../misc/advertisement.gif", 5 more...], 
  ["http://i.cdn.turner.com...der/hat/arrow_black.png",
   "http://i2.cdn.turner.co...ds.military.dogs.gi.jpg",
   "http://i2.cdn.turner.co...bl.house.cnn.120x68.jpg", 8 more...], 
  ["http://i.cdn.turner.com...element/img/3.0/1px.gif",
   "http://i.cdn.turner.com...bal/header/hdr-main.gif", 
   "http://i.cdn.turner.com...al/header/nav-arrow.gif", 9 more...]
]

Not too bad for such simplicity.

As you see, I have a cache object, so it takes care of any duplicates so I don't end up with the same component repeating.

The drawback of course is that it's unlikely that you'll get exactly the same number of components in each bucket. But from what I tried on a few sites, that's not a big issue. The benefit of having a stable and simple (no databases, cookies, local storage, etc) resolution of file path to a hostname is a win.

Another "brilliant" idea that I had recently - how about combining audio files into a single file to reduce HTTP requests, just like we do with CSS sprites? Then use the audio APIs to play only selected parts of the audio. Unlike pretty much all brilliant ideas I have, I decided to search for this one before I dive in. Turned out Remy Sharp has already talked about this. So I knew it was possible and wanted to check the server-side or things. (Remy is amazing, by the way, and I was happy to have him as a reviewer of "JavaScript Patterns")

Here's the demo - parts of Voodoo Chile covered by yours truly.

Playing separate files

Markup is a few audio elements:

<audio id="in">
  <source src="in.mp3">
  <source src="in.ogg" type="video/ogg">
</audio>
<audio id="1">
  <source src="1.mp3">
  <source src="1.ogg" type="video/ogg">
</audio>
<audio>
  ...

I have the files:

1. in.mp3 - intro
2. 1.mp3 - figure 1
3. 2.mp3 - figure 2
4. out.mp3 - the end

1 and 2 repeat a few times.

I play these files in JavaScript using a next() iterator function, which contains (in a private closure) the melody (which file after which) and a pointer to the current file being played. After play()-ing each audio, I subscribe to the "ended" event and play the next() file.

var thing = 'the thing';
var next = (function() {  
    //log('#: file');
    //log('-------');
    var these = ['in', '1', '2', '1', '2', '1', '2', '1', 'out'],
        current = 0;
    return function() {
        thing = document.getElementById(these[current]);
        //log(current + ': ' + these[current] + ' ' + thing.currentSrc);
        thing.play();
        
        if (current < these.length - 1) {
            thing.addEventListener('ended', next, false);
            current++;
        } else {
            current = 0;
        }        
    }
}());

That was easy enough. And worked (except on iPhone, see later). But we should be able to do better with sprites.

Sprite

For the sprite I have this audio:

<audio id="sprite">
  <source src="combo.mp3">
  <source src="combo.ogg" type="video/ogg">
</audio>

There's only one file - combo.mp3 which contains all the other four files played one after the other.

So we need to know the start and the length of each piece of audio. There are two parts to playing the sprite. First is knowing the lenghts and the "song" (meaning the succession of audios) and starting to play:

var sprites = {
      // id: [start, length]
       'in': [0, 3.07],
        '1': [3.07,  2.68],
        '2': [3.07 + 2.68,  2.68],
        out: [3.07 + 2.68 + 2.68, 11.79]
    },
    song = ['in', '1', '2', '1', '2', '1', '2', '1', 'out'],
    current = 0,
    id = song[current],
    start = 0,
    end = sprites[id][1],
    interval;
 
thing = document.getElementById('sprite');
thing.play();

Next is "listening" and stopping when one audio should be stopped, then seeking through the file and playing another part of it. This is done with a setInterval(), I couldn't find a better audio event to listen to.

// change
interval = setInterval(function() {
    if (thing.currentTime > end) {
        thing.pause();
        if (current === song.length - 1) {
            clearInterval(interval);
            return;
        }
        current++;
        
        id = song[current];
        start = sprites[id][0];
        end = start + sprites[id][1]
        thing.currentTime = start;
        thing.play();
        log(id + ': start: ' + sprites[id].join(', length: '));
    }
}, 10);

And this is it. The property currentTime is read/write - you can figure out where we are and also fast-forward or rewind to where you want to go.

Results

• Sprites play fine in FF, Chrome, O, Safari, iPhone's mobile webkit.
• I haven't tested IE9.
• All the browsers played y stuff off by a few milliseconds, I think some early, some late. This is probably due to unreliable setTimeout(). Also I didn't cut the audio pieces very well, so that might have someting to do. Also adding a few milliseconds of silence between the sprites may help. A follow up experiment will be to have a piano of sorts and see how timely the audio is played after a click/button press.
• iPhone didn't play properly the non-sprited verison - I believe because it won't let you autoplay unless there's a user action. There might be a workaround, but I only cared about the sprites and they are fine!

Server side

I was imagining the whole thing as a combo service like YUI's JS/CSS combo handler. The browser says: i need these 5 files, the server then creates a new audio file and sends it back. In this case it should also somehow send the data about start/length of each audio, so maybe a JSONP thing. I was mostly curious about those file formats and how the stitching would work.

In terms of file formats, it's not that bad, turns out all I need is MP3 and OGG in order to support all these browsers (I was prepared for worse).

(I could also probably support IE3? and above with a <bgsound> and a WAV, but the WAV is too big to be practical. So any IE (before 9) enthusiasm should probably end up in Flash.)

I recorded my audio pieces in Garage Band and exported as MP3.

ffmpeg is teh tool! It's like imagemagick for audio/video.

Cutting out extra 4-5 seconds Garage Band adds to each file you export:

$ ffmpeg -i in.mp3 -ss 0 -t 2.43 in-ok.mp3

(I didn't do that very precisely I think)

Converting MP3 to OGG is like:

$ ffmpeg -i in.mp3 -acodec vorbis -aq 60 -strict experimental in.ogg

Then the stitching.

MP3 files can actually be concatenated together just like JS/CSS, provided they have the same bitrate. I've done it in the past.

You can also combine by reading the files and piping them into ffmpeg. That somehow feels better:

$ cat in.mp3 1.mp3 2.mp3 out.mp3 | ffmpeg -i - combo.mp3

You can also consider putting a bit of silence between the separate audios.

In order to get the length info to return it to the client, you can use ffmpeg -i filename.mp3 (I haven't done that part)

OGGs cannot be concatenated like MP3, so the combo service should `cat` the mp3s as shown above, then convert to OGG (also shown )

Voila

You can now roll your own on-demand audio combo handler and use audio sprites to have fewer HTTP requests a more responsive app/game/html5 thing.

Warning: not practical blog post, don't read, move on.

So this is a post about a thought I had - creating rounded corners in IE678 by using roundness that they already have built-in, meaning the character O.

But first:

1. My opinion is that browsers that don't support border-radius should never ever get rounded corners. Let them rest in peace.
2. Don't use the technique below, it's just a thought. Plus it only has one corner

Demo

Live page demo and a screenshot in IE6:

Sales pitch

• Rounded corners in IE 6, 7, 8
• No images
• No JavaScript (a tiny self-rewriting CSS expression doesn't count)
• No extra markup

The drawbacks later.

The big idea

Use the letter O in monospace and position it in the corner.

Implementation

Markup

As promised, nothing to see here:

<p>Hello world</p>

JavaScript

None. *

* The catch here is that a tiny piece of JavaScript exists as a CSS expression. It's there to trade for clean markup. You can remove it, but then you need a bit of extra markup.

CSS

p {
  /* blah-blah, border, padding... */
  border-radius: 16px; /* for good browsers */
  background-image: expression(...); /* IE[678] */
}

The expression goes like:

this.runtimeStyle.backgroundImage="none",this.innerHTML += "<b>O</b>"

(Expression stolen from Thierry, btw)

The first part of the expression overwrites itself for performance reasons. You know that expressions are bad, cause they execute too often. Well, this.runtimeStyle shuts down the expression. If you wonder, runtimeStyle is IE thing which makes styles even more specific than inline style attributes. And this refers to the HTML element, in our case the P.

The second part of the expression (note the , separator, that's kinda funny) updates the innerHTML of the P adding a B element. So the end result of running the expression at initial page load is DOM like:

<p>Hello world<b>O</b></p>

And if you prefer, you can put that markup and get rid of the CSS expression.

The rest of the CSS is just wrestling to position the O in the corner:

b {
  background: white;
  display: block;
  font-family: monospace;
  font-size: 72px;
  font-weight: bold;
  height: 41px;
  left: -18px;
  overflow: hidden;
  position: relative;
  top: -74px;
  width: 25px;
}

And this is it.

Drawbacks

I'm sure my critical readers can think of drawbacks but let me start:

• In general principle, why would you care about rounded corners in browsers that don't know about border-radius?
• You can't have different background inside and outside the box, because you can't style the inside of the O with a color different than the outside. However you might be able to find a character that can.
• Playing with font sizes and positions is tricky. However there's probably a better way to position the O
• If you managed to select text outside the "hello world" (if you do Select-All for example) to copy, you'll paste "hello worldO" Which is exactly what screen readers will read and your page might sound like a weirdO

So there

Maybe someone else have already thought of the idea of using a character as a corner (and has a better implementation), but that's all from me. I'm not recommending this approach, just an itch I needed to put out there. Thanks for reading!

Background

• HTTPWatch (automation)
• ...with PHP (and again and again, and response)
• JavaScript shell scripting

I gave this short presentation at the recent Yahoo FE summit's open mic, here are the slides and some notes.

Slides and screencast vid

Screencast to see the thing in motion:

Notes

Here's the transcript of the slides as produced by slideshare.net. I've added some notes here and there to make it more readable when the presenter is missing.

JavaScript shell scripting - Presentation Transcript

1. JavaScript is everywhere #42:
         C:> WSH

2. Stoyan

   I do programming.

3. Programming

   There are many options to choose from when you decide top practice the
   art and craft of programming.

4. JavaScript

   ... is a very good one. Simply because JavaScript...

5. ... is everywhere

6. On the server

   node.js, asp.net

7. Mobile

   html5, phonegap, titanium

8. Desktop

   XULRunner lets you create cross-OS GUI apps

9. Browser extensions

   FF, Chrome, bookmarklets...

10. Photoshop

    yep, that too
    Several Adobe products actually let you script common/uncommon/programmable tasks

11. Form validation too!

    (this was supposed to be funny)

12. Shell scripting

    let's talk about shell scripting with JavaScript

13. In Windows

14. WSH: Windows Scripting Host

    All reasonable Windows machines (at least as old as Win2000) have this Windows Scripting Host in there.
    You can write VBScript or JavaScript (OK, JScript) to ... well, script.
    How does it work?

15. C:>edit hello.js

    You create a file.

16. var a = "Hello",
        b = " WSH!",
        c = a + b;
            
    WScript.Echo(c);

    Put any old JavaScript in there and print out some results

17. C:>cscript hello.js
    Hello WSH!

    And this is how you run it.

    Or this:

    C:>wscript hello.js

18. Open apps

    In addition to regular sysadmin tasks (copy, write files, move) you can open and script applications too.

19. var ie = new ActiveXObject("InternetExplorer.Application");
    ie.Visible = true;
    ie.navigate(yahoo.com);

    This is how you open IE and point it to a page.

    Notice something familiar? ActiveXObject - the thing we used in IE before it got native XMLHttpRequest

20. Firefox?

    Can you also open FF?

    Not really, as it doesn't have COM interface (whatever that is).

    But there's an easy solution

21. HTTPWatch

    Finally we come to the topic of the talk.

22. Speed

    Performance is critical for the success of any web app.

    Really, it is.

    When talking about improving speed there are two main steps:

23. 1. Fix with YSlow

    Take a slow page, run YSlow, do what it says.

    Voila - a fast(er) page.

24. 2. No regressions

    The second step is to not allow regressions.

    Whatever you fix in step 1 will be slow in the next few months.

    Even less than months the bigger the team or the rate of changes.

    So to prevent regressions, you need to constantly...

25. Monitor

26. Set limits

    The simplest way to prevent regressions is to set some limits.

    If you go over the limits, an email is sent, an alarm sounds, panic instills and you've got to fix whatever cause it was.

27. e.g.
    max 2 scripts
    max 2 styles
    max 9 images
    max 0 redirects

28. Scripting HTTPWatch

    Watching for violations of the limits manually every day is not a job anyone would want.

    So automating it will help a great way towards employee satisfaction

29. var http = new ActiveXObject("HTTPWatch.Controller"),
        ie = http.IE.New(),
        ff = http.Firefox.New();

    This is how you open IE and FF with HTTPWatch's help.

    FF - yey!

30. // browser cache
    ie.clearCache();
    
    // show HTTPWatch
    ie.OpenWindow(false);

    Examples of stuff you can do with the HTTPWatch API.

    You can for example hit the page with empty cache and then again with full cache.

    Best of all - these are the real browsers with their sometimes kinky behaviors.
    Actually if you setup several machines for the monitoring (or somehow do multiple IEs)
    you can test with different versions of the browsers. Nothing synthetic!

31. ie.Record();
    ie.GotoUrl("yahoo.com");
    http.Wait(ie, -1);
    ie.Stop();

    Start monitoring, go to a page, stop monitoring after the page "settles" meaning some time after onload.

    ie.CloseBrowser();

32. new HTTPWatch()
          http://github.com/stoyan/etc/

    I did this JavaScript thingie to make everything a little easier.

33. var http = new HTTPWatch(ff);
    http.go(search.yahoo.com);
    http.done();

    Example usage.

34. [video]

35. var har = http.toHAR();
    har = eval(( + har + ));
    
    print(har.log.browser.name);
    print(har.log.browser.version);
    print(# requests: );
    print(har.log.entries.length);

    Opening and navigating browsers is cool. But we need some data back.

    HTTPWatch can export a HAR (HTTP Archive) file. I have this toHAR() method.
    It writes the file, than reads and returns it.
    You can than eval() it because it's just a JSON string.
    And you get the data back in convenient JS objects and arrays.

36. Internet Explorer 6.0.29...
    # requests: 10
    
    Firefox 3.5.6
    # requests: 15

    Result of running the above.

37. [video]

38. var comps = http.getComponentsByType();
          
    for (var i in comps) {
      print(i);
      print(comps[i].length);
    }

    Another method I thought would be useful is getComponentsByType()

39. redirect: 1
    text/html: 3
    image/gif: 4
    image/png: 3
    text/javascript: 1

    Results of the code above.

40. But wait...

    There's more

41. What about DOM?

    So far we only talked about HTTP traffic inspection - headers and such.

    Good news is that you can also inspect the DOM (in IE only) for any potential red flags.

    For example having the number of DOM elements sharply increase.

42. [video]

43. var http = new HTTPWatch();
    http.go(search.yahoo.com);
    
    var d = http.watch.container.document;
    
    print(d.getElementsByTagName(*).length);
    print(d.documentElement.innerHTML);

    That works!

    All your DOM voodoo skillz are suddenly reusable.

44. require(statz.js);
          
    var doc = http.watch.container.document;
    var html = http.har.log.entries[0].response.content.text;
    
    var out = statz(document, html);
    print(out.join("\n"));

    This is me repurposing two old bookmarklets that gather some interesting stats (one of them was even featured on Ajaxian, remember?).

    It was pretty easy to repurpose the bookmarklets, because it's just JavaScript.

    The stats thingie can inspect both raw HTML that went over the wire, as well as innerHTML that was the result of any additional DOM manipulations.

45. JS attributes (e.g. onclick): 1207 bytes
    CSS style attributes: 883
    Inline JS: 5243
    Inline CSS: 5015
    All innerHTML: 17283
    # DOM elements: 134
    
    Total size: 14124 bytes
    Content size: 401 bytes
    Content-to-markup ratio: 0.03

    Sample results.

46. To summarize...

47. JavaScript
    WSH
    HTTPWatch
    Monitor
    DOM and HTTP
    IE and Firefox

48. Thanks
          
    phpied.com

If you haven't seen it yet, YSlow for Chrome hit the streets couple of weeks ago. (And Google's own PageSpeed did too yesterday. (And there's now DynaTrace for Firefox. (And WebPageTest for Chrome. (What a month for x-browsering (word?) the performance tools! (And the month's not even over yet)))))

BTW, I closed all the parentheses (or else...).

So anyway, I was eavesdropping on a twitter conversation where Sergey (of ShowSlow) was asking for beacons from YSlow for Chrome, more specifically - when will they start working. I thought I should check how my old baby YSlow 2.0 (this presentation is still pretty relevant) is doing in its new environment.

YSlow 2.0

In YSlow 2.0 things are pretty decoupled. Makes it easier to bring to any possible environment or browser. So rules are rules (you can add, remove, tweak them, combine them into rulesets), results are results, presentation is separate, and so are the additional tools, HAR import/export (forthcoming), etc. Only (ideally) small additions are needed to glue the core of YSlow (the linting part) with a new environment.

In Chrome

It's my first time touching anything Chrome-y, but turned out its pretty easy. Just a bit of file system hunting revealed where code for the extension goes.

/Users/[USERNAME]/Library/Application Support/Google/Chrome/Default/Extensions/[WEIRD-EXTENSION-ID]/

e.g.

/Users/stoyanstefanov/Library/Application Support/Google/Chrome/Default/Extensions/ninejjcohidippngpapiilnmkgllmakh/

On Windows:

C:\Documents and Settings\[USERNAME]\Local Settings\Application Data\Google\Chrome\User Data\Default\Extensions\ninejjcohidippngpapiilnmkgllmakh\

In there you can find three JavaScript files. I could be wrong but here's what I think goes in there:

1. yslow-chrome.js looks like it contains the reusable parts - rules, etc - packaged for Chrome minus the Firefox stuff and bundled into a single file
2. content.js is small and not too exciting
3. controller.js is the Chrome-related parts

In controller.js is where we hack.

YSlow events

In YSlow 2.0 we decided to make use of a simple observer pattern implementation and fire events whenever interesting stuff happens. Especially useful since figuring all the data for all resources better be asynchronous.

Once yslow "peels" the page figuring out the components, gets each component data, headers, etc, then runs the lint rules, finally it fires a lintResultsReady event.

("Peeling a page" I heard for the first time from Steve Souders and for what I know he should be credited with this term describing the activity of figuring all components that go into a page)

All we need to do is listen to this event and send the beacon.

YSLOW.util.event.addListener(
  'lintResultReady', 
  function (o) {
    //...b-b-beacon! ...
  }
);

There's a YSLOW.util.sendBeacon() which does precisely that, so we need to call it and we're done.

Preferences

Firefox has a built-in (native) system to manage preferences. You know, the stuff you tweak in about:config. This is where we put the preferences - beacon yes/no, beacon URL, beacon data verbosity.

In Chrome such native preference system probably exists, but YSlow is currently not taking advantage. (Just guessing here.)

Luckily all calls to get preferences are abstracted in YSLOW.util.Preferences.getPref(prefname, defaultvalue). The default value is returned if a better one cannot be found.

So we can just overwrite the getPref() method to return the default value, unless it's a preference we care about, such as the beacon URL:

YSLOW.util.Preference.getPref = function(what, defaultval) {
  switch (what) {
    case 'beaconUrl':
      return 'http://www.phpied.com/beacon.png';
    case 'beaconInfo':
      return 'all'; // or "basic"
    default:
      return defaultval;
  }
};

Integration

As mentioned we'll hack into the controller.js, we don't want to touch the yslow core stuff. The controller.js is just one immediate function and our hack goes right before the last line. (Or even after it, probably doesn't matter)

(function () {
    // ... yslowy stuff ...
 
    // hack start
    // ...
    // hack end
 
    doc.ysview.setSplashView(...
}());

The complete thing is something like:

(function () {
 
    // ... slo, slo ...
 
    // hack start
    YSLOW.util.Preference.getPref = function(what, defaultval) {
      switch (what) {
        case 'beaconUrl':
          return 'http://www.phpied.com/beacon.png';
        case 'beaconInfo':
          return 'all'; // or "basic"
        default:
          return defaultval;
      }
    };
 
    YSLOW.util.event.addListener('lintResultReady', function (o) {
      var con = o.yslowContext,
          result = con.result_set;
      YSLOW.util.sendBeacon(result.url, result.overall_score, con);
    });
    // hack end
 
    doc.ysview.setSplashView(...
}());

Conclusion

So there - you can run YSlow in Chrome and send yourself (or showslow.com) beacons.

You probably don't need that so bad that you can't wait till next YSlow for Chrome ships with this thing working. But here it is.

And hopefully you learned a bit about YSlow internals so that you can start hacking yourself and/or wait till YSlow shows up on github (soon!) and start sending diffs. I personally can't wait.

Shoutout goes out to Marcel and Betty who are doing awesome stuff with YSlow (slides). And looks like even more is to come!

Often we want to load a CSS file on-demand by inserting a link node. And we want to know when the file finished loading in order to call a callback function for example.

Long story short: turns out this is harder than it should be and really unnecessary hard in Firefox. I hereby beg on behalf of many frustrated developers: please, Firefox 4, please fire a load event when a stylesheet loads.

With that out of the way, let's see what we have here.

// my callback function
// which relies on CSS being loaded
function CSSDone() {
  alert('zOMG, CSS is done');
}
  
 
// load me some stylesheet
var url = "http://tools.w3clubs.com/pagr/1.sleep-1.css",
    head = document.getElementsByTagName('head')[0];
    link = document.createElement('link');
 
link.type = "text/css";
link.rel = "stylesheet"
link.href = url;
 
// MAGIC
// call CSSDone() when CSS arrives
 
head.appendChild(link);

Options for the magic part, sorted from nice-and-easy to ridiculous

1. listen to link.onload
2. listen to link.addEventListener('load')
3. listen to link.onreadystatechange
4. setTimeout and check for changes in document.styleSheets
5. setTimeout and check for changes in the styling of a specific element you create but style with the new CSS

5th option is too crazy and assumes you have control over the content of the CSS, so forget it. Plus it checks for current styles in a timeout meaning it will flush the reflow queue and can be potentially slow. The slower the CSS to arrive, the more reflows. So, really, forget it.

So how about implementing the magic?

  // MAGIC
  // #1
  link.onload = function () {
    CSSDone('onload listener');
  }
  // #2
  if (link.addEventListener) {
    link.addEventListener('load', function() {
      CSSDone("DOM's load event");
    }, false);
  }
  // #3
  link.onreadystatechange = function() {
    var state = link.readyState;
    if (state === 'loaded' || state === 'complete') {
      link.onreadystatechange = null;
      CSSDone("onreadystatechange");
    }
  };
  
  // #4
  var cssnum = document.styleSheets.length;
  var ti = setInterval(function() {
    if (document.styleSheets.length > cssnum) {
      // needs more work when you load a bunch of CSS files quickly
      // e.g. loop from cssnum to the new length, looking
      // for the document.styleSheets[n].href === url
      // ...
      
      // FF changes the length prematurely  )
      CSSDone('listening to styleSheets.length change');
      clearInterval(ti);
      
    }
  }, 10);
  
  // MAGIC ends

Test

Test page - riiiight here. I'm loading a CSS file delayed two seconds on the server. Attaching all those event listeners and timeouts above. Adding another timeout that just says "... and two seconds later ... " after (you guessed it!) two seconds. Now observing los resultados...

Results

• IE fires readystatechange and load (tested years ago, too lazy to test now again). Now with IE9 maybe addEventListener will work too?
• Firefox (like before) fires nothing. It updates the length of document.styleSheets immediately, not waiting for the file to actually arrive. So the outcome in my test log is:

  zOMG, CSS #1 is done: listening to styleSheets.length change
  ... and two seconds later ...
  

• Opera fires load via onload and via addEventListener too. Like FF it also increments document.styleSheets.length immédiatement. The outcome:

  zOMG, CSS #1 is done: listening to styleSheets.length change
  ... and two seconds later ...
  zOMG, CSS #1 is done: onload listener
  zOMG, CSS #1 is done: DOM's load event

• Chrome and Safari will not fire events but will update document.styleSheets only when the file arrives, yey!

  ... and two seconds later ...
  zOMG, CSS #1 is done: listening to styleSheets.length change

All in all, there's at least one way to tell when the stylesheet is loaded in each browser, except Firefox. Now that's embarrassing.

Is there really no hope for Firefox?

If you go really crazy than yes - implement magic #5 but it has serious drawbacks.

Otherwise the object trick should do - all browsers seem to fire load and/or readystatechange event consistently. Obviously it's a little more complicated. Although probably not as complicated as monitoring the document.styleSheets collection

I also tried MozAfterPaint - it might work but didn't for me, because my CSS didn't change anything on the page that required repaint. Obviously not a fit-all solution.

Another thing that failed was checking document.styleSheets[n].cssRules. Although document.styleSheets.length is updated immediately, I was thinking FF cannot update document.styleSheets[n].cssRules (document.styleSheets[n].sheet.cssRules) until the CSS actually arrives. However since FF 3.5 (or thereabouts) you don't have access to cssRules collection when the file is hosted on a different domain (CDN say). Security thing, you see. Even cssRules.length would've been enough, but nah.

Takeaways

1. Any ideas about a clever (or not so clever) workaround that lets us figure out when CSS is loaded in FF? Please comment.
2. Libraries can benefit from document.styleSheets.length trick to support Chrome and Safari. I know at least that YUI3 doesn't support callbacks on Y.Get.css() in Safari (nor FF)
3. Firefox 4 must implement load on stylesheets No doubt about it. IMO all browsers should fire load on everything related to external resources.

UPDATE: FF solved!

Thanks to Ryan's comment below, Zach and Oleg, turns out there is something that works. And that is:

1. you create a style element, not a link
2. add @import "URL"
3. poll for access to that style node's cssRules collection

It just happens so that Firefox will not populate this collection until the file arrives!

var style = document.createElement('style');
style.textContent = '@import "' + url + '"';
 
var fi = setInterval(function() {
  try {
    style.sheet.cssRules; // <--- MAGIC: only populated when file is loaded
    CSSDone('listening to @import-ed cssRules');
    clearInterval(fi);
  } catch (e){}
}, 10);  
 
head.appendChild(style);

Updated my test and seems to work just fine.

Whew!

I still maintain that this is unnecessarily complex and all browsers should simply fire load event. Seems to me this FF behavior might change at any time, as well as Safari's not populating document.styleSheet

One thing to note: this access to cssRules is not a failure of the same security check that doesn't allow cssRules access with outside domains. In this case we're accessing cssRules of the inline style and that's fine. We still cant access the styles of the @import-ed file from a different domain.

Side note: makes me wonder - this could be a technique to preload JS without executing too

Very rough notes from the event that just ended a few minutes ago

======== 1 ========

Tony Gentilcore (google) and Anderson Quach (MSFT) talked about the W3C working group and the specs for measuring performance:

• Navigation timing spec - HTML document: DNS, handshakes, load events
• Resource timing - how long JS, CSS, audio, video... take
• User timing - interactions, such as clicking a link

Challenges: user security, privacy. Small overhead for browsers. Useful actionable data out.

Navigation timing:

• window.performance - IE9 has no vendor prefix
• otherwise window.webkitPerformance
• FF4 probably shipping it too

Properties:

window.performance.timing
  .domainLookup
  .connect
  .domContentLoaded
  .loadEvent
  ....etc

Specs: http://test.w3.org/webperf/specs/NavigationTiming/

Resource Timing spec - upcoming

foreach resource 
  redirect
  fetchStart
  fetchEnd
  ...

User Timing - upcoming

  mark('pagelet')
  mark('something-else')
  ...
  
  >>> getMarks();
  {
    'pagelet': 123456767
  }

Stretch goals:

• painting times
• CPUs
• visibility (is the page in a background tab)
• allow real yealding setTimeout(0)
• don't animate faster than the browser paints

mailto: public-web-perf@w3c.org

======== 2 ========

Above the Fold Time: Measuring Web Page Performance Visually
Speaker: Jake Brutlag
Help from: Zoe Abrams, Pat Meenan (webpagetest) Google

What the user perception of page load timing is.

AFT (Above the Fold Timing) algo
AFT: When content above the fold stops changing and reaches its final state

Integrated in WebPageTest.org

Motivation

• AFT more representative than DOMContentLoaded or onload
• onload - different in IE and FF, e.g. async scripts block onload or not

Method:

• video capture
• AFT cutoff time - upper bound, e.g. 45 sec for DSL
• split pixels into static and dymanic (ads)
• AFT = latest change of a static pixel

Magic number - 5 determined empirically
static pixels are those that change less than 5 times

Application - validate other metrics, cross-browser, visual
Limitation - lab only

{I ran webpagetest.org on facebook.com with AFT on.
The results are very close to onload, as expected.
It's a simple page. Results:

http://www.webpagetest.org/result/110315_P9_5MDJ/

}

== break ==

Note from the sponsor StrangeLoop: a downloadable "State of the Union"
A study of top 2000 Alexa retail sites. Findings:

1. 11.2 sec onload avg
2. top 100 sites slower than the lower ranked
3. those using CDNs are not faster
4. failed to implement the basics - YSlow stuff

======== 3 ========

Next-gen YSlow
Betty Tso and Marcel Duran, Yahoo!

YSlow for Chrome last week
Not as accurate as Firefox cause headers, timing come from:
1. Crawl DOM
2. x-domain requests

Announcing YSlow bookmarklet version
for mobile and all other browsers to be released early april
YSlow now on Git?

Lots coming up, e.g:

• Multi-language support
• HAR-to-YSlow
• YSlow export to JDrop

Bookmarket works like:
1. crawl DOM
2. Use YQL to get headers

     SELECT * FROM data.headers WHERE
      url IN ('http://...', 'http://...')

======== 4 ========

Visibility into Mobile Performance
Steve Souders, Google

Demo #1: http://blaze.io/mobile/
WebPageTest.org framework connected to iPhone and Android apps
Mobile waterfalls
Demo #2: pcapperf
1. setup wifi hotspot on a pc/mac
2. tcpdump to capture .pcap file
3. upload .pcap to http://pcapperf.appspot.com app
4. observe waterfall via pcap2har and harviewer
5. run pagespeed on the har
Demo #3: JDrop
http://jdrop.org
Upload JSON so you can run something on mobile and view/analyze results in desktop's large screen
Works with the bookmarklet tools part of Steve's package of mobile meta-tool

The first part is here, the second is here. This third post is more about PHP and COM, rather than HTTPWatch or monitoring web performance, so feel free to skip if the title mislead you Keep reading if you want to use and improve/update my HTTPWatch class in the future.

The problem

After running a HTTPWatch-ed browser via a script I wanted to have an easy way to dump all the data collected. Since the PHP-HTTPWatch bridge is via COM interface, all the objects returned by HTTPWatch are Variants and not ready for introspection with the usual PHP functions, like get_object_vars() for example.

The solution

Interestingly, turns out there exist a function called com_print_typeinfo(). You give it a COM object (works with those variants HTTPWatch gives you) and it returns you source code for a PHP class defining this COM object. So the hack here is to evaluate this source code with eval() (oh, the horror!) and then inspect it with get_class_vars(). Luckily there's no naming collision between the PHP built-in classes and those defied by HTTPWatch.

// the input is a class name 
// and an object of that class
$class = "Entry";
$object = $http->watch->Log->Entries->Item(0);
 
// buffer output
ob_start();
// print out class definition
// derived from an object
com_print_typeinfo($obj);
$typeinfo = ob_get_contents();
ob_end_clean();
 
// evaluate the generated PHP source
eval($typeinfo);
 
// get the properties
$properties = get_class_vars($class);
 
// Horay!

In order for this to work, as you can see, we need to know the class names and we need access to an example object of each class. This is where I needed to study the HTTPWatch API and find a suitable example page that will generate enough objects to derive the API from.

The free HTTPWatch version

The free version has restrictions where you don't have access to all properties. I wanted my class to be able to do the best job possible with or without the presence of restrictions. That's why I first load google.com which is unrestricted and then an image on my blog, which is restricted.

From the first page I derive the complete API (that I'm interested in) and then I use the derived API to study the second URL request. Accessing each property in a try-catch blows up when a property is restricted, so I write it to a second array of API properties $paidproperties.

Source: the end

In the end when you run the script dumpapi.php. It uses my HTTPWatch class to derive the API for the HTPWatch class itself. How meta! The result of the run you write to an API file and then this file is included by the class. Nice and clean after a messy hack

Run:

$ php dumpapi.php > HTTPWatchAPI.php

(This is a one-off operation, no need to run it at all if you don't need to change anything in the class)

Then, before you instantiate the constructor, you go:

// point to the API dump
HTTPWatch::$apipath = "HTTPWatchAPI.php";
 
// the usual
$http = new HTTPWatch();

The default name and location for that API dump is the same directory and file name HTTPWatchAPI.php, so you skip that first line unless you have a valid reason to store the API in a different location.

In part 1 I demonstrated how you can use PHP to script and automate HTTPWatch. And how you can get data back, either reading the API docs or using a quick HAR hack to get a lot of data in one go.

Now I want to share a little class I wrote to make all that a little easier.

The code is here on GitHub.

Basic usage

Open IE, navigate, close:

$http = new HTTPWatch();
$http->go('http://phpied.com/');
$http->done();

To do the same in Firefox, just pass "ff" to the constructor:

$http = new HTTPWatch('ff');
$http->go('http://phpied.com/');
$http->done();

The constructor accepts a second param with options, like empty cache, hide browser (ie only), etc, largely underused for the time being.

Handle to the HTTPWatch plugin

After $http = new HTTPWatch(); a watch property will be added to $http. This is the HTTPWatch instance which gives you access to all its APIs, so you can do e.g.

$summary = $http->watch->Log->Entries->Summary;

Data out

My main motivation behind this class, other than simpler api, has been to provide the ability to just dump all the data that HTTPWatch collects in a quick print_r(). That has been a challenge with the COM PHP bridge, but I found a hack around it. In any event, most of the HTTPWatch API I've exported to a second PHP file - the HTTPWatchAPI.php script. (This is an auto-generated file, created by another script, but let's leave that out for now.)

So after you've navigated to a page you have two convenient methods to grab a bunch of data from HTTPWatch. The first is:

$http->getSummary();

This gives a summary stats for the http observation session. The second is

$http->getEntries();

It gives you details about every HTTPWatch log entry - be it cached or an actual HTTP request.

Here's an example of what getSummary() can give you. Here's how this file was generated:

$http = new HTTPWatch();
$http->go("http://google.com");
print_r($http->getSummary());
$http->done();

And here's some output print_r()-ed from getEntries(). Here's the code that produced it:

$http = new HTTPWatch();
$http->go("http://google.com");
print_r($http->getEntries());
$http->done();

If you look carefully at the dump, you may notice something like [Stream] => [BYTESTREAM]. Most of the times you don't need the raw HTTP streams (gzipped, chunked, etc), but you can get them if you want by setting:

$http->skipStreams = false;

Here's the same google.com example, this time including the raw streams. And the code:

$http = new HTTPWatch();
$http->go("http://google.com");
$http->skipStreams = false;
print_r($http->getEntries());
$http->done();

Free vs. paid

One pain with HTTPWatch is that the free version has restrictions. The summary for example doesn't include TimingSummaries and WarningSummaries properties. The entry log has almost nothing - no headers or content or streams. My class handles that by giving you as much as it can. If you're using the free version, it will return the limited data for the restricted URLs, but still the full data for those URLs that HTTPWatch's demo version allows - the top Alexa sites.

So here's a dump of visiting http://givepngachance.com with my free HTTPWatch edition.

The data has restricted information about givepngachance.com URL but full data related to the embeded youtube.com resource.

The code:

$http = new HTTPWatch();
$http->go("http://givepngachance.com");
print_r($http->getEntries());
$http->done();

Again with the video

If you've read part 1, you've probably seen the video, but here's the link again (try the HD version). This is a screencapture of loading FF and IE using my new class. The code that produced it is:

$ie = new HTTPWatch();
$ie->go('http://google.com/');
$sum = $ie->getSummary();
$ff = new HTTPWatch('ff');
$ff->go('http://google.com/');
$sumff = $ff->getSummary();
 
echo "\nRun 1 ";
echo $ie->watch->Log->BrowserName, ' ';
echo $ie->watch->Log->BrowserVersion;
echo "\nSent: ", $sum['BytesSent'], "; Received: ", $sum['BytesReceived'];
 
echo "\nRun 2 ";
echo $ff->watch->Log->BrowserName, ' ';
echo $ff->watch->Log->BrowserVersion;
echo "\nSent: ", $sumff['BytesSent'], "; Received: ", $sumff['BytesReceived'];
 
$ie->done();
$ff->done();

As you can see I'm accessing the HTTPWatch plugin object ($http->watch) directly to get the browser version and name. I didn't think this was worth wrapping in a more convenient API the way I did with getSummary() and getEntries().

The result of this is:

$ php examples.php

Run 1 Internet Explorer 6.0.2900.5512
Sent: 7102; Received: 89188
Run 2 Firefox 3.5.6
Sent: 6388; Received: 166473 

If you're wondering why FF gets twice the bytes, it's because google.com in IE6 is very basic - no search-as-you-type so much less JavaScript and one less sprite.

That's all folks

Enjoy, fork and keep an eye on what's up with the HTTP traffic. What goes through the tubes is too important not to be observed and monitored

HTTPWatch is a nice tool to inspect HTTP traffic in easy and convenient way and it works in both IE and FF now. Drawback - windows-only and paid. But the free version is good enough for many tasks.

HTTPWatch can be automated and scripted which is pretty cool for a number of monitoring-like tasks. Their site and help section lists C# and Ruby+Watir examples. So I was curious - what about PHP (and no Watir).

In general with PHP you can open/close/navigate IE using COM (whatever that is) which is nice, but you can't do that with Firefox as it doesn't expose a COM interface. But HTTPWatch fills the gap. K, let's see an example.

Prerequisites

OS: Windows
Install: IE, Firefox, HTTPWatch, php (command-line is fine, no need for Apache, MySQL, etc)

Getting started

Create a file, say C:\http.php, open command prompt and go:

cd \
C:\>php http.php

Now all that's left is to put something worth executing in http.php

Instantiating HTTPWatch

$controller = new COM("HttpWatch.Controller");
if(!method_exists($controller, 'IE')) {
  throw new Exception('failed to enable HTTPWatch');
}

Opening a new Firefox window:

$plugin = $controller->Firefox->New();

BTW, it's the same for IE:

$plugin = $controller->IE->New();

Disabling any filters (filters defined in HTTPWatch that is)

$plugin->Log->EnableFilter(false);

Clear HTTPWatch's log (the list of requests), clear the browser cache and start recording traffic:

// clear log and cache
$plugin->Clear();
$plugin->ClearCache();
 
// start
$plugin->Record();

Navigate to a URL and wait for it to complete - that means wait a bit after onload even

// browse
$plugin->GotoUrl('http://google.com');
$controller->Wait($plugin, -1);

Stop monitoring traffic and quit the browser:

$plugin->Stop();
$plugin->CloseBrowser();

This is nice, we opened the browser, visited a URL and closed. Now we can even get some meaningful data out of the whole experience.

$plugin->Log->Entries is an object that has a list of all requests. It also has a property Summary. So we can see how many bytes we sent and how many received as a result of this visit to google.com

$sum = $plugin->Log->Entries->Summary;
echo "in: {$sum->BytesReceived}, out: {$sum->BytesSent}";

Note: oh, you need to get your data before closing the browser, otherwise the Log object gets destroyed it seems

So the result:

C:\>php http.php
in: 89185, out: 7102

Yeah!

This may look like nothing, but is pretty impressive in an of itself. At least I know I was happy the first time it worked. Because, you see, any monitoring that doesn't use a real browser is kinda smelly, isn't that right? Plus this is awesome for performance tests, research and experiments. You can create page A and page B and go out for a walk. Meanwhile your script can load the pages 200 times in the two browsers (at least, because you can have FF+IE[678]), with empty and full cache... and you come back for the results! Tired of all the walking, not of hitting REFRESH.

Below you can see (HD!) video of a script that opens IE and FF, loads Google and then gives you the bytes in/out in the two browsers. This example uses a PHP class I created and will talk about later, but you can still see the idea.

A better experience in IE

One thing I don't like is that HTTPWatch won't let you control the browser very well. Two features I'm looking for: being able to see HTTPWatch's log while running (for testing) and then being able to completely hide the window (for "production"). Luckily IE let's you do that and HTTPWatch let's you "attach" an already running IE instance.

So. We open IE with its own COM interface:

$browser = new COM("InternetExplorer.Application");
if(!method_exists($browser, 'Navigate')) {
  throw new Exception('didn\'t create IE obj');
}
$browser->Visible = true;

As you can see - not very different. But there's Visibile which can be false if you so like. This way you can still work on something while tests are running in the background without windows popping up all the time.

Also if you open HTTPWatch manually and close the browser, then the next time (in your scripted runs) HTTPWatch will stay open and you can check what's up.

So, connecting HTTPWatch with the IE instance means instantiating HTTPWatch as before and passing the IE object to Attach() method (was New() before).

// watch this!
$controller = new COM("HttpWatch.Controller");
if(!method_exists($controller, 'IE')) {
  throw new Exception('failed to enable HTTPWatch');
}
 
// enable plugin
$plugin = $controller->IE->Attach($browser);

The rest is all the same.

There's more

The most interesting part is getting data back from HTTPWatch. Dunno about you, but I love just dumping whatever structure I have with print_r() or var_dump() and then deciding what I want from it and how to to go about getting it.

That doesn't happen here because these COM objects are Variants and you can't just dump'em. You have to read the API docs. That sucks. So I did a hack (next post) and also read the APIs ("Stoyan: reading the APIs so you don't have to!") to enable just dumping the httpwatch's log.

Meawhile...

HAR

HTTPWatch can write you a HAR file with the log. Not everything is in there, but it's still a lot and it's easy. HAR is JSON so you json_decode() it and voila - a log!

$filename = tempnam('/tmp', 'watchmenowimgoindown');
$plugin->Log->ExportHAR($filename);
$json = file_get_contents($filename);
 
print_r(json_decode($json));

If you're curious as to what that prints - here it is.

Want to see a HAR (from another run)? Here it is.

So here you go - much data can be extracted and dumped for inspection from the HAR output. For the full httpwatch data, there's the API.

(to be continued...)

Last year I compared some CSS minifiers, namely YUICompressor, CSSTidy (with "small" vs. "safe" settings), PHP PEAR's CSS lib and Minify (detailed results). Now that I've done some work on the YUICompressor and since there's a new kid on the block from Microsoft I thought I should give it another go.

I only compared CSSTidy with best compression, YUICompressor and MS' Ajaxmin. The verdict is still the same - CSSTidy looks the best, but there really isn't that big of a difference between the three (and virtually no difference between YUIC and Ajaxmin), especially if you take into account gzipping.

Still on average you get about 35% size reduction when you minify and 80% when you gzip the minified CSS. Meaning when all is done your users only download 19-20% of what they would normally do if you don't perform these simple optimizations.

Experiment

Source CSS files came from csszengarden.com - 213 CSS files.

For the YUICompressor I used the JS verison, on windows with WSH, described in the previous post.

For CSSTidy I used exactly the same code as the last year.

Lastly, Ajaxmin I ran with the default options:

$ AjaxMin.exe in.css -out out.css

Results

CSSTidy wins with bringing the size down to 63.29% of the original on avegare. The other two follow closely with 66.47% (ajaxmin) and 66.41% (YUI). After gzipping: 19.44% (tidy), 19.62% (ajaxmin) and 19.63% (yui). In other words YUI's cssmin performs ever so slightly better that Ajaxmin by 0.06% before gzipping, but after gzipping Ajaxmin wins with 0.01%.

Overall the results show that you shouldn't sweat too much over the choice of CSS minifier. They are all pretty close and you should pick the one that is most convenient for your own use and build process.

CSSTidy and Ajaxmin have parsers, so theoretically they have a much better chance. YUI's cssmin is a bunch of regular expressions and no proper parsing. In reality they are all performing really similarly. Part of the reason is that CSS is not as minifiable as JavaScript as most of it is long selectors and long prperty names (background-image, text-decoration, -webkit-transform-origin, anyone?), which are not renamed to shorter names the way JS variables are.

One random observation (and hint for Ajaxmin folks) is that cssmin strips selectors with empty rulesets while Ajaxmin does not. E.g. #selector {} is totally gone in cssmin but stays in ajaxmin.

Detailed results

JavaScript can run virtually anywhere, including as a windows exe and the windows command line.

Say you have a JavaScript function foo()

function foo(input) {
  var output = input;
  // .. unicorns
  return output;
}

In order to make this a windows shell script you add at the and a way to read standard input and then write to the standard output:

(function () {
 
    var input  = WScript.StdIn.ReadAll(),
        output = foo(input);
 
    WScript.StdOut.Write(output);
 
}());

Then you run this script, say foo.js, like:

$ cscript foo.js < input.txt

And it prints the output to the console.

If you want to read and print the code of foo.js itself you go:

$ cscript foo.js < foo.js
Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

function foo(input) {
  var output = input;
...

You can remove this "Microsoft (R) Windows..." stuff with //NoLogo parameter:

$ cscript //NoLogo foo.js < foo.js
function foo(input) {
  var output = input;
...

CSSMin.JS

Alrighty, going back to the title of the post.

CSSMin.js is a port of YUICompressor's CSS minifier (source, hosted tool). Now adding a few lines at the end makes a windows shell script:

(function () {
 
    var src = WScript.StdIn.ReadAll();
 
    if (!src) {
        // help!
        WScript.StdOut.WriteLine("cscript //NoLogo cssmin.js < in.css > out.css");
        return;
    }
 
    WScript.StdOut.Write(YAHOO.compressor.cssmin(src));
 
}());

Download it here.

Use it like:

$ cscript //NoLogo cssmin.js < in.css > out.css

Don't forget the //NoLogo or you'll end up with "Microsoft..." in your minified files

Random observation: "dude"[0] === "d" in most JS environments but is undefined in WSH (Windows Scripting Host). So "dude".substring(0, 1)

I talked before about using an object element to load scripts and styles without executing/applying them to the current document. And this is nice for preloading assets - faster and less error-prone than simple inclusion if the document or iframe.

But what about preloading assets (scripts) and then executing them on the current page? Turns out it works just fine, like so:

• create an object element and point its data to the JavaScript file to be loaded
• listen to the load event of the object
• once loaded, create a script element and point its src to the file, which would be already cached
• optionaly, execute a callback function once the script element is properly inserted into the DOM

Something like this:

function yield(file, callback) {
    var o = document.createElement('object');
    o.data = file;
        
    o.width  = 0;
    o.height = 0;
    
    o.onload = function() {
        var s = document.createElement('script');
        s.src = file;
        s.onload = function() {
            callback(o, file);
        };
        document.getElementsByTagName('head')[0].appendChild(s);
        
    };
    
    document.body.appendChild(o);
}

But, but.. why?

Well, first off it's yet another way to load JavaScript files asynchronously, in a non-blocking fashion. And that's a good thing. Async = win. This method sure beats others such as document.write (eek) or iframe. It has advantage over XHR because there's no same-domain restriction. DOM include is probably the only simpler and more robust method, but there are cases where the object element might be preferable.

DOM-included scripts in FF and Opera always execute in order. That's cool, but sometimes you may prefer out of order async execution. Whenever a script is ready - go! Or sometimes you may prefer if the script is not executed at all! That may sound odd (why would I load it if I don't want to execute it?) but I have a use case - auto-complete. When your data source is on another domain and you can't use XHR, then JSON-P would be the way to go. But the network is a jungle and anything can happen.

Say you make requests for auto-complete suggestions with queries "h", "he", "hel", "hell", "hello". For some reason "hel" is really slow and you already have the results for "hell" and even "hello". What you want to do is just kill and cancel the "hel" response, who cares - it's outdated already. But if you use JSONP in FF and Opera "hel" will block the others. If order is not preserved it might be even weirder - you update the page with the "hello" suggestion and then "hel" finally arrives and updates the suggestion once again. Fail.

Using the object element approach you can keep a list of requests that went out and simply choose to ignore previous requests if newer responses are already there.

I have a test case even.

Small print

This technique assumes that the scripts you load don't have any headers preventing caching, otherwise they'll be requested twice.

My example code above ignored IE completely, but you can easily make it work, either using new Image() instead of object or tweaking the object with height and width of 1 as shown in the old article. Or you can use the trick that LABjs uses (I think) which is to DOM-insert a script element with invalid type such as cache/javascript. Also you need to listen to onreadystatechange instead of load event when inserting a DOM element.

MHTML and Data URIs in the same CSS file is totally doable and gives us nice support for IE6+ and all modern browsers. But the question is - what about inline styles. In other words can you have a single-request web application which bundles together markup, inline styles, inline scripts, inline images? With data URIs - yes, clearly. But MHTML?

I remember hacker extraordinaire Hedger Wang coming up with a test page, which proved it's doable. Problems with the test are that a/ I can't find the page anymore, his domain has expired b/ there was some funky IE7/Vista stuff (probably now solvable) in there which even included an undesired redirect c/ was complex - the whole HTML becomes a multipart document, if I remember correctly there was something that required html served as text/plain....

So I tried something simple - shove an MHTML doc inside an inline style comment. It so totally worked! Including IE6 and IE8 in IE7 mode on Windows 7 (which in my experience behaves as badly as IE7 proper on Vista)

Here's the test page. Look ma', no extra HTTP requests

So it's a simple HTML doc:

<!doctype html>
<html>
  <head>
    <title>Look Ma' No HTTP requests</title>
    <style type="text/css">
 
/* magic here */
 
    </style>
  </head>
  <body>
    <h1>MHTML + Data:URIs inline in a <code>style</code> element</h1>
    <p class="image1">hello<br>hello</p>
    <p class="image2">bonjour<br>bonjour</p>
  </body>
</html>

And the magic is two parts: the MHTML doc inside a CSS comment and the actual CSS which uses data URIs for normal browsers and refers to the MHTML parts in IE6,7.

/*
Content-Type: multipart/related; boundary="_"
 
--_
Content-Location:locoloco
Content-Transfer-Encoding:base64
 
iVBORw0KGgoAAAAN ... [more crazyness]... QmCC
--_
Content-Location:polloloco
Content-Transfer-Encoding:base64
 
iVBORw0KGgoAAAANSUh ... [moarrr] ... ggg==
--_--
*/
.image1 {
  background-image: url("data:image/png;base64,iVBORw0KGgoAAAAN ... QmCC"); 
  *background-image: url(mhtml:http://phpied.com/files/mhtml/mhtml-html.html!locoloco); 
}
 
.image2 {
  background-image: url("data:image/png;base64,iVBORw0KGgoAAAANSUh ... ggg=="); 
  *background-image: url(mhtml:http://phpied.com/files/mhtml/mhtml-html.html!polloloco); 
}
 
body {
  font: bold 24px Arial;
}

How cool is that!

Please report any issues you might find in any browser/os combination

The obvious drawback is repeating the long base64'd image content twice, but it's solvable with either server-side sniffing or... one crazy hack, found on the Russian site habrahabr.ru. I should talk about it separately and help spread the word to the larger English-speaking audience, but for the impatient - click!

So there you go - MHTML inline in CSS inline in HTML or building single-request x-browser web apps

Reducing the number of HTTP requests is a must, sprites are cool, but a pain to maintain, so there come data URIs (for all browsers) and MHTML (IE6 and 7). I've talked about these things on this blog to a point where the blog comes up in top 10 results in search engines for queries like "mhtml" and "data url". Therefore I think it's my duty to clarify a point for the good of the mankind

MHTML works in IE6 and IE7 even in the deadly IE7/Vista and IE7/Win7 combos

In the community we've long considered MHTML in IE7/Vista a problem and I've personally come up with complex voodoo workarounds how to mitigate the issue and still make use of the technique. Turns out the whole problem all the time was a small syntax glitch.

Pointed by a comment at a previous post all we ever needed was to close the boundary delimiter and add two dashes at the end. The double-dash of doom as I like to call them since I've spent so much time wrestling.

So let's take a look at the syntax.

Update: Also check this comment for additional insight from Vincent, Aaron and _cphr_ regarding double line break of doom

One part

MHTML is a multi-part document. One document containing several parts. One part looks like this:

Content-Location: myimage
Content-Transfer-Encoding: base64

iVBORw0KGgoAAAANSU....U5ErkJggg==

In other words it has headers, base64-encoded content and two empty lines to divide them.

Multi parts

The different parts in the document are divided by a separator string. And at the top of the document you define what this separator is. Anything you like. So

Content-Type: multipart/related; boundary="MYSEPARATOR"

--MYSEPARATOR

[here comes part one]

--MYSEPARATOR

[here's part two]

--MYSEPARATOR--

Did you notice -- at the very end? Yes, this is the double-dash of doom. Forget it and you get IE7/Vista problems (only on cached documents) and permanent hair loss. The thing is that in IE6 and other IE7s you can omit the whole last separator and it's all good. So historically you never needed it, but come Vista and Win7 and problems start.

All together now

Finally, let's see the whole thing, a whole CSS file, including the way you refer to the parts later on in the CSS.

/*
Content-Type: multipart/related; boundary="MYSEPARATOR"
 
--MYSEPARATOR
Content-Location: myimage
Content-Transfer-Encoding: base64
 
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAMAAADXqc3KAAAD....U5ErkJggg==

--MYSEPARATOR
Content-Location: another
Content-Transfer-Encoding: base64
 
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAMAAADXqc3KAAAA....U5ErkJggg==

--MYSEPARATOR--
*/
.myclass {
    background-image:url(mhtml:http://example.org/styles.css!myimage);
}
.myotherclass {
    background-image:url(mhtml:http://example.org/styles.css!another);
}

Updated PHP class

Previously when I fought IE7/Vista I came up with a PHP class that would take some images and create "data sprites" on the fly, creating two different versions - one with data URIs and one with MHTML, depending on the browser. The old code is here.

Now, I've updated it (basically just deleted serious portions of it that dealt with Vista) and put it up on github. Right here.

Updated test pages

• Updated test page from the original MHTML article with the double dash of doom correction. Just the CSS.
• Test page for the DataSprites php class. Just the CSS (changes based on the User-Agent)

Thanks for reading, that's about it. Now off I go to correct the older posts, catch ya later

So it's been pretty quiet around here. I'm still alive and very much so. Just awfully busy.

Bulgaria

I took a long trip in Bulgaria. About 1 month and 20 days. Including the traditional around-Bulgaria let's-get-drunk-in-different-cities-every-day tour de force with the gang. With unusual amount of guitar playing this time.

Not all fun and roses, though, I had to work most of the time for a superimportant gamechanging ultrasecret project for Yahoo! Search. And work on the finishing touches for a book.

France, Italy, UK

While spending time in Bulgaria I did some quick jumps to other European countries - France, Italy, UK. Everything is so close in Europe, it's a shame not to wander around once you cross the big pond.

In Paris I had the pleasure to practice some French, but most importantly to meet with the local speed freaks at the Web Perf Meetup organized by Éric Daspet. Great experience to interrupt sightseeing and soufflé-eating just to be among fellow geeks in a small and cozy group.

photo: @tbassetto

Second tour leg

Back in California to the normal life and kids' school year start end of August. But a second European tour leg was just around the corner.

JSConfEU in Berlin was last week, this is hands-down the best conference. I cannot speak high enough about it, the organizers, the people, speakers, everyone, the parties (ouch, the parties). I met so many people I have been in contact with (or wanted to be) for quite a while, including the new book's tech reviewer Andrea aka @WebReflection. It's surprising, for the regular introvert geek I am, the pleasure of meeting and talking to people. Oh, the miracles of the little, or not so little, quantities of Berliner Pilsner.

Back from Berlin, a copy of the new book was waiting, how adorable! (I really need to put some marketing effort into it... at least put up a page what's it about, since people are asking). It's been rating very well in Amazon's "JavaScript" category. Saw it at #3 a few times, watch out "good parts"!

Next week it's Fronteers conference in Amsterdam. Really looking forward to this one. After all, the one and only PPK started it all. I'll be speaking about progressive downloads and progressive rendering. Progress is important, progress is critical! And I've never been to Amsterdam.

Then Moscow and highload++ conference. How cool that would be. I always wanted to see Moscow or any part of Russia really.

So... busy, busy, busy. And another book baby on the road. And one last conference but here in LA's "backyard" Las Vegas this time.

I came across this blog post (via @pornelski and @souders) where Markus has stumbled upon a case where an IE6-only stylesheet included with a conditional comment blocks the downloads in IE8. Whaaat?

I had to dig in. To give you a summary: turned out that any conditional comment, not only for an extra CSS, will block further downloads until the main CSS file arrives. Also the solution offered on the blog post (using X-UA-Compatible) seems to be more of an error due to an accidentally left comment.

Check out the tests.

Base page

The first test is the base page. It follows a pretty common style pattern - CSS at the top, a bunch of images in the middle, JS at the bottom.

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">
    
<html>
<head>
    <title>The base page</title>
    <link type="text/css" rel="stylesheet" 
          href="http://tools.w3clubs.com/pagr/1.expires.css">
</head>
<body>
<p>
    <img src="http://tools.w3clubs.com/pagr/1.expires.png" alt="1">
    <img src="http://tools.w3clubs.com/pagr/2.expires.png" alt="2">
    <img src="http://tools.w3clubs.com/pagr/3.expires.png" alt="3">
    <img src="http://tools.w3clubs.com/pagr/4.expires.png" alt="4">
</p>
<script type="text/javascript" 
        src="http://tools.w3clubs.com/pagr/1.expires.js"></script>
</body>
</html>

The waterfall produced by WebPageTest looks like so:

The page is here, the results of the WebPageTest's test are here.

Conditional IE6 stylesheet

Adding a second stylsheet to the head like so:

<head>
  <title>base page</title>
  <link type="text/css" rel="stylesheet" 
        href="http://tools.w3clubs.com/pagr/1.expires.css">
  <!--[if IE 6 ]>    
    <link type="text/css" rel="stylesheet" 
          href="http://tools.w3clubs.com/pagr/2.expires.css">
  <![endif]-->
</head>

Turns out that this conditional comment blocks further downloads until the main CSS arrives.

Test page, test results, waterfall:

Just like that the total page to onload time went up from 1 second to almost 1.3 seconds. Ouch.

And this is because of an IE6 stylesheet, which IE8 has no use for. My wild guess is that IE needs to parse through those conditional comments and treats them sort of like inline script. And we know that inline scripts following a stylesheet tend to block.

Conditional markup

What if we don't include IE6 specific stylsheet, but use the conditional comments to write different body tags with different class names, as described by Paul Irish.

<!--[if IE 6]> <body class="ie6"> <![endif]--> 
<!--[if !IE]><!--> <body> <!--<![endif]-->

Turns out that these markup conditional comments will also block downloads until the CSS arrives.

Test page, test results, waterfall looks as bad (the same blocking).

Browser-sniffing comments

I blogged about this a few days ago - using conditional comments to do the browser sniffing and include appropriate CSS - one for normal browsers and a complete alternative for IE6,7.

<head>
  <!--[if lte IE 7]>
    <link type="text/css" rel="stylesheet" 
          href="http://tools.w3clubs.com/pagr/2.expires.css">
  <![endif]-->
  <!--[if gt IE 7]><!-->
    <link type="text/css" rel="stylesheet" 
          href="http://tools.w3clubs.com/pagr/1.expires.css">
  <!--<![endif]-->
</head>

Turns out this is OK to do. The conditional comments are processed before the download is initiated, so there't nothing to block on after the stylesheet. Yeey!

Test page, test results, waterfall looks like the first one for the base page.

X-UA-Compatible not a solution

The blog post suggested using the X-UA-Compatible meta tag to say that the UA is the latest IE possible.

<meta http-equiv="X-UA-Compatible" content="IE=edge">

It didn't work for me.

Test page, test results, waterfall like the second (the blocking) one.

Looking closely and thanks to the screenshots digging out the original test page I noticed that it contains a dangling comment. Putting that dangling comment in a test page, and the blocking effect was gone. But this is a bug. In fact the comment shows up on the page! My wild guess is that this improper comments invalidates the following one and that's why there's no blocking. I guess that IE6 will not load the stylesheet, but I didn't test.

So my tests - with X-UA meta tag (results) and with comment bug (results)

Conclusions, conclusions

To summarize, if you worry about performance, don't use conditional comments.

There might be an exception when you put a script in the head after the CSS - then there are two blocking things, so the effect of the conditional comment is not visible. But that's still bad for performance, there's still blocking.

It's OK to use the browsers-sniffing comments approach, provided of course, that there's no other CSS file before the sniff. If there is one, it will block.

Best - just use _ and * hacks, go with a single CSS and forget sniffing.

Update: empty conditional comment

Thanks to Markus who kept looking into the extra conditional comment comes a solution: an empty conditional comment early on solves the blocking issue.. By "early on" I mean before the main CSS which caused the blocking.

I did two more tests to validate the solution and it absolutely works.

One test has empty comment + conditional comment for writing Paul's body tags. The empty comment (aka the solution) is right before the blocking CSS file.

<head>
    <title>base page</title>
    <!--[if IE 6]><![endif]-->
    <link type="text/css" rel="stylesheet" 
          href="http://tools.w3clubs.com/pagr/1.expires.css">
</head>
 
<!--[if IE 6]> <body class="ie6"> <![endif]-->
<!--[if !IE]><!--> <body> <!--<![endif]-->

Test page, webpagetest results. Works as advertised, no more blocking.

The second test uses empty comment and conditional stylesheet. In this case I even put the empty comment way at the top. Sort of like declaring upfront - hey this page uses conditional comments and the empty comment is the solution to the blocking effect.

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">
<!--[if IE 6]><![endif]-->
 
<html>
<head>
  <title>base page</title>
  <link type="text/css" rel="stylesheet" 
        href="http://tools.w3clubs.com/pagr/1.expires.css">
  <!--[if IE 6 ]>    
    <link type="text/css" rel="stylesheet" 
          href="http://tools.w3clubs.com/pagr/2.expires.css">
  <![endif]-->
</head>

Test page, webpagetest results. No more blocking

Answering Andrea's question - what it if you have several conditionals - for IE6, IE7 and so on, I did one more test with two conditional CSS files. Turns out it's fine, as long as there's one conditional comment before the CSS. I actually updated the comment so it checks for all IE versions.

Test page, results, code:

<!--[if IE]><![endif]-->
<html lang="en">
<head>
    <title>base page</title>
    <link type="text/css" rel="stylesheet" 
          href="http://tools.w3clubs.com/pagr/1.expires.css">
    <!--[if IE 6]>    
        <link type="text/css" rel="stylesheet" 
              href="http://tools.w3clubs.com/pagr/2.expires.css">
    <![endif]-->
    <!--[if IE 7]>    
        <link type="text/css" rel="stylesheet" 
              href="http://tools.w3clubs.com/pagr/3.expires.css">
    <![endif]-->
</head>

In conclusion #2... conditional comments cause CSS to block. The workaround it to have an extra empty comment before the blocking CSS, or, to be safe, right after the doctype. Even better - don't use conditional comments at all. Except for browser-sniffing and loading two complete and completely separate CSS files, not just the IE fixes.

The previous parts are here (building and testing) and here (what gets minified). Now let's see how YUI CSS min handles CSS hacks.

As you know CSS hacks often use errors in CSS parsers in browsers to target specific browser versions and supply additional rules to work around other issues in said browsers. That makes any CSS tool's job slightly more challenging. Not only does the tool have to avoid repeating the browsers errors, but also has to understand what browsers got wrong and support it too. Fun stuff. Isn't it a joy being a web developer?

So here are some hacks that are tested to work with the YUICopmpressor's CSS min.

Underscore/star hack

The simplest ever hack to target IE6 and IE7. In the example below normal browsers see 1px dropping _width and *width as invalid, IE7 ignores the *, drops the _width as invalid and sees 3pt, IE6 ignores the _ and sees _width as width, so it sees 2em.

CSS min doesn't parse and doesn't understand CSS properties, so it accepts pretty much any property.

Before:

#element {
    width: 1px;
    *width: 3pt;
    _width: 2em;
}

After:

#element{width:1px;*width:3pt;_width:2em}

Child selector hack

CSS min strips comments but there is this child selector hack people use to hide declarations from IE7 and below.

CSS min retains empty comments that immediately follow > (thanks go out to Chris Burroughs)

Before:

html >/**/ body p {
    color: blue; 
}

After:

html>/**/body p{color:blue}

IE5/Mac hack

This hack targets IE5/Mac, if anyone still worries about this browser. The hack is retained after minification, only it's minified.

Before:

/* Ignore the next rule in IE mac \*/
.selector {
   color: khaki;
}
/* Stop ignoring in IE mac */

After:

/*\*/.selector{color:khaki}/**/

Box model hack

This hack uses valid CSS and there's no special use of comments so it's retained.

Before:

#elem { 
    width: 100px; /* IE */
    voice-family: "\"}\""; 
    voice-family:inherit;
    width: 200px; /* others */
}
html>body #elem {
    width: 200px; /* others */
}

After:

#elem{width:100px;voice-family:"\"}\"";voice-family:inherit;width:200px}html>body #elem{width:200px}

Seems like the code highlighter chokes here though. It ain't easy

That's all, folks!

Thanks and please, feel free to suggest improvements and report bugs. Also play with the web UI of the JS-version here to see for yourself what it does to your code.

The first part is here. It was more about building the YUICompressor, writing and running test cases. Now let's see what the compressor does exactly to your CSS.

BTW, you can play with the web UI to see for yourself how the minifier works.

Stripping comments and white space

This is the bare minimum a minifier can do. And when it comes to CSS, this is also the place where ther biggest improvement comes from. In JS for example you can rename variables and save bytes, but in CSS the possibilities are more limited. No shorter way to say text-decoration, unfortunately.

So before:

/***** 
  classmates stuff
*****/
.classmates {
    /* after 10 years */
    weight: considerable;
}

After:

.classmates{weight:considerable}