Archive for the 'JavaScript' Category

Preload CSS/JavaScript without execution

Wednesday, April 21st, 2010

Preloading components in advance is good for performance. There are several ways to do it. But even the cleanest solution (open up an iframe and go crazy there) comes at a price - the price of the iframe and the price of parsing and executing the preloaded CSS and JavaScript. There's also a relatively high risk of potential JavaScript errors if the script you preload assumes it's loaded in a page different than the one that preloads.

After a bit of trial and lot of error I think I came up with something that could work cross-browser:

  • in IE use new Image().src to preload all component types
  • in all other browsers use a dynamic <object> tag

Code and demo

Here's the final solution, below are some details.

In this example I assume the page prefetches after onload some components that will be needed by the next page. The components are a CSS, a JS and a PNG (sprite).

window.onload = function () {

    var i = 0,
        max = 0,
        o = null,

        // list of stuff to preload
        preload = [
            'http://tools.w3clubs.com/pagr2/<?php echo $id; ?>.sleep.expires.png',
            'http://tools.w3clubs.com/pagr2/<?php echo $id; ?>.sleep.expires.js',
            'http://tools.w3clubs.com/pagr2/<?php echo $id; ?>.sleep.expires.css'
        ],
        isIE = navigator.appName.indexOf('Microsoft') === 0;

    for (i = 0, max = preload.length; i < max; i += 1) {

        if (isIE) {
            new Image().src = preload[i];
            continue;
        }
        o = document.createElement('object');
        o.data = preload[i];

        // IE stuff, otherwise 0x0 is OK
        //o.width = 1;
        //o.height = 1;
        //o.style.visibility = "hidden";
        //o.type = "text/plain"; // IE 
        o.width  = 0;
        o.height = 0;

        // only FF appends to the head
        // all others require body
        document.body.appendChild(o);
    }

};

A demo is here:
http://phpied.com/files/object-prefetch/page1.php?id=1
In the demo the components are delayed with 1 second each and sent with Expries header. Feel free to increment the ID for a new test with uncached components.

Tested in FF3.6, O10, Safari 4, Chrome 5, IE 6,7,8.

Comments

  • new Image().src doesn't do the job in FF because it has a separate cache for images. Didn't seem to work in Safari either where CSS and JS were requested on the second page where they sould've been cached
  • the dynamic object element has to be outside the head in most browsers in order to fire off the downloads
  • dynamic object works also in IE7,8 with a few tweaks (commented out in the code above) but not in IE6. In a separate tests I've also found the object element to be expensive in IE in general.

That's about it. Below are some unsuccessful attempts I tried which failed for various reasons in different browsers.

Other unsuccessful attempts

1.
I was actually inspired by this post by Ben Cherry where he loads CSS and JS in a print stylesheet. Clever hack, unfortunately didn't work in Chrome which caches the JS but doesn't execute it on the next page.

2.
One of the comments on Ben's post suggested (Philip and Dejan said the same) using invalid type attribute to prevent execution, e.g. text/cache.

var s = document.createElement('script');
s.src = preload[1];
s.type = "text/cache";
document.getElementsByTagName('head')[0].appendChild(s);

That worked for the most parts but not in FF3.6 where the JavaScript was never requested.

3.
A dynamic link prefetch didn't do anything, not even in FF which is probably the only browser that supports this.

for (i = 0, max = preload.length; i < max; i += 1) {
    var link = document.createElement('link');
    link.href = preload[i];
    link.rel = "prefetch";
    document.getElementsByTagName('head')[0].appendChild(link);
}

Then it took a bit of trial/error to make IE7,8 work with an object tag, before I stumbled into IE6 and gave up in favor of image src.

In conclusion

I believe this is a solution I could be comfortable with, although it involves user agent sniffing. It certainly looks less hacky than loading JS as CSS anyways. And object elements are meant to load any type of component so no semantic conflict here I don't believe. Feel free to test and report any edge cases or browser/OS combos. (JS errors in IE on the second page are ok, because I'm using console.log in the preloaded javascript)

Thanks for reading!

 

Publishing 5 books this year

Thursday, April 1st, 2010

So I'll be publishing 5 books this year. Isn't that incredible? Is it even possible? And good quality books at that? It's a nice challenge (my last year's challenge failed, I didn't even bother to count how bad it failed). I think it's possible, especially if you bend a little bit the meaning of "5", "year", "publishing" and "me" :)

Book #1 - High-Performance JavaScript

hpjs

Let's start bending - this is a book where I wrote just one chapter. It's a book by Nicholas Zakas with contributions from:

And I wrote my chapter mainly the last year. My chapter is about the DOM. But the book became available just now, few days ago, so it's published this year (bending, bending...)

Book #2 - JavaScript Patterns

I am hard at work on this one currently (explains the low activity on this blog). I started last year but only finished two chapters in '09. The bending part here is that I've already given presentations on the topic and have been writing a "patterns" column for JSMag for a while, so I can recycle quite a bit of content.

You can see the tentative cover, I hope it stays tentative and we can replace the hen with a nice cute little zebra (a.k.a. donkey with patterns). Between you and me, I think there's a new designer in O'Reilly with a bird fetish.

I expect the first draft for this one to complete within weeks. And no, it's not about implementing the Gang of Four patterns in JavaScript (has been done already by Ross, see above), although there's one chapter on a selected few - Singleton, Factory, Observer, Proxy, Decorator...

Book #3 - Speed Matters

I've contracted with Peachpit Press to write a book about performance targeted mainly at designers. It will be about the business (why go fast), technology (how) and psychology (perception of speed) of web performance. I'm excited about this one for a number of reasons:

  • there's a lot of misconceptions being spread around in designer blogs and books, especially sad when one of the books in question is a sort of a bible for web designers. I mean things like PNG vs. GIF, gzipping and others. I hope I can present a readable, concise and, above all, technically correct text for designers who may find Steve Souders' HPWS, a.k.a. "The Bible" a little too dry because it's from O'Reilly and has no colors
  • the publisher is considering a sort of novel approach to writing the book, fingers crossed, because I believe it's the right way to write technical books.
  • at the very least, the book will be available as early drafts while it's being written, which is new to me, but always wanted to do.
  • the book will be full color - again, new experience to me

The bending here comes from the fact that I'll try to reuse from the perf advent calendar if I can. So some content may be pre-written.

Book #4 - Object-Oriented JavaScript (2nd edition)

The bending here is obvious - it's just a second edition, not a completely new book from scratch. My goal here is:

  • address errata
  • address some excellent critiques (of this otherwise bestselling book!), such as this one by @kangax, which is the article that actually prompted me to pitch a second edition to the publisher. So many thanks to Yuri! Also thanks to Asen who's been sending me invaluable and detailed feedback on the first edition. And now thanks to Asen and Kangax (and also Dmitry) I'm spending some time lurking on comp.lang.javascript mailing list, which is full of great discussions.
  • ECMAScript5 update
  • some concepts such as hoisting, NFE, property attributes, etc
  • one completely new chapter on testing and docs
  • answers to the end-of-chapter exercises - an often-requested update

Hoping this title will not take a lot of time.

And since these 4 books should be finished by the end of August or thereabouts, this will give me whole 4 months (1/3 of an year) to dive into something I've been thinking about, two things actually - CSS and self-publishing.

Book #5 - CSS for web devs

CSS is widely misunderstood by many people, me including. I'm convinced we only use a portion of all that CSS is, and use it badly. I'm not saying it will be CSS: The Good Parts, but I plan to address what I consider bad habits in CSS (mis)use and write a book as a learning experience. This is the best way to learn IMO. It will be self-published and probably available online for free too. And by self-publish I don't mean lulu.com or some of the other resellers, but working with the printer and distributor directly.

Too ambitious? April Fool's?

Probably, but with all the pre-written stuff and other cheating, it may very well be doable. Then I guess I'll take a 5 year break :)

 

YUICompressor’s CSSMin

Wednesday, March 10th, 2010

Honored to be a part of the YUI project, I am now helping with the maintenance of the CSSMin part of the YUICompressor. My changes are now part of the trunk on github, so I'm official. Next on the agenda is documenting the thing, so that's what I'll try to do here, maybe in a few posts. You know, divide and conquer.

PHP, Java and a JavaScript port

Originally written in PHP by Isaac Schlueter and ported to Java by Julien Lecomte, CSSMin got a JavaScript port by yours truly some time ago. Because, after all, JavaScript is the language of the web, isn't it?

You can play with the latest git version of the JS port online here.

I'm also happy to report that the JS port is now used in PageSpeed and YSlow (as you probably know Firefox extensions are written in JavaScript)

Page Speed

YSlow

Building

If you want to play on your own with the source version of YUICompressor without waiting for the next release, you can build it like so:

  1. Checkout or download the code from http://github.com/yui/yuicompressor/
  2. Navigate to the root yuicompressor/ directory
  3. Type ant and hit enter

In order for this to work you need a somewhat recent Java SDK installed and also Ant running. (On the Mac, just do port install apache-ant to get Ant)

This is for the Java version, the JS version needs no building, of course.

Tests

There's a bunch of new tests now (and if you want to contribute to the project, you can always write more tests and test cases for any bugs), you can run them with the suite script that Isaac wrote:

  1. cd tests/
  2. ./suite.sh

One thing I added (and loved it) is to run the tests using the JS port as well. Since the JS min part is using Mozilla's Rhino (slightly modified), Rhino is part of the code. So I'm using this already available JavaScript interpreter to run the JS port. Convenient.

The procedure to write new tests is simple:

  1. Create source CSS file in the tests/ directory, e.g. new-test.css
  2. Create a new file with the expected result and name it with a .min extension, e.g. new-test.css.min

You can use the handy-dandy online version to help with the tests creation.

Next time

With those details out of the way, the next time I'll talk more about the different things that CSSMin does to your CSS code. Thanks for reading!

 

One-click Minifier Gadget (OMG) - initial checkin

Sunday, January 31st, 2010

So I've been thinking and talking to folks about this idea of having one-stop shop for all your minification needs. Minification of JS and CSS as well as image optimization helps site performance by reducing download sizes. This is good. But not a lot of people do it.

People don't do it, because it's a PITA :) It's simple enough, but with deadlines upon you and all that, you don't want to do an extra step. That's why having a build process helps, by automating this. But setting up a build process is yet another PITA. So it goes.

So my idea was to help busy designers and developers, that wouldn't invest their time researching which minifiers are good, downloading setting up, learning about the 10+ PNG optimization tools... That's how the the idea for the one-click OMG tool came about. (One-drag is more appropriate, come to think of it...) One tool that runs on all operating systems - Win, Mac, Linux - and delivers all minification and optimization tools you need as one package.

Running

Running the tool is as simple as drag/dropping a bunch of files and directories. Here I've dropped "wordpress" directory. The tool recursively looks into the dropped files for things it can optimize. More information here.

OMG screenshot

Download

Version 0.0.1 is here. It doesn't do image optimization, only JS and CSS minification, but please feel free to download and give it a shot. Unzip the package for your OS and run omg.exe (Windows), OMG.app (Mac), or the omg binary (Linux)

Open source

The code is on GitHub. Fork and enjoy.

The developer's notes are there too - how to setup, run, package. Also a list of todos if you want to help.

Next?

This is just a preliminary version. Feel free to join, comment, suggest. Hate the name? Say so :)

Personally, looks like my plate is very full for the next moth or two, so I probably won't be actively working on the tool. I hope though the foundation is good enough and relatively documented, should be easy to pick up if anyone's interested in contributing.

Built with XUL

This has been a learning experience for me with XULRunner. I loved it. I love the idea of being able to create cross-OS desktop apps with JavaScript alone.

Behind the scenes, I'm using my JavaScript port of YUICompressor's CSSmin and Doug Crockford's JSMin. JSMin should be replaced with YUICompressor (or Google closure compiler) in the next release.

 

Extreme JavaScript optimization

Sunday, December 20th, 2009

Dec 20 This article is part of the 2009 performance advent calendar experiment. Today's article is a second contribution from Ara Pehlivanian (here's the first).

There's a Belorussian translation provided by Patricia. Thanks!

Ara PehlivanianAra Pehlivanian has been working on the Web since 1997. He's been a freelancer, a webmaster, and most recently, a Front End Engineer at Yahoo! Ara's experience comes from having worked on every aspect of web development throughout his career, but he's now following his passion for web standards-based front-end development. When he isn't speaking and writing about best practices or coding professionally, he's either tweeting as @ara_p or maintaining his personal site at http://arapehlivanian.com/.

There's an odd phenomenon underway in the JavaScript world today. Though the language has remained relatively unchanged for the past decade, there's an evolution afoot among its programmers. They're using the same language that brought us scrolling status bar text to write some pretty heavy duty client-side applications. Though this may seem like we're entering a Lada in an F1 race, in reality we've spent the last ten years driving an F1 race car back and forth in the driveway. We were never using the language at its full potential. It took the discovery of Ajax to launch us out of the driveway and onto the race track. But now that we're on the track, there's a lot of redlining and grinding of gears going on. Not very many people it seems, know how to drive an F1 race car. At least not at 250 mph.

The thing of it is, it's pretty easy to put your foot to the floor and get up to 60 mph. But very soon you'll have to shift gears if you want to avoid grinding to a halt. It's the same thing with writing large client-side applications in JavaScript. Fast processors give us the impression that we can do anything and get away with it. And for small programs it's true. But writing lots of bad JavaScript can very quickly get into situations where your code begins to crawl. So just like an average driver needs training to drive a race car, we need to master the ins and outs of this language if we're to keep it running smoothly in large scale applications.

Variables

Let's take a look at one of the staples of programming, the variable.Some languages require you to declare your variables before using them, JavaScript doesn't. But just because it isn't required doesn't mean you shouldn't do it. That's because in JavaScript if a variable isn't explicitly declared using the 'var' keyword, it's considered to be a global, and globals are slow. Why? Because the interpreter needs to figure out if and where the variable in question was originally declared, so it goes searching for it. Take the following example.

function doSomething(val) {
    count += val;
};

Does count have a value assigned to it outside the scope of doSomething? Or is it just not being declared correctly? Also, in a large program, having such generic global variable names makes it difficult to keep collisions from happening.

Loops

Searching the scope chain for where count is declared in the example above isn't such a big deal if it happens once. But in large-scale web applications, not very much just happens once. Especially when loops are concerned. The first thing to remember about loops, and this isn't just for JavaScript, is to do as much work outside the loop as possible. The less you do in the loop, the faster your loop will be. That being said, let's take a look at the most common practice in JavaScript loops that can be avoided. Take a look at the following example and see if you can spot it:

for (var i = 0; i < arr.length; i++) {
    // some code here
}

Did you see it? The length of the array arr is recalculated every time the loop iterates. A simple fix for this is to cache the length of the array like so:

for (var i = 0, len = arr.length; i < len; i++) {
    // some code here
}

This way, the length of the array is calculated just once and the loop refers to the cached value every time it iterates.

So what else can we do to improve our loop's performance? Well, what other work is being done on every iteration? Well, we're evaluating whether the value of i is less than the value of len and we're also increasing i by one. Can we reduce the number of operations here? We can if the order in which our loop is executed doesn't matter.

for (var i = 100; i--; ) {
    // some code here
}

This loop will execute 50% faster than the one above because on every iteration it simply subtracts a value from i, and since that value is not "falsy," in other words it isn't 0, then the loop goes on. The moment the value hits 0, the loop stops.

You can do this with other kinds of loops as well:

while (i--) {
    // some code here
}

Again, because the evaluation and the operation of subtracting 1 from i is being done at the same time, all the while loop needs is for i to be falsy, or 0, and the loop will exit.

Caching

I touched briefly on caching above when we cached the array length in a variable. The same principle can be applied in many different places in JavaScript code. Essentially, what we want to avoid doing is sending the interpreter out to do unnecessary work once it's already done it once. So for example, when it comes to crawling the scope chain to find a global variable for us, caching it the reference locally will save the interpreter from fetching it every time. Here, let me illustrate:

var aGlobalVar = 1;

function doSomething(val) {
    var i = 1000, agv = aGlobalVar;
    while (i--) {
        agv += val;
    };
    aGlobalVar = agv;
};

doSomething(10);

In this example, aGlobalVar is only fetched twice, not over a thousand times. We fetch it once to get its value, then we go to it again to set its new value. If we had used it inside the while loop, the interpreter would have gone out to fetch that variable a thousand times. In fact, the loop above takes about 3ms to run whereas if avg += val; were replaced with aGlobalVar += val; then the loop would take about 10ms to run.

Property Depth

Nesting objects in order to use dot notation is a great way to namespace and organize your code. Unforutnately, when it comes to performance, this can be a bit of a problem. Every time a value is accessed in this sort of scenario, the interpreter has to traverse the objects you've nested in order to get to that value. The deeper the value, the more traversal, the longer the wait. So even though namespacing is a great organizational tool, keeping things as shallow as possible is your best bet at faster performance. The latest incarnation of the YUI Library evolved to eliminate a whole layer of nesting from its namespacing. So for example, YAHOO.util.Anim is now Y.Anim.

Summary

These are just a few examples of how to improve your code's performance by paying attention to how the JavaScript interpreter does its work. Keep in mind though that browsers are continually evolving, even if the language isn't. So for example, today's browsers are introducing JIT compilers to speed up performance. But that doesn't mean we should be any less vigilant in our practices. Because in the end, when your web app is a huge success and the world is watching, every millisecond counts.

 

The new game show: “Will it reflow?”

Saturday, December 19th, 2009

Dec 19 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the articles to come.

Intrigued by Luke Smith's comment and also Alois Reitbauer's comment on the previous post about rendering I did some more testing with dynaTrace and SpeedTracer. Also prompted by this tweet, I wanted to provide an example of avoiding reflows by using document fragments as well as hiding elements with display: none. (btw, sorry that I'm slow to respond to tweets and blog comments, just too much writing lately with the crazy schedule, but I do appreciate every tweet and comment!)

So welcome to the new game show: "Will it reflow?" where we'll look into a few cases where it's not so clear if the browser will do a reflow or just a repaint. The test page is here.

Changing classnames

The first test is fairly straightforward - we only want to check what happens when you change the class name of an element. So using "on" and "off" class names and changing them on mouse over.

.on {background: yellow; border: 1px solid red;}
.off {background: white; border: 1px dashed green;}

Those CSS rules shouldn't trigger a reflow, because no geometry is being changed. Although the test is pushing it a bit by changing borders, which may affect geometry, but not in this case.

The test code:

// test #1 - class name change - will it reflow?
var onoff = document.getElementById('on-off');
onoff.onmouseover = function(){
  onoff.className = 'on' ;
};
onoff.onmouseout = function(){
  onoff.className = 'off';
};

Sooo.. will it reflow?

In Chrome - no! In IE - yes.

In IE, even changing the class name declarations to only change color, which is sure not to cause reflow, still caused a reflow. Looks like in IE, any type of className change causes a reflow.

cssText updates

The recommended way to update multiple styles in one shot (less DOM access, less reflows) is to update the element's style.cssText property. But.. will it reflow when the style changes do not affect geometry?

So let's have an element with a style attribute:

...style="border: 1px solid red; background: white"...

The JavaScript to update the cssText:

// test #2 - cssText change - will it reflow?
var csstext = document.getElementById('css-text');
csstext.onmouseover = function(){
  csstext.style.cssText += '; background: yellow; border: 1px dashed green;';
};
csstext.onmouseout = function(){
  csstext.style.cssText += '; background: white; border: 1px solid red;';
};

Will it reflow?

In Chrome - no! In IE - yes.

Even having cssText (and the initial style) only play with color, there's still a reflow. Even trying to just write the cssText property (as opposed to read/write with += ) still causes a reflow. The fact that cssText property is being updated causes IE to reflow. So there might be cases where setting individual properties separately (like style.color, style.backgroundColor and so on) which don't affect geometry, might be preferable to touching the cssText.

Next contestant in the game show is...

addRule

Will the browser reflow when you update stylesheet collections programatically? Here's the test case using addRule and removeRule (which in Firefox are insertRule/deleteRule):

// test #3 - addRule - will it reflow?
var ss = document.styleSheets[0];
var ruletext = document.getElementById('ruletext');
ruletext.onmouseover = function(){
  ss.addRule('.bbody', 'color: red');
};
ruletext.onmouseout = function(){
  ss.removeRule(ss.rules.length - 1);
};

Will it? Will it?

In Chrome - yes. The fact that style rules in the already loaded stylesheet have been touched, causes Chrome to reflow and repaint. Even though class .bbody is never used. Same when creating a new rule with selector body {...} - reflow, repaint.

In IE there's a repaint definitely, and there's also a kind of reflow. Looks like dynaTrace shows two kinds of rendering calculation indicators: "Calculating generic layout" and "Calculating flow layout". Not sure what is the difference (web searches disappointingly find nada/zero/rien for the first string and my previous blog post for the second). Hopefully "generic" would be less expensive than "flow".

display: none

In my previous post I boldly claimed that elements with display: none will not have anything to do with the render tree. IE begs to differ (thanks to dynaTrace folks for pointing that out).

A good way to minimize reflows is to update the DOM tree "offline" out of the live document. One way to do so is to hide the element while updates are taking place and then show it again.

Here's a test case where rendering and geometry are affected by simply adding more text content to an element by creating new text nodes.

// test #4 - display: none - will it reflow
var computed, tmp;
var dnonehref = document.getElementById('display-none');
var dnone = document.getElementById('bye');
if (document.body.currentStyle) {
  computed = dnone.currentStyle;
} else {
  computed = document.defaultView.getComputedStyle(dnone, '');
}

dnonehref.onmouseover = function() {
  dnone.style.display = 'none';
  tmp = computed.backgroundColor;
  dnone.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  dnone.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  dnone.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
}
dnonehref.onmouseout = function() {
  dnone.style.display = 'inline';
}

Will it reflow?

In Chrome - no. Although it does do "restyle" (calculating non-geometric styles) every time a text node is added. Not sure why this restyling is needed.

In IE - yes. Unfortunatelly display: none seems to have no effect on rendering in IE, it still does reflows. I tried with removing the show/hide code and having the element hidden from the very beginning (with an inline style attribute). Same thing - reflow.

document fragment

Another way to preform updates off-DOM is to create a document fragment and once ready, shove the fragment into the DOM. The beauty is that the children of the fragment get copied, not the fragment itself, which makes this method pretty convenient.

Here's the test/example. And will it reflow?

// test #5 - fragment - will it reflow
var fraghref = document.getElementById('fragment');
var fragment = document.createDocumentFragment();
fraghref.onmouseover = function() {
  fragment.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  fragment.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
  fragment.appendChild(document.createTextNode('No mo tests. '));
  tmp = computed.backgroundColor;
}
fraghref.onmouseout = function() {
  dnone.appendChild(fragment);
}

In Chrome - no! And no rendering activities take place until the fragment is added to the live DOM. Then, just like with display: none a restyle is being performed for every new text node inserted. And even though the behavior is the same for fragments as for updating hidden elements, fragments are still preferable, because you don't need to hide the element (which will cause another reflow) initially.

In IE - no reflow! Only when you add the final result to the live DOM.

Thanks!

Thank you for reading. Tomorrow if all goes well there should be a final post related to JavaScript and then moving on to ... other topics ;)

 

DOM access optimization

Friday, December 18th, 2009

Dec 18 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the articles to come.

This blog series has sailed from the shores of networking, passed down waterfalls and reflows, and arrived in ECMAScriptland. Now, turns out there's one bridge to cross to get to DOMlandia.

(OK, I need to get some sleep, evidently. Anyway.) Ara Pehlivanian talked about strategies for loading JavaScript code. Yesterday's post was about rendering and how you can prevent making things worse in JavaScript. Today's post will be about DOM access optimizations and, if all is good, tomorrow's post will round up the JavaScript discussion with some techniques for extreme optimization.

What's with the DOM

Document Object Model (DOM) is a language-independent API for accessing and working with a document. Could be an HTML document, or XML, SVG and so on. DOM is not ECMAScript. ECMAScript is just one way to work with the DOM API. They both started in the web browser but now things are different. ECMAscript has many other uses and so has the DOM. You can generate a page server side, using the DOM is you like. Or script Photoshop with ECMAScript.

All that goes to show that ECMAScript and DOM are now separate, they make sense on their own, they don't need each other. And they are kept separate by the browsers.

For example WebCore is the layout, rendering and DOM library used by WebKit, while JavaScriptCore (most recently rewritten as SquirrelFish) is the implementation of ECMAScript. In IE - Trident (DOM) and JScript. In Firefox - Gecko (DOM) and SpiderMonkey (ECMAScript).

The toll bridge

An excellent analogy I heard in this video from John Hrvatin of MSIE is that we can think of the DOM as a piece of land and JavaScript/ECMAScript as another piece of land. Both connected via a toll bridge. I tried to illustrate this analogy here.

domland and ecmaland connected with a bridge

All your JavaScript code that doesn't require a page - code such as loops, ifs, variables and a handful of built-in functions and objects - lives in ECMALand. Anything that starts with document.* lives in DOMLand. When your JavaScript needs to access the DOM, you need to cross that bridge to DOMlandia. And the bad part is that it's a toll bridge and you have to pay a fee every time you cross. So, the more you cross that bridge, the more you pay your performance toll.

How bad?

So, how serious is that performance penalty? Pretty serious actually. DOM access and manipulations is probably the most expensive activity you do in your JavaScript, followed by layouting (reflowing and painting activities). When you look for problems in your JavaScript (you use a profile instead of shooting in the dark, of course, but still) most likely it's the DOM that's slowing you down.

As an illustration, consider this bad, bad code:

// bad
for (var count = 0; count < 15000; count++) {
    document.getElementById('here').innerHTML += 'a';
}

This code is bad because it touches the DOM twice on every loop tick. It doesn't cache the reference to the DOM element, it looks for that element every time. Then this code also updates the live DOM which means it causes a reflow and a repaint (which are probably buffered by the browsers and executed in batches, but still bad).

Compare with the following code:

// better
var content = '';
for (var count = 0; count < 15000; count++) {
    content += 'a';
}
document.getElementById('here').innerHTML += content;

Here's we only touch the DOM twice at the end. The whole time otherwise we work in ECMAland with a local variable.

And how bad is the bad example? It's over 100 times worse in IE6,7 and Safari, over 200 times worse in FF3.5 and IE8 and about 50 times worse in Chrome. We're not talking percentages here - we talk 100 times worse.

Now obviously this is a bad and made up example, but it does show the magnitude of the problem with DOM access.

Mitigating the problem - don't touch the DOM

How to speed up DOM access? Simply do less of it. If you have a lot of work to do with the DOM, cache references to DOM elements so you don't have to query the DOM tree every time to find them. Cache the values of the DOM properties if you'll do a chunk of of work with them. And by cache I mean simply assign them to local variables. Use selectors API where available instead of crawling the DOM yourself (upgrade your JavaScript library if it's not taking advantage of the selectors API). Be careful with HTML Collections.

// bad
document.getElementById('my').style.top = "10px";
document.getElementById('my').style.left = "10px";
document.getElementById('my').style.color = "#dad";

// better
var mysty = document.getElementById('my').style;
mysty.top = "10px";
mysty.left = "20px";
mysty.color = "#dad";

// better
var csstext = "; top: 10px; left: 10px; color: #dad;";
document.getElementById('my').style.cssText += csstext

Basically, every time you find you're accessing some property or object repeatedly, assign it to a local variable and work with that local variable.

HTMLCollections

HTMLCollections are objects returned by calls to document.getElementsByTagName(), document.getElementsByClassName() and others, also by accessing the old-style collections document.links, document.images and the like. These HTMLCollection objects are array-like, list-like objects that contain pointers to DOM elements.

The special thing about them is that they are live queries against the underlying document. And they get re-run a lot, for example when you loop though the collection and access its length. The fact that you touch the length requires re-querying of the document so that the most up-to-date information is returned to you.

Here's an example:

// slow
var coll = document.getElementsByTagName('div');
for (var count = 0; count < coll.length; count++) {
    /* do stuff */
}

// faster
var coll = document.getElementsByTagName('div'),
    len = coll.length;
for (var count = 0; count < len; count++) {
    /* do stuff */
}

The slower version requeries the document, the faster doesn't because we use the local value for the length. How slower is the slower? Depends on the document and how many divs in it, but in my tests anywhere between 2 times slower (Safari) to 200 times slower (IE7)

Another thing you can do (especially if you'll loop the collection a few times) is to copy the collection into an array beforehand. Accessing the array elements will be significantly faster than accessing the DOM elements in the collection, again 2 to 200 times faster.

Here's an example function that turns the collection into an array:

function toArray(coll) {
    for (var i = 0, a = [], len = coll.length; i < len; i++) {
        a[i] = coll[i];
    }
    return a;
}

If you do that you also need to account for the one-off cost of copying that collection to an array.

Using event delegation

Event delegation is when you attach event listener to a parent element and it handles all the events for the children because of the so-called event bubbling It's a graceful way to relieve the browser from a lot of extra work. The benefits:

  • You need to write less event-attaching code.
  • You will usually use fewer functions to handle the events because you're attaching one function to handle parent events, not individual function for each child element. This means less functions to store in memory and keep track of.
  • Less events the browser needs to monitor
  • Easier to detach event handlers when an element is removed and therefore easier to prevenk IE memory leaks. Sometimes you don't even need to detach the event handler if children change, but the event-handling parent stays the same.

Thanks for reading!

  • Don't touch the DOM when you can avoid it, cache DOM access to local references
  • Cache length of HTMLCollections to a local variable while looping (good practice for any collections or arrays looping anyway). Copy the collection to an array if you'll be looping several times.
  • Use event delegation

Links

 

Rendering: repaint, reflow/relayout, restyle

Thursday, December 17th, 2009

Dec 17 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the articles to come.

Nice 5 "R" words in the title, eh? Let's talk about rendering - a phase that comes in the Life of Page 2.0 after, and sometimes during, the waterfall of downloading components.

So how does the browser go about displaying your page on the screen, given a chunk of HTML, CSS and possibly JavaScript.

The rendering process

Different browsers work differently, but the following diagram gives a general idea of what happens, more or less consistently across browsers, once they've downloaded the code for your page.

Rendering process in the browser

  • The browser parses out the HTML source code (tag soup) and constructs a DOM tree - a data representation where every HTML tag has a corresponding node in the tree and the text chunks between tags get a text node representation too. The root node in the DOM tree is the documentElement (the <html> tag)
  • The browser parses the CSS code, makes sense of it given the bunch of hacks that could be there and the number of -moz, -webkit and other extensions it doesn't understand and will bravely ignore. The styling information cascades: the basic rules are in the User Agent stylesheets (the browser defaults), then there could be user stylesheets, author (as in author of the page) stylesheets - external, imported, inline, and finally styles that are coded into the style attributes of the HTML tags
  • Then comes the interesting part - constructing a render tree. The render tree is sort of like the DOM tree, but doesn't match it exactly. The render tree knows about styles, so if you're hiding a div with display: none, it won't be represented in the render tree. Same for the other invisible elements, like head and everything in it. On the other hand, there might be DOM elements that are represented with more than one node in the render tree - like text nodes for example where every line in a <p> needs a render node. A node in the render tree is called a frame, or a box (as in a CSS box, according to the box model). Each of these nodes has the CSS box properties - width, height, border, margin, etc
  • Once the render tree is constructed, the browser can paint (draw) the render tree nodes on the screen

The forest and the trees

Let's take an example.

HTML source:

<html>
<head>
  <title>Beautiful page</title>
</head>
<body>

  <p>
    Once upon a time there was
    a looong paragraph...
  </p>

  <div style="display: none">
    Secret message
  </div>

  <div><img src="..." /></div>
  ...

</body>
</html>

The DOM tree that represents this HTML document basically has one node for each tag and one text node for each piece of text between nodes (for simplicity let's ignore the fact that whitespace is text nodes too) :

documentElement (html)
    head
        title
    body
        p
            [text node]

        div
            [text node]

        div
            img

        ...

The render tree would be the visual part of the DOM tree. It is missing some stuff - the head and the hidden div, but it has additional nodes (aka frames, aka boxes) for the lines of text.

root (RenderView)
    body
        p
            line 1
	    line 2
	    line 3
	    ...

	div
	    img

	...

The root node of the render tree is the frame (the box) that contains all other elements. You can think of it as being the inner part of the browser window, as this is the restricted area where the page could spread. Technically WebKit calls the root node RenderView and it corresponds to the CSS initial containing block, which is basically the viewport rectangle from the top of the page (0, 0) to (window.innerWidth, window.innerHeight)

Figuring out what and how exactly to display on the screen involves a recursive walk down (a flow) through the render tree.

Repaints and reflows

There's always at least one initial page layout together with a paint (unless, of course you prefer your pages blank :)). After that, changing the input information which was used to construct the render tree may result in one or both of these:

  1. parts of the render tree (or the whole tree) will need to be revalidated and the node dimensions recalculated. This is called a reflow, or layout, or layouting. (or "relayout" which I made up so I have more "R"s in the title, sorry, my bad). Note that there's at least one reflow - the initial layout of the page
  2. parts of the screen will need to be updated, either because of changes in geometric properties of a node or because of stylistic change, such as changing the background color. This screen update is called a repaint, or a redraw.

Repaints and reflows can be expensive, they can hurt the user experience, and make the UI appear sluggish.

What triggers a reflow or a repaint

Anything that changes input information used to construct the rendering tree can cause a repaint or a reflow, for example:

  • Adding, removing, updating DOM nodes
  • Hiding a DOM node with display: none (reflow and repaint) or visibility: hidden (repaint only, because no geometry changes)
  • Moving, animating a DOM node on the page
  • Adding a stylesheet, tweaking style properties
  • User action such as resizing the window, changing the font size, or (oh, OMG, no!) scrolling

Let's see a few examples:

var bstyle = document.body.style; // cache

bstyle.padding = "20px"; // reflow, repaint
bstyle.border = "10px solid red"; // another reflow and a repaint

bstyle.color = "blue"; // repaint only, no dimensions changed
bstyle.backgroundColor = "#fad"; // repaint

bstyle.fontSize = "2em"; // reflow, repaint

// new DOM element - reflow, repaint
document.body.appendChild(document.createTextNode('dude!'));

Some reflows may be more expensive than others. Think of the render tree - if you fiddle with a node way down the tree that is a direct descendant of the body, then you're probably not invalidating a lot of other nodes. But what about when you animate and expand a div at the top of the page which then pushes down the rest of the page - that sounds expensive.

Browsers are smart

Since the reflows and repaints associated with render tree changes are expensive, the browsers aim at reducing the negative effects. One strategy is to simply not do the work. Or not right now, at least. The browser will setup a queue of the changes your scripts require and perform them in batches. This way several changes that each require a reflow will be combined and only one reflow will be computed. Browsers can add to the queued changes and then flush the queue once a certain amount of time passes or a certain number of changes is reached.

But sometimes the script may prevent the browser from optimizing the reflows, and cause it to flush the queue and perform all batched changes. This happens when you request style information, such as

  1. offsetTop, offsetLeft, offsetWidth, offsetHeight
  2. scrollTop/Left/Width/Height
  3. clientTop/Left/Width/Height
  4. getComputedStyle(), or currentStyle in IE

All of these above are essentially requesting style information about a node, and any time you do it, the browser has to give you the most up-to-date value. In order to do so, it needs to apply all scheduled changes, flush the queue, bite the bullet and do the reflow.

For example, it's a bad idea to set and get styles in a quick succession (in a loop), like:

// no-no!
el.style.left = el.offsetLeft + 10 + "px";

Minimizing repaints and reflows

The strategy to reduce the negative effects of reflows/repaints on the user experience is to simply have fewer reflows and repaints and fewer requests for style information, so the browser can optimize reflows. How to go about that?

  • Don't change individual styles, one by one. Best for sanity and maintainability is to change the class names not the styles. But that assumes static styles. If the styles are dynamic, edit the cssText property as opposed to touching the element and its style property for every little change.
    // bad
    var left = 10,
        top = 10;
    el.style.left = left + "px";
    el.style.top  = top  + "px";
    
    // better 
    el.className += " theclassname";
    
    // or when top and left are calculated dynamically...
    
    // better
    el.style.cssText += "; left: " + left + "px; top: " + top + "px;";
  • Batch DOM changes and perform them "offline". Offline means not in the live DOM tree. You can:
    • use a documentFragment to hold temp changes,
    • clone the node you're about to update, work on the copy, then swap the original with the updated clone
    • hide the element with display: none (1 reflow, repaint), add 100 changes, restore the display (another reflow, repaint). This way you trade 2 reflows for potentially a hundred
  • Don't ask for computed styles excessively. If you need to work with a computed value, take it once, cache to a local var and work with the local copy. Revisiting the no-no example from above:
    // no-no!
    for(big; loop; here) {
        el.style.left = el.offsetLeft + 10 + "px";
        el.style.top  = el.offsetTop  + 10 + "px";
    }
    
    // better
    var left = el.offsetLeft,
        top  = el.offsetTop
        esty = el.style;
    for(big; loop; here) {
        left += 10;
        top  += 10;
        esty.left = left + "px";
        esty.top  = top  + "px";
    }
  • In general, think about the render tree and how much of it will need revalidation after your change. For example using absolute positioning makes that element a child of the body in the render tree, so it won't affect too many other nodes when you animate it for example. Some of the other nodes may be in the area that needs repainting when you place your element on top of them, but they will not require reflow.

Tools

Only about a year ago, there was nothing that can provide any visibility into what's going on in the browser in terms of painting and rendering (not that I am aware of, it's of course absolutely possible that MS had a wicked dev tool no one knew about, buried somewhere in MSDN :P). Now things are different and this is very, very cool.

First, MozAfterPaint event landed in Firefox nightlies, so things like this extension by Kyle Scholz showed up. mozAfterPaint is cool, but only tells you about repaints.

DynaTrace Ajax and most recently Google's SpeedTracer (notice two "trace"s :)) are just excellent tools for digging into reflows and repaints - the first is for IE, the second for WebKit.

Some time last year Douglas Crockford mentioned that we're probably doing some really stupid things in CSS we don't know about. And I can definitely relate to that. I was involved in a project for a bit where increasing the browser font size (in IE6) was causing the CPU go up to 100% and stay like this for 10-15 minutes before finally repainting the page.

Well, the tools are now here, we don't have excuses any more for doing silly things in CSS.

Except, maybe, speaking of tools..., wouldn't it be cool if the Firebug-like tools showed the render tree in addition to the DOM tree?

A final example

Let's just take a quick look at the tools and demonstrate the difference between restyle (render tree change that doesn't affect the geometry) and reflow (which affects the layout), together with a repaint.

Let's compare two ways of doing the same thing. First, we change some styles (not touching layout) and after every change, we check for a style property, totally unrelated to the one just changed.

bodystyle.color = 'red';
tmp = computed.backgroundColor;
bodystyle.color = 'white';
tmp = computed.backgroundImage;
bodystyle.color = 'green';
tmp = computed.backgroundAttachment;

Then the same thing, but we're touching style properties for information only after all the changes:

bodystyle.color = 'yellow';
bodystyle.color = 'pink';
bodystyle.color = 'blue';

tmp = computed.backgroundColor;
tmp = computed.backgroundImage;
tmp = computed.backgroundAttachment;

In both cases, these are the definitions of the variables used:

var bodystyle = document.body.style;
var computed;
if (document.body.currentStyle) {
  computed = document.body.currentStyle;
} else {
  computed = document.defaultView.getComputedStyle(document.body, '');
}

Now, the two example style changes will be executed on click on the document. The test page is actually here - restyle.html (click "dude"). Let's call this restyle test.

The second test is just like the first, but this time we'll also change layout information:

// touch styles every time
bodystyle.color = 'red';
bodystyle.padding = '1px';
tmp = computed.backgroundColor;
bodystyle.color = 'white';
bodystyle.padding = '2px';
tmp = computed.backgroundImage;
bodystyle.color = 'green';
bodystyle.padding = '3px';
tmp = computed.backgroundAttachment;

// touch at the end
bodystyle.color = 'yellow';
bodystyle.padding = '4px';
bodystyle.color = 'pink';
bodystyle.padding = '5px';
bodystyle.color = 'blue';
bodystyle.padding = '6px';
tmp = computed.backgroundColor;
tmp = computed.backgroundImage;
tmp = computed.backgroundAttachment;

This test changes the layout so let's called it "relayout test", the source is here.

Here's what type of visualization you get in DynaTrace for the restyle test.

DynaTrace

Basically the page loaded, then I clicked once to execute the first scenario (requests for style info every time, at about 2sec), then clicked again to execute the second scenario (requests for styles delayed till the end, at about 4sec)

The tool shows how the page loaded and the IE logo shows onload. Then the mouse cursor is over the rendering activity following the click. Zooming into the interesting area (how cool is that!) there's a more detailed view:

dynatrace

You can clearly see the blue bar of JavaScript activity and the following green bar of rendering activity. Now, this is a simple example, but still notice the length of the bars - how much more time is spent rendering than executing JavaScript. Often in Ajax/Rich apps, JavaScript is not the bottleneck, it's the DOM access and manipulation and the rendering part.

OK, now running the "relayout test", the one that changes the geometry of the body. This time check out this "PurePaths" view. It's a timeline plus more information about each item in the timeline. I've highlighted the first click, which is a JavaScript activity producing a scheduled layout task.

dynatrace

Again, zooming into the interesting part, you can see how now in addition to the "drawing" bar, there's a new one before that - the "calculating flow layout", because in this test we had a reflow in addition to the repaint.

dynatrace

Now let's test the same page in Chrome and look at the SpeedTracer results.

This is the first "restyle" test zoomed into the interesting part (heck, I think I can definitely get cused to all that zooming :)) and this is an overview of what happened.

speedtracer

Overall there's a click and there's a paint. But in the first click, there's also 50% time spent recalculating styles. Why is that? Well, this is because we asked for style information with every change.

Expanding the events and showing hidden lines (the gray lines were hidden by Speedtracer because they are not slow) we can see exactly what happened - after the first click, styles were calculated three times. After the second - only once.

speedtracer

Now let's run the "relayout test". The overall list of events looks the same:

speedtracer

But the detailed view shows how the first click caused three reflows (because it asked for computed style info) and the second click caused only one reflow. This is just excellent visibility into what's going on.

speedtracer

A few minor differences in the tools - SpeedTracer didn't show when the layout task was scheduled and added to the queue, DynaTrace did. And then DynaTrace didn't show the details of the difference between "restyle" and "reflow/layout", like SpeedTracer did. Maybe simply IE doesn't make a difference between the two? DynaTrace also didn't show three reflows instead of one in the different change-end-touch vs. change-then-touch tests, maybe that's how IE works?

Running these simple examples hundreds of times also confirms that for IE it doesn't matter if you request style information as you change it.

Here's some more data points after running the tests with enough repetitions:

  • In Chrome not touching computed styles while modifying styles is 2.5 times faster when you change styles (restyle test) and 4.42 times faster when you change styles and layout (relayout test)
  • In Firefox - 1.87 times faster to refrain from asking computed styles in restyle test and 1.64 times faster in the relayout test
  • In IE6 and IE8, it doesn't matter

Across all browsers though changing styles only takes half the time it takes to change styles and layout. (Now that I wrote it, I should've compared changing styles only vs. changing layout only). Except in IE6 where changing layout is 4 times more expensive then changing only styles.

Parting words

Thanks very much for working through this long post. Have fun with the tracers and watch out for those reflows! In summary, let me go over the different terminology once again.

  • render tree - the visual part of the DOM tree
  • nodes in the render tree are called frames or boxes
  • recalculating parts of the render tree is called reflow (in Mozilla), and called layout in every other browser, it seems
  • Updating the screen with the results of the recalculated render tree is called repaint, or redraw (in IE/DynaTrace)
  • SpeedTracer introduces the notion of "style recalculation" (styles without geometry changes) vs. "layout"

And some more reading if you find this topic fascinating. Note that these reads, especially the first three, are more in depth, closer to the browser, as opposed to closer to the developer which I tried to do here.

 

JavaScript loading strategies

Tuesday, December 15th, 2009

Dec 15 This article is part of the 2009 performance advent calendar experiment. Today's article is a contribution from Ara Pehlivanian, author of two JavaScript books. Please welcome Ara and stay tuned for the articles to come.

Ara PehlivanianAra Pehlivanian has been working on the Web since 1997. He's been a freelancer, a webmaster, and most recently, a Front End Engineer at Yahoo! Ara's experience comes from having worked on every aspect of web development throughout his career, but he's now following his passion for web standards-based front-end development. When he isn't speaking and writing about best practices or coding professionally, he's either tweeting as @ara_p or maintaining his personal site at http://arapehlivanian.com/.

JavaScript has a dark side to it that not many people are aware of. It causes the browser to stop everything that it's doing until the script has been downloaded, parsed and executed. This is in sharp contrast to the other dependencies which get loaded in parallel--limited only by the number of connections the browser and server are able to create. So why is this a problem?

Good question! Before I can answer that, I need to explain how the browser goes about building a page. The first thing a it's does once it receives an HTML document from the server is to build the DOM--an object representation of the document in memory. As the browser goes about converting HTML into the DOM, it invariably encounters references to external dependencies such as CSS documents and images. Every time it does so, it fires off a request to the server for that dependency. It doesn't need to wait for one to be loaded before requesting another, it makes as many requests as it's capable of. This way, the page gets built one node at a time and as the dependencies come in, they're put in their correct placeholders. What gums up the works though, is when a JavaScript dependency is encountered. When this happens, the browser stops building the DOM and waits for that file to arrive. Once it receives the file, it parses and executes it. Only once all of that's done does the browser continue building the DOM. I suspect this has to do with wanting to provide as stable a DOM to the script as possible. If things were in flux while the script attempted to access or even modify a DOM node, things could get dicey. Either way, the time it takes before the browser can continue depends entirely on the size and complexity of the script file that's being loaded.

Now imagine loading a 200k JavaScript file right in the <head> of a document. Say it's a JavaScript file that's not only heavy but also does some fairly complex computing that takes half a second to complete. Imagine now what would happen if that file took a second to transfer. Did you guess? Yup, the page would be blank until that transfer and the computation were complete. A second and a half of a blank page that the visitor has to endure. Given that most people don't spend more than a few seconds on the average web page, that's an eternity of staring at a blank page.

Reduce

So how can this problem be overcome? Well, the first thing that should be done, is to reduce as much as possible, the amount of data that's being sent over the pipe. The smaller the JavaScript file, the less waiting the visitor has to do. So what can be done to reduce file size? JavaScript files can be run through a minifier such as YUI Compressor (which removes unnecessary white space and formatting, as well as comments, and is proven to reduce file size by 40-60%). Also, if at all possible, servers should be set up to gzip files before they're sent. This can drastically reduce the number of bytes that get transferred since JavaScript is plain text, and plain text compresses really well.

Defer

So, once you've made sure your file is as small as possible, what next? Well, the first thing is to make sure the visitor has something to look at while the script is loading. Instead of loading JavaScript files in the document's <head>, put your <script> tags immediately before your page's closing </body> tag. That way, the browser will have built the DOM and begun inserting images and applying CSS long before it encounters your script tags. This also means that your code will execute faster because it won't need to wait for the page's onload event--which only fires once all the page's dependencies are done loading.

So with the script tags placed at the end of the document, when the browser does encounter them, will still halt operations for however long it needs to, but at this point the visitor is reading your page and unaware of what's going on behind the scenes. You've just bought yourself the time to surreptitiously load your script files.

Go Async

There is another way to load JavaScript files which won't block your browser, and that's to insert the script tags into your page using JavaScript. Dynamically including a script tag into the DOM causes it to be loaded asynchronously. The only trouble with that is that you can't rely on the code within the script file to be available immediately after you've included it. What you'll need is a callback function that is executed once your script is done loading. There are several ways of doing this. A lot of libraries have built in async script loading functionality, so you're likely better off using that. But if you want to do it yourself, be ready to deal with the idiosyncrasies of different browsers. For example, where one browser will fire off an onload event for the script, another will not.

Be Lazy

So now that we know how to load scripts behind the scenes, is there anything more we can do to improve performance? Of course.

Say for example your page loads up a large script that gives your site a fancy navigation menu. What if the user never uses the navigation menu? What if they only navigate your site through links in your content? Did you really need to load that script in the first place? What if you could load the necessary code only when it was needed? You can. It's a technique called lazy loading. The principle is simple, instead of binding your fancy navigation script to the menu in your page, you'd bind a simple loader script instead. It would detect an onmouseover event for example, and then insert a script tag with the fancy nav code into the page. Once the tag is done loading, a callback function wires up all the necessary events and presto bingo, your nav menu starts working. This way, your site doesn't have to needlessly bog visitors down with code they'll never use.

Bite Size

In keeping with lazy loading, try to also load only the core components that are needed to make your page work. This is especially the case when it comes to libraries. A lot of the time a library will force you to load up a huge amount of code when all you want to do is add an event handler, or modify class names. If the library doesn't let you pull down only what you need, try ripping out what you want and only load that instead. There's no point in forcing visitors to download 60k of code when all you need is 4k of it.

Do You Need It?

Finally, the best way to speed up JavaScript load times is to not include any JavaScript at all. A lot of times people go nuts for the latest fad and include it in their site without even asking themselves if they really need it. Does this fancy accordion thing actually help my visitors get to my content easier? Does fading everything in and out and bouncing things all over the place actually improve my site's usability? So the next time you feel like adding a three dimensional spinning rainbow tag cloud to your site, ask yourself, "do I really need this?"

Note from Stoyan:

I'd like to thank Ara for the great article, it's pleasure for me to be the blog host!

Also wanted to offer some additional links for your reading pleasure:

Please comment if you can think of more good resources on the topic.

 

Reducing the payload: compression, minification, 204s

Friday, December 11th, 2009

Dec 11 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the next articles.

After removing all the extra HTTP requests you possibly can from your waterfall, it's time to make sure that those that are left are as small as they can be. Not only this makes your pages load faster, but it also helps you save on the bandwidth bill. Your weapons for fighting overweight component include: compression and minification of text-based files such as scripts and styles, recompression of some downloadable files, and zero-body components. (A follow-up post will talk about optimizing images.)

Gzipping plain text components

Hands down the easiest and at the same time quite effective optimization - turning on gzipping for all plain text components. It's almost a crime if you don't do it. Doesn't "cost" any development time, just a simple flip of a switch in Apache configuration. And the results could be surprisingly pleasant.

When Bill Scott joined Netflix, he noticed that gzip is not on. So they turned it on. And here's the result - the day they enabled it, the outbound traffic pretty much dropped in half (slides)

Netflix traffic after turning on gzipping

Gzip FAQ

How much improvement can you expect from gzip?
On average - 70% reduction of the file size!
Any drawbacks?
Well, there's a certain cost associated with the server compressing the response and the browser uncompressing it, but it's negligible compared to the benefits you get
Any browser quirks?
Sure, IE6, of course. But only in IE6 service pack 1 and fixed for after that. You can boldly ignore this edge case, but if you're extra paranoid you can disable gzip for this user agent
How to tell if it's on?
Run YSlow/PageSpeed and they'll will warn you if it's not on. If you don't have any of those tools just look at the HTTP headers with any other tool, e.g. Firebug, webpagetest.org. You should see the header:

Content-Encoding: gzip

provided, of course, that your browser claimed it supports compression by sending the header:

Accept-Encoding: gzip, deflate
What types of components should you gzip?
All text components:

  • javascripts
  • css
  • plain text
  • html, xml, including any other XML-based format such as SVG, also IE's .htc
  • JSON responses from web service calls
  • anything that's not a binary file...

You should also gzip @font-files like EOT, TTF, OTF, with the exception of WOFF. Average about 40% to be won there with font files.

How-to turn on gzipping

Ideally you need control over the Apache configuration. If not full control, at least most hosting providers will offer you ability to tweak configuration via .htaccess. If your host doesn't, well, change the host.

So just add this to .htaccess:

AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/javascript application/json

If you're on Apache before version 2 or your unfriendly host don't allow any access to configuration, not all is lost. You can make PHP do the gzipping for you. It's not ideal but the gzip benefits are so pronounced that it's worth the try. This article describes a number of different options for gzipping when dealing with uncooperative hosts.

Rezipping

As Billy Hoffman discovered, there's potential for file size reduction with common downloadable files, which are actually zip files in disguise. Such files include:

  • Newer MS Office documents - DOCX, XLSX, PPTX
  • Open Office documents - ODT, ODP, ODS
  • JARs (Java Applets, anyone?)
  • XPI Firefox extensions
  • XAP - Silverlight applications

These ZIP files in disguise are usually not compressed with the maximum compression. If you allow such downloads from your website, consider recompressing them beforehand with maximum compression.

There could be anywhere from 1 to 30% size reduction to be won, definitely worth the try, especially since you can do it all on the command line, as part of the build process, etc. (re)Compress once, save bandwidth and offer faster downloads every time ;)

15% uncompressed traffic

Tony Gentilcore of Google reported his findings that a significant chunk of their traffic is still sent uncompressed. Digging into it he realized there's a number of anti-virus software and firewalls that will mingle with the browser's Accept-Encoding header changing into the likes of:

Accept-Encoding: xxxx, deflxxx
Accept-Enxoding: gzip, deflate

Since this is an invalid header, the server will decide that the browser doesn't support gzip and send uncompressed response. And why would the retarded anti-virus program do it? Because it doesn't want to deal with decompression in order to examine the content. Probably not to slow down the experience? In doing so it actually hurts the user to a greater extend.

So compression is important, but unfortunately it's not always present. That's why minification helps - not only because compressing minified responses is even smaller, but because sometimes there is no compression despite your best efforts.

Minification

Minification means striping extra code from your programs that is not essential for execution. The code in question is comments, whitespace, etc from styles and scripts, but also renaming variables with shorter names, and various other optimizations.

This is best done with a tool, of course, and luckily there a number of tools to help.

Minifying JavaScript

Some of the tools to minify JavaScript include:

How much size reduction can you expect from minification? To answer that I ran jQuery 1.3.2. through all the tools mentioned above (using hosted versions) and compared the sizes before/after and with/without gzipping the result of minification.

The table below lists the results. All the % figures are % of the original, so smaller is better. 29% means the file was reduced to 29% of its original version, or a saving of 71%

File original size size, gzipped % of original gzip, % of original
original 120619 35088 100.00% 29.09%
closure-advanced 49638 17583 41.15% 14.58%
closure 55320 18657 45.86% 15.47%
jsmin 73690 21198 61.09% 17.57%
packer 39246 18659 32.54% 15.47%
shrinksafe 69516 22105 57.63% 18.33%
yui 57256 19677 47.47% 16.31%

As you can see gzipping alone gives you about 70% savings, minification alone cuts script sizes with more than half and both combined (minifying then gzipping) can make your scripts 85% leaner. Verdict: do it. The concrete tool you use probably doesn't really matter all that much, pick anything you're comfortable with to run before deployment (or best, automatically during a build process)

Minifying CSS

In addition to the usual stripping of comments and whitespaces, more advanced CSS minification could include for example:

// before
#mhm {padding: 0px 0px 0px 0px;}
// after
#mhm{padding:0}

// before
#ha{background: #ff00ff;}
// after
#ha{background:#f0f}
//...

A CSS minifier is much less powerful than a JS minifier, it cannot rename properties or reorganize them, because the order matters and for example text-decoration:underline cannot get any shorter than that.

There's not a lot of CSS minifiers, but here's a few I tested:

  • YUI compressor - yes, the same YUI compressor that does JavaScript minification. I've actually ported the CSS minification part of it to JavaScript (it's in Java otherwise) some time ago. There's even an online form you can paste into to test. The CSS minifier is regular expression based
  • Minify is a PHP based JS/CSS minification utility started by Ryan Grove. The CSS minifier part is also with regular expressions, I have the feeling it's also based on YUICompressor, at least initially
  • CSSTidy - a parser and an optimizer written in PHP, but also with C version for desktop executable. There's also a hosted version. It's probably the most advanced optimizer in the list, being a parser it has a deeper understanding of the structure of the styleshets
  • HTML_CSS from PEAR - not exactly an optimizer but more of a general purpose library for creating and updating stylesheets server-side in PHP. It can be used as a minifier, by simply reading, then printing the parsed structure, which strips spaces and comments as a side effect.

Trying to get an average figure of the potential benefits, I ran these tools on all stylesheets from csszengarden.com, collected simply like:

<?php
$urlt = "http://csszengarden.com/%s/%s.css";
for ($i = 1; $i < 214; $i++) {
  $id = str_pad($i, 3, "0", STR_PAD_LEFT);
  $url = sprintf($urlt, $id, $id);
  file_put_contents("$id.css", file_get_contents($url));
}
?>

3 files gave a 404, so I ran the tools above on the rest 210 files. CSSTidy ran twice - once with its safest settings (which even keep comments in) and then with the most aggressive. The "safe" way to use CSSTidy is like so:

<?php
// dependencies, instance
include 'class.csstidy.php';
$css = new csstidy();

// options
$css->set_cfg('preserve_css',true);
$css->load_template('high_compression');

// parse
$css->parse($source_css_code);

// result
$min = $css->print->plain();
?>

The aggressive minification is the same only without setting the preserve_css option.

Running Minify is simple:

<?php
// dependencies, instance
require 'CSS.php';
$minifier = new Minify_CSS();

// minify in one shot
$min = $minifier->minify($source_css_string_or_url);

As for PEAR::HTML_CSS, since it's not a minifier, you only need to parse the input and print the output.

<?php
require 'HTML/CSS.php';

$options = array(
    'xhtml' => false,
    'tab' => 0,
    'oneline' => true,
    'groupsfirst' => false,
    'allowduplicates' => true,
);

$css = new HTML_CSS($options);
$css->parseFile($input_filename);
$css->toFile($output_filename);
// ... or alternatively if you want the result as a string
// $minified = $css->toString();

So I ran those tools on the CSSZenGarden 200+ files and the full table of results is here, below are just the averages:

  Original YUI Minify CSSTidy-safe CSSTidy-small PEAR
raw 100% 68.18% 68.66% 84.44% 63.29% 74.60%
gzipped 30.36% 19.89% 20.74% 28.36% 19.44% 20.20%

Again, the numbers are percentage of the original, so smaller is better. As you can see, on average gzip alone gives you 70% size reduction. The minification is not so successful as with JavaScript. Here even the best tool cannot reach 40% reduction (for JS it was usually over 50%). But nevertheless, gzip+minification on average gives you a reduction of 80% or more. Verdict: do it!

An important note here is that in CSS we deal with a lot of hacks. Since the browsers have parsing issues (which is what hacks often exploit), what about a poor minifier? How safe are the minifiers? Well, that's a subject for a separate study, but I know I can at least trust the YUICompressor, after all it's used by hundreds of Yahoo! developers daily and probably thousands non-Yahoos around the world. PEAR's HTML_CSS library also looks pretty safe because it has a simple parser that seems to tolerate all kinds of hacks. CSSTidy also claims to tolerate a lot of hacks, but given that the last version is two years old (maybe new hacks have surfaced meanwhile) and the fact that it's the most intelligent optimizer (knows about values, colors and so on) it should be approached with care.

204

Let's wrap up this lengthy posting with an honorable mention of the 204 No Content response (blogged before). It's the world's smallest componet, the one that has no body and a Content-Length of 0.

Often people use 1x1 GIFs for logging and tracking purposes and other types of requests that don't need a response. If you do this, you can return a 204 status code and no response body, only headers. Look no further that Google search results with your HTTP sniffer ON to see examples of 204 responses.

The way to send a 204 response from PHP is simply:

header("HTTP/1.0 204 No Content");

A 204 response saves just a little bit but, hey, every little bit helps.

And remember the mantra: every extra bit is a disservice to the user :)

Thank you for reading!

Stay tuned for the next article continuing the topic of reducing the component sizes as much as possible.

 

Duplicates and near-duplicates

Wednesday, December 9th, 2009

Dec 9 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the next articles.

One of Yahoo!'s first batch of performance best practices has always been "Avoid duplicate scripts" (check Steve Souders' post). Later we added "... and styles". This is a pretty obvious, kind of a "Duh!" type of recommendation, it's like saying "Avoid sleep() in your server-side scripts". But it didn't come up out of thin air, duplicates were noticed on some quite high-profile sites.

Duplicates are easy to spot (and YSlow will warn you), but let's talk a bit about another concept - let's call it near-duplicates - when two components are similar, almost the same, but not quite.

Duplicate scripts and styles

As a refresher and a quick illustration of the effects of duplicate scripts and styles, fire off your HTTP sniffer and hit this test page.

(btw, this is a simple page I put up to test different YSlow scenarios, you can actually use it as a web service of sorts to create any type of components with different options)

Firefox 2 downloading both duplicate styles and scripts:

FF2 duplicate scripts and styles

IE6 and duplicate scripts:

IE duplicate scripts

Exact details of when/which browsers chose to download duplicates are not that interesting, it's obviously bad to waste time downloading the same resource. Even if no repeating download happens, the browser still has to parse through and execute the script/style for a second time.

Even if you have iframes you don't need to repeat the same JS/CSS in each frame, you can "borrow" them from the parent page, here's an example.

Near-duplicates

Near duplicates can be:

  • components with the exact same response bodies but different URLs causing the browser to do double work
  • components (images) that are too close to each other - in terms of looks or purpose. Only one component should be selected in this case.

Same component, different URLs

This could happen especially when you have user-generated content such as image uploads for profile photos and avatars in social sites, forums, images people put in comments on MySpace and so on.

Also images of stuff for sale (Craigslist, eBay). Often different sellers offering the same item would take the same photo from the manufacturer's site and upload it over and over again.

Luckily, PageSpeed warns about components with identical content, so those can be identified:

In the screenshot above, you see one image (2.3K) repeated 3 times, another (the iPhone, 1.7K) is repeated 4 times, and yet another one (2.8K) repeated 2 times.

It's not exactly trivial to avoid this type of duplications with user-generated content (for example, the first poster may delete the photo, in which case the second poster's photo will need to "shine through"). But it's not impossible, using for example a hash of the component's content as an identifier.

Loading...

Ajax loading indicators are a great idea to give feedback to the user that something is happening. They come in all shapes and sizes... sometimes on the same page, unfortunately. And again, sometimes it's the same stock image but used at different stages of gradual "ajaxification" of the page and with different URLs.

As we're moving more and more towards modular pages and client-side logic, often different modules on the same page are coded by different teams at different times, independently, without being aware of each other's assets. This way of building pages has it's challenges and one is that common components, such as Ajax loading indicators, should be shared.

Too similar modules

Along the same lines - different modules are sometimes created by different designers at different times. The result - one rounded corner box with 1px shadow and one with 2px shadow, both on the same page. Or two different shades of the same gray color, which no one can tell apart. That's just a waste. (See Nicole Sullivan's presentation for illustration, e.g. slides 44, 45)

Below is an example, can you tell that these 5 rounded corner boxes are all different - slightly different shadow, color or radius? How many different boxes does this page need?

Different sizes of the same image

It's highly recommended to not scale images in HTML (or CSS). If you need a 100x100 image you don't use a 400x400 one with <img width="100" height="100" ... />. That's a good rule of thumb... to break sometimes ;)

In cases where the same image is used with different sizes and likely even on the same page, it may be beneficial to reuse the same bigger image and scale it down, because this could be saving extra HTTP requests of downloading the same (but slightly smaller) image.

Facebook is an example, the same hairy guy on the screenshot has two images with different sizes. It's actually the same image but resized in CSS.

The relevant CSS which shows the profile image in LARGE and SMALL (and looks like there's even a TINY view, although I couldn't find an example on this page)

.UIProfileImage_LARGE{width:50px;height:50px}
.UIProfileImage_SMALL{width:32px;height:32px}
.UIProfileImage_TINY{width:25px;height:25px}

Thank you!

Thanks for reading! Reducing HTTP requests is critical for page performance. You've merged your scripts and styles as much as reasonable, you've crafted CSS sprites and inlined images with data URIs. Now it's time to look at what's left - are there components that are way too similar, are there any near-duplicates or even exact duplicates? Same image on different backgrounds? Ever-so-subtle gradients and shadows? Time to pick up the old axe and cut.

 

Reducing the number of page components

Saturday, December 5th, 2009

Dec 5 This is the fifth in the series of performance articles as part of my 2009 performance advent calendar experiment. Stay tuned for the next articles.

Let's talk a but about waterfall optimization - the first thing that happens in Mr.Page's life. The best way to optimize and speed up the waterfall is to have less stuff in it. The fewer page components, the faster the page - simple as that.

Fewer components vs. component weight

The size of the page components, meaning their size in kB, is important. It makes sense - smaller pages will load faster, 100K JavaScript will load faster than 150K. It's important to keep sizes low, but it should be clear that the number of components is even more important than their filesize.

Why? Because every HTTP request has overhead.

OK, but how bad can it be, someone might ask. If you look at an HTTP request - it has a header and a body. A 100K body will greatly outweigh the size of the headers, no matter how bloated they are.

Here's the headers for a request to Yahoo! Search:

Host: search.yahoo.com
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5;) Firefox/3.5.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

To that request the server responds with the body (content) of the response prepended with some headers like:

HTTP/1.1 200 OK
Date: Sat, 05 Dec 2009 07:36:25 GMT
P3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR... blah, blah"
Set-Cookie: sSN=nTMt3Lo2...crazy stuff...nLvwVxUU; path=/;domain=.search.yahoo.com
Cache-Control: private
Connection: close
Content-Type: text/html; charset=ISO-8859-1

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">

<html lang="en"><head><meta...

This is 352 bytes request header and 495 bytes response header. Not so bad, eh? No matter how hard you try to make your cookies monster-size, the response body (9k gzipped in this case) will always be significantly bigger. So what's the problem with the overhead of the HTTP requests then?

The size of the headers is a problem when you make requests for small components - say requests for small icons - for example 1K or under. In this case you exchange 1K headers to get 1K of useful data to present to the user. Clearly a waste. Additionally this 1K of headers can grow once you start writing more cookies. It may very well happen that the HTTP headers size is bigger than the actual icon you need. And even if the headers are not bigger that the component, they are still big when you think percent-wise. 1K of 10K is 10%.

But the HTTP headers size is only one (and the smaller) of the problems.

The bigger problem is the HTTP connection overhead.

HTTP connection overhead

What happens (at a high level) when you type a URL and hit Enter? The browser sends a request to the server. Which server? The browser needs to know the IP address of the server, so if it doesn't have it in the cache, it makes a DNS lookup. Than the browser establishes a connection to the server. Then it waits for the first byte of the response from the server. Then it receives the full response (payload).

Here's how this looks like graphically represented by webpagetest.org

And the color legend:

  1. DNS lookup
  2. Initial connection
  3. TTFB (Time to first byte)
  4. Payload

So what do we have here - looks like in this particular case the browser is downloading content about 40% of the time. The rest of the time it's... well, not downloading content. How's that for an overhead. And the part of not downloading can be even greater, this above was just one example.

Now how about this - a bird-eye view of the 4 last pages in webpagetest.org's test history - just some random pages people have tested.

Do you see a lot of blue (time spent downloading content). Not as much as you would've hoped. There's some DNS lookups, some orange... and OMG, talk about going green! :)

In fact you may notice that the smaller the component, the smaller is the blue part.

What does all this tell us?

  1. A significant chunk of time is spent in activities other than downloading.
  2. Smaller components still incur HTTP overhead and for them the relative penalty (relative to their size) is atrocious.

So what's a performance optimizer to do? Reduce the number of components and so pay fewer penalties.

Just remove stuff

Truth is - a lot of stuff on the pages today is not needed. Features no one likes or uses clutter the page and make it heavier. Well, what can you do, the boss/client/marketing guy wants that feature there. What you can do is at least try. You can introduce some science into the marketing activities - measure how much a specific feature is used. Or if you already have the data - look at it. Decide what can a page go without.

It's going to be tough convincing people to remove stuff. After all you spend time developing it. Someone dreamt up that feature initially. Someone (not the users) loves it. People hate to let go. But still, it's worth to try.

Combine components

Now that the phase of convincing people to remove stuff is done, what's left needs to be combined. How do you combine components? Simple - all JavaScripts go into a single file, all CSS into a single file. All decoration images go into a sprite.

JavaScript example (from a page to remain anonymous)

Before:

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js"></script>
<script src="/javascripts/application.js?1258423604"></script>
<script src="/javascripts/ui/minified/jquery.ui.all.min.js?1258423604"></script>
<script src="/javascripts/ui-ext/ui.bgiframe.min.js?1258423604"></script>
<script src="/javascripts/ui-ext/ui.stars.pack.js?1258423604"></script>
<script src="/javascripts/ui-ext/ui.dimensions.js?1258423604"></script>
<script src="/javascripts/ext/jquery.form.min.js?1258423604"></script>

After:

<script src="/javascripts/all.js"></script>

Sizes, gzipped: 70029 bytes before, 65194 bytes after. Simply merging files and there's even 6.9% saving!
And the more important saving: 6 less HTTP requests

Repeat for CSS. Before:

/stylesheets/general.css?1258423604
/stylesheets/global.css
/stylesheets/ui.stars.css
/stylesheets/themes/enation/enation.all.css
/public/template/css/132/1245869225

After:

<link type="text/css" rel="stylesheet" href="/stylesheets/all.css" />

Sizes, gzipped: before 14781 bytes, after 13352 bytes, saving 9.6%.
But the bigger saving: 4 less HTTP requests.

If you wonder how come the sizes before and after are different since we merely concatenate the contents of the files, well, the savings come from gzip compression. When you have more characters in the file, there's more chance that some will repeat which means they will compress better. That's one. And then, the compression itself has an overhead which you incur once for the whole bundle of files, as opposed to for every file.

Now - let's make decoration images into sprites. Before:

... 15 image requests, 6.8K

After: (1 sprited image)

Result size: 1.4K, 7 times smaller!

Here the savings are so dramatic partially because the source files are GIFs and the result is a PNG8 but that's a whole other post.

So in conclusion: file concatenation is just awesome. You save both: bytes to download and, much more importantly, HTTP requests. Less of the green stuff in the waterfall!

x-type component concatenation

So far we combined .js with .js, css with css and images with images. How about a cross-component type concatenation?

You can inline images inside HTML and CSS (and why not JS if you want) using data URIs (another post coming).

And you can inline CSS and JS inside HTML too.

This means you can have your whole application inside of just one HTML file if you wish. Inside of the HTML you have inline styles, scripts and images.

Combining CSS with JS

Now how about mixing CSS and JS into one component. You can do that, and it's especially suitable for lazy-loaded, widget-type functionality.

Say you've loaded the page, then the user clicks some rarely used button. You haven't downloaded that content that is supposed to wow the user upon click of the button. So you send a request to grab it. The new content may come in the form of a JSON string. And what if the new content requires some stylesheet that was not part of the base page? You'll have to make another request to download that stylesheet too.

Or, you can download both content and styles in the same JSON response. You simply inline the style information as a string in the JSON. So:

1. uses clicks, you request feature.js which goes like:

{"content":"<p class=\"wow\">I'm a feature</p>", "style": "wow{font-size: 60px}"}

2. You process the JSON and shove the content into the page

var o = JSON.parse(xhr.responseText);
$('result').innerHTML = o.content;

3. You add the styles to the head:

var wow = document.createElement('style');
wow.type = "text/css";
if (wow.textContent) { // FF, Safari
    wow.textContent = o.style;
} else {
    wow.styleSheet.cssText = o.style; // FF, IE
}
document.documentElement.firstChild.appendChild(wow);

Nice and simple. Makes the features (that progressively enhance the page) atomic and self-contained.

More to reducing components?

For more and creative ways to reduce HTTP components you can take a look at MXHR and Comet

Another thing to check is the Keep-Alive setting on your server. Remember how there were 4 steps in the component download. When you request a second component you can have the connection open so that you don't need to re-establish it (skipping step 2). And since the DNS lookup was already made you get rid of step 1. Skipping 2 out of 4 is not bad at all.

Summary

Reducing the number of page of components is the top priority of any web performance optimization effort. HTTP requests are costly. In part because of the headers size overhead, but mostly because of the connection overhead. The browser spends a disturbing amount of time not downloading stuff, and we can't allow this!

 

Statsy - more data points for markup quality

Monday, November 30th, 2009

In the spirit of the content-to-markup ratio bookmarklet, here's another one that gives you some more data points to help you judge the quality of a page's markup and help answer the old question - where does all this page weight go.

Install the statsy bookmarklet

Drag this link to your bookmarks:

statsy

the results

Once you run the bookmarklet it alerts these stats points:

  • JS attributes (e.g. onclick) - this is the sum of all onclick, onmouseover and so on including the attribute names. So for example <a onclick="#"> is 11 characters (bytes) of JavaScript attributes code
  • CSS style attributes - the sum of all style="..."
  • Inline JS - the sum of all the contents of all script tags (excluding the tag itself)
  • Inline CSS - sum of all <style> tag contents
  • All innerHTML - this is document.documentElement.innerHTML.length, it should be close to the ungzipped size of a page, provided the page is not doing a lot of DOM manipulation
  • # DOM elements - the total number of elements on the page is counted simply using document.getElementsByTagName('*').length

Here's example output:
statsy bookmarklet output

The code

The code is here for your tweaking pleasure

Thanks!

Hope you'll find this bookmarklet useful when looking at a page as a companion to YSlow/PageSpeed.

What else should I add to this bookmarklet? # of font tags, # of table tags...?

 

Browser’s implied globals

Tuesday, October 20th, 2009

Like it's not bad enough that JavaScript has implied globals (forget var and you create a global), but the browsers have decided it's a good idea to add more pollution to the global namespace.

This has been a source of frustration before with IE, it's really hard to understand the logic behind it, but it's also happening in other browsers.

Consider this:

<meta name="description" content="test me" />

A normal META tag, right? But in IE this will create in a global variable called "description" pointing to that DOM node. Yep.

alert(description.content); // "test me"

That's pretty annoying. Even more annoying is that getElementById('description') will also point to the DOM node, although it doesn't even have an ID.

A test is born

Anyway, I wanted to test the effect of other name and id attributes in different tags and different browsers. With the exception of Firefox which doesn't create any globals, all other did to some degree. Rather disappointing. I tested IE6, 8 (plus compat view), FF 3.5, Safari 4 and Opera 10.

Here's the test page

And below are the results. The yellow x means that testing for the presence of this global returned "undefined", the white o means that the global variable points to an object. So for example continuing with the meta example above, typeof window.description will return undefined in FF (yellow x) and object in IE (white o).

global description IE FF Saf O
description <meta name="description"... o x x o
robots <meta name="robots"... o x x o
paragraph-id <p id="paragraph-id"... o x o o
paragraph-name <p name="paragraph-name"... x x x o
form-name <form name="form-name"... o x o o
form-id <form id="form-id"... o x o o
input-name <input name="input-name"... x x x x
input-id <input id="input-id"... x x o x
link-name <a name="link-name"... o x x o
link-id <a id="link-id"... o x o o
div-name <div name="div-name"... x x x o
div-id <div id="div-id"... o x o o

So...?

So this is a useless feature if you ask me. Not reliable, not cross-browser, maybe considered convenient back when rollover buttons and animated gifs were all the rage (and animated window.status, remember?), but today can only cause troubles where you least expect it. Should be removed in future browser versions.

For the time being we just have to remember to always declare and initialize our local variables because it looks like someone else might also decide to do so for us. Which can lead to errors if we assume too much.

 

cssmin.js

Wednesday, September 23rd, 2009

cssmin.js is a JavaScript port of YUICompressor's CSS minifier.

The motivation

Minifying CSS helps reduce file sizes and makes your pages faster and your users happier. YUICompressor is cool but is written in Java, which can be a blocker for some folks - you know JVM, command line, classpaths... No more excuses, now you have a simple light JavaScript version. And as you know, JavaScript is everywhere, so you can run it however you want, integrate with your editor and so on.

The links

The integration

If you want to integrate the library into your environment, it's really easy. It's just one file with one function in it. So, just a simple function call:

var result = YAHOO.compressor.cssmin(input_css_code);

The credits

Julien Lecomte - creator of YUICompressor
Isaac Schlueter - he maintains the YUICompressor and is the author of the original cssmin utility which was ported to Java by Julien.

Ha, what about a quiz? Guess the language of Isaac's original cssmin and I'll send you a free copy of Even Faster WebSites and I'll sign my chapter. Seriously.

UPDATE: For Ruby folks, there's a Ruby port from the Ryan Grove.

 

JavaScript shell scripting

Thursday, September 3rd, 2009

As you probably know, JavaScript is not limited to the browser. There's server-side JavaScript, JS for various extensions, you can script Photoshop operations with JavaScript if you feel like it. Or compile Windows executables. You see where I'm going with this. JavaScript is everywhere :)

And yes, you can do shell scripting in JavaScript. On any platform you can use Rhino to run your scripts. On Windows there's this WSH, Windows Scripting Host you can benefit from, built right into the OS so you don't have to install anything. You can run your JavaScript shell scripts with the cscript utility like:

C:\> cscript jslint.js

And on the Mac there's JavaScriptCore by WebKit. WebKit is not limited to Safari, it's used all over the place on the Mac. So there's a utility called jsc which can run your scripts.

JSC test run

JSC (JavaScriptCore) is well hidden in

/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc

Check it out, it should be there. And if it is, why not use it via a "shortcut"? So step 1:

$ sudo ln /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc /bin/jsc

Step 2... no, there's no step 2, just give it a shot:

$ jsc -h
Usage: jsc [options] [files] [-- arguments]
  -d         Dumps bytecode (debug builds only)
  -e         Evaluate argument as script code
  -f         Specifies a source file (deprecated)
  -h|--help  Prints this help message
  -i         Enables interactive mode (default if no files are specified)
  -s         Installs signal handlers that exit on a crash (Unix platforms only)

So you can just run any JavaScript code on the command line. You can also use jsc as a JavaScript console to type away stuff. Probably not that useful, because you have Firebug console to type into. But still, an option.

$ jsc
> var a = 1;
undefined
> a++
1
> a
2
>

Shell version of an online tool

Here's an example. Last night I worked on a quick hack to experiment with minifying CSS. There's an online tool over here as a result. Can I run this tool on the command line? Sure.

So imagine you have a JavaScript that parses CSS that looks like this. It defines an object called cssparse. You can use it in a browser-based tool, but also in the command-line version, without any changes. All you need is to create a new file that will be the shell version of the tool, say csspsh.js. In it put the something like:

if (!arguments[0]) {
    print('usage:\n $ jsc csspsh.js "`cat parseme.css`"');
    quit();
}

load('cssp.js');

print(cssparse.parse(arguments[0]));

You can probably guess but:

  • arguments[] array-like object contains command-line arguments
  • print() prints to the console
  • quit() exits JSC
  • load() loads and executes an external file

How do you pass arguments to your shell script? After a -- delimiter, like so:

$ jsc csspsh.js -- one two three

And since this particular script works with contents of files, I can use cat to read the file and pass it to the script.

$ jsc csspsh.js -- "`cat my.css`"

Shell-script away!

So there you have it. Shell scripting with JavaScript is at your fingertips, either Mac or Windows, or anywhere with Rhino. If you have some cool scripts that you want to run the on the command line, like cron jobs or some automated process, there has never been a better time :)

 

The art and craft of postload preloads

Thursday, August 27th, 2009

What can your tired old page, once loaded and used and read, can do for your user? It can preload components needed by the next page, so when the users visit the next page, they have the new scripts, styles and images already in the cache. Next page loads faster and the user's happy. On its death bed you tired old page has left a good inheritance for future generations. Good old page.

How can you go about preloading for the next page? By waiting for onload of the current page and requesting the new components. Here are 4 ways to do so, all using a timeout of 1 second after page load so that prefetching doesn't interfere with the user experience on the page.

One way... (DOM)

Using DOM you can create a new LINK element and a new SCRIPT element and append them to the HEAD. For images - it's a one-liner new Image.src="..."

The drawback of this method is that your CSS is executed against the current page and might affect display. Same for JavaScript - it's executed. The image is simply requested and never displayed.

window.onload = function() {
    setTimeout(function(){

        // reference to <head>
        var head = document.getElementsByTagName('head')[0];

        // a new CSS
        var css = document.createElement('link');
        css.type = "text/css";
        css.rel = "stylesheet";
        css.href = "new.css";

        // a new JS
        var js = document.createElement("script");
        js.type = "text/javascript";
        js.src = "new.js";

        // preload JS and CSS
        head.appendChild(css);
        head.appendChild(js);

        // preload image
        new Image().src = "new.png";

    }, 1000);
};

DOM test page

For this way of doing it you can use any JavaScript library's helper methods to load stuff on demand. Good examples - YUI Get and LazyLoad

... or another ... (using iframe)

Another option is to create an iframe and append your components to its head. Using an iframe you can avoid the CSS potentially affecting the current page. JavaScript will still be executed.

window.onload = function() {
    setTimeout(function(){

        // create new iframe
        var iframe = document.createElement('iframe');
        iframe.setAttribute("width", "0");
        iframe.setAttribute("height", "0");
        iframe.setAttribute("frameborder", "0");
        iframe.setAttribute("name", "preload");
        iframe.id = "preload";
        iframe.src = "about:blank";
        document.body.appendChild(iframe);

        // gymnastics to get reference to the iframe document
        iframe = document.all ? document.all.preload.contentWindow : window.frames.preload;
        var doc = iframe.document;
        doc.open(); doc.writeln("<html><body></body></html>"); doc.close(); 

        // create CSS
        var css = doc.createElement('link');
        css.type = "text/css";
        css.rel = "stylesheet";
        css.href = "new.css";

        // create JS
        var js = doc.createElement("script");
        js.type = "text/javascript";
        js.src = "new.js";

        // preload CSS and JS
        doc.body.appendChild(css);
        doc.body.appendChild(js);

        // preload IMG
        new Image().src = "new.png";

    }, 1000);
};

IFRAME test page

... I'm gonna find ya ... (static page in iframe)

If your components are static you can create a page that has them all and load that page into the dynamic iframe. Static means knowing them in advance, not relying on page's JavaScript to figure them out on the fly. As you can see, much simpler than the previous code.

window.onload = function() {
    setTimeout(function(){

        // create a new frame and point to the URL of the static
        // page that has all components to preload
        var iframe = document.createElement('iframe');
        iframe.setAttribute("width", "0");
        iframe.setAttribute("height", "0");
        iframe.setAttribute("frameborder", "0");
        iframe.src = "preloader.html";
        document.body.appendChild(iframe);

    }, 1000);
};

IFRAME static page test

... I'm gonna GETcha, GETcha, GETcha! (with Ajax)

Finally - Ajax. Scripts are not executed, CSS not used for rendering. Nice and clean. For simplicty the code has no support for browsers that don't know what XMLHttpRequest is.

window.onload = function() {
    setTimeout(function(){

        // XHR to request a JS and a CSS
        var xhr = new XMLHttpRequest();
        xhr.open('GET', 'new.js');
        xhr.send('');
        xhr = new XMLHttpRequest();
        xhr.open('GET', 'new.css');
        xhr.send('');

        // preload image
        new Image().src = "new.png";

    }, 1000);
};

Ajax test page

Thanks!

Any other ways you can think of?

 

Slides from JSConf

Tuesday, April 28th, 2009

I'm back from the most excellent JSConf (JavaScript Conference) in Washington D.C. I'm tired and need sleep but the conference was, hands down, the best conference I've ever attended. It was all about the community, it was inexpensive, with parties all around, both speakers and attendees were treated exceptionally well, in fact there wasn't a big difference since attendees presented just as good content on the B-track as did the A-track.

Here are my slides, "High Performance Kick Ass Web Apps (the JavaScript edition)". I changed the slides on the night before the presentation (and after the first night's party). The thing is that I was halfway through the slides and approaching 90 slides and I realized there's no way I will deliver so much content. One day... I'll prepare my slides way in advance and will actually practice delivering the talk...

 

Relative to absolute links with JavaScript

Tuesday, March 17th, 2009

I was toying with a completely different thing and specifically a Yahoo service that gives you the ability to use HTML as data and then lets you use xpath to query this data. I came up with a somewhat interesting idea (will post tomorrow, too late now), but all of a sudden I realized I need to convert relative links to absolute ones. I thought it's just a trivial thing (I mean how hard could it be) but then it turns out there are these little edge cases...

Anyway, I came up with something and posting here in case someone else might need it (or even me, for example two years from now. Hmm, will there be URLs in the future, in the distant 2011, or we'll all be just pure non-material consciousness ?)

So, the test page is here

And the actual code (also on github) is as follows:

function toAbs(link, host) {

  var lparts = link.split('/');
  if (/http:|https:|ftp:/.test(lparts[0])) {
    // already abs, return
    return link;
  }

  var i, hparts = host.split('/');
  if (hparts.length > 3) {
    hparts.pop(); // strip trailing thingie, either scriptname or blank 
  }

  if (lparts[0] === '') { // like "/here/dude.png"
    host = hparts[0] + '//' + hparts[2];
    hparts = host.split('/'); // re-split host parts from scheme and domain only
    delete lparts[0];
  }

  for(i = 0; i < lparts.length; i++) {
    if (lparts[i] === '..') {
      // remove the previous dir level, if exists
      if (typeof lparts[i - 1] !== 'undefined') {
        delete lparts[i - 1];
      } else if (hparts.length > 3) { // at least leave scheme and domain
        hparts.pop(); // stip one dir off the host for each /../
      }
      delete lparts[i];
    }
    if(lparts[i] === '.') {
      delete lparts[i];
    }
  }

  // remove deleted
  var newlinkparts = [];
  for (i = 0; i < lparts.length; i++) {
    if (typeof lparts[i] !== 'undefined') {
      newlinkparts[newlinkparts.length] = lparts[i];
    }
  }

  return hparts.join('/') + '/' + newlinkparts.join('/');

}

Update: added a check for ./ in the URLs thanks to QA from BoĊĦtjan

 

FireEagle and geo-location fun

Thursday, March 12th, 2009

fireeagle logo

FireEagle is a newer service from Yahoo, it's an API and service that stores your geo-location and lets other application read or update it. With your permission, of course.

Now there's a FireEagle Firefox extension, still marked experimental in Add-ons.Mozilla.org so you need a free AMO account in order to download it.

Once you install it, it will lead you through installing a prerequisite - the Geode extension from Mozilla Labs which checks the WiFi networks that are available to you and figures out where you are. The FireEagle extension then uses Geode to get the location and update its database. Of course you have full control over how precisely you want to share your location (exact, zip, neighborhood, city, state, country).

So what then? Well, then there's the FireEagle api and a bunch of applications using it to do all kinds of stuff, like update you Facebook profile and so on. Also the extension is just one way to figure out your location, there are also other ways like iPhone apps.

Geo-location via JavaScript

What I found fascinating is that once you have Geode, pages can request your location via JavaScript. This is actually a w3c standard.

A simple example of logging the position object - just type into Firebug's console:

navigator.geolocation.getCurrentPosition(console.log)

A warning appears that the page has requested you location and you can say No! or you can allow a degree of access - exact, neighborhood or city.

geo warning

Once you allow access, an async process kicks in and your callback (in this case console.log) gets notified when the location information is available. The callback receives a "position" object which has properties such as latitude, longitude, velocity, accuracy...

Pretty neat stuff.

 

Content-to-markup ratio bookmarklet

Thursday, March 5th, 2009

When you care about performance, or SEO (or just doing a good job as web dev) an interesting data point is the ratio of page content vs. the markup used to present this content. Or... how much crap we put in HTML in order to present what the users want to see - the content.

So I played tonight with a bookmarklet to provide this piece of stats.

Install

Right-click, add to favourites/bookmarks. Or simply click to see the ratio of this page.

content/markup

How it works

Since the scripts on the page may modify the content and markup, the bookmarklet makes an Ajax request to get a fresh copy of the page from the server. Then it runs a few regular expressions ("borrowed" from prototype.js) to strip all tags and the scripts/styles content. The first metric it provides is the size of the stripped content divided by the size of the original markup.

Then the bookmarklet tries to be more fair and count the alt, title and value attributes as content, including the size of attribute names themselves. And this is the second, "fair", metric. The content attributes are inspected using DOM methods, not regexp, so they can be affected by any javascript that has modified the page. Oh well, life's not fair.

Code

The bookmarklet code is served from here. The code is also on github.

Results

Here are some random results of running the bookmarklet on different sites.

http://www.cnn.com:
Total size: 92004 bytes
Content size: 11475 bytes
Content-to-markup ratio: 0.12
Fair ratio * : 0.16

http://www.sitepoint.com
Total size: 65989 bytes
Content size: 16199 bytes
Content-to-markup ratio: 0.25
Fair ratio * : 0.60

Article on http://en.wikipedia.org:
Total size: 21648 bytes
Content size: 3315 bytes
Content-to-markup ratio: 0.15
Fair ratio * : 0.35

http://www.phpied.com
Total size: 31899 bytes
Content size: 7933 bytes
Content-to-markup ratio: 0.25
Fair ratio * : 0.48

http://www.google.com SERP
Total size: 29963 bytes
Content size: 3351 bytes
Content-to-markup ratio: 0.11
Fair ratio * : 0.14

 

Replace the Home button with a script

Tuesday, March 3rd, 2009

Robert Ames commented on my previous post suggesting replacing the Home button with my little site search bookmarklet. I didn't even know this was possible, but I found it pretty cool, so I just had to try.

x-browser

I tried this in Firefox/Mac and IE/Windows and it works just fine.

ok, what is it?

Well, the idea is simple: when setting the homepage in your browser, instead of providing an http://... URL, you can use the javascript:... pseudo-protocol. So for example you can type:

javascript:alert(document.title)

And then every time you hit the Home button in the browser, an alert with the page title will pop up. How cool is that!

Naturally, you can replace the simple javascript with any sophisticated bookmarklet of your choosing (hint, hint)

how-to

In Firefox on the Mac you go Firefox / Preferences. On Windows: Tools / Options. Then in the Main tab, you type/paste the bookmarklet code.

Setting the Home to a javascript bookmarklet in Firefox

For IE/Windows go Tools / Internet Options and select the General tab

Internet Explorer - setting the Home button to a javascript bookmarklet

 

Search site bookmarklet with YUI and BOSS

Saturday, February 21st, 2009

Ever wanted to search only the web site you're currently on? Not the page, but the whole site. And only this site, not the rest of the web. This bookmarklet does just that.

Install

Right-click this link and add to your bookmarks/favorites. Or just drag to your bookmarks toolbar.

search site

Or if you don't want to add the bookmarklet, you can simply click the link above and try it out.

Use

You can search any site. Just visit the site and click the bookmarklet. There's also keyboard navigation to move up/down the results. The search results display as you type and the first results is highlighted so you can just press Enter and follow the first match of your query.

Here's a short video that shows how you can use the bookmarklet to search my blog.


search site bookmarklet @ Yahoo! Video

Under the hood

The bookmarklet uses YUI and BOSS to do its magic.

YUI (Yahoo User Interface) library's Dom, Event and Get utilities make a task like this so much easier.

BOSS (Build your Own Search Service) is a mildly confusing name for Yahoo Search's API second major iteration. The API allows you to specify a site (or sites) that you want to limit your search query.

The bookmarklet simply includes searchsite.js served from here; the bookmarklet was generated with the help of this bookmaker.

Show me code!

It's on GitHub, right here under the searchsite directory, it's just one script file searchsite.js, not at all documented, but I hope you'll make sense of it.

 

JSLint on Mac + TextMate integration

Saturday, February 21st, 2009

UPDATE: Ryan Grove has a better script to display the JSLint results. So basically follow the instructions here until you get to Step 2, point 5 (where you write the command to run JSLint). Then head over to Ryan's blog post to get the better script.

JSLint is an indispensable tool if you're serious about your JavaScript code quality. You can run it online for curiosity but for real development it has to be part of your coding environment and just a click/keystroke away.

While on PC I integrated JSLint with my text editor of choice - TextPad - and shared here. Now, ladies and gentlemen...[drum roll] I give you...[bzfghgang!] JSLint on the Mac!

Prerequisite: get Rhino running on your OSX

Don't worry, it's pretty straightforward, described here

Step 1: get JSLint

The Rhino version of JSLint is here. It's just one JS file. Find an appropriate place to copy it, I think ~/Library/JSLint is as good as any.

$ mkdir ~/Library/JSLint
$ curl http://jslint.com/rhino/jslint.js > ~/Library/JSLint/jslint.js

Test how it works from the command line:

$ java org.mozilla.javascript.tools.shell.Main ~/Library/JSLint/jslint.js myjavascript.js

Step 2: integrate with TextMate

TextMate extensions work their magic through the so called bundles. Here's what you do.

  1. Select menu: Bundles / Bundle Editor / Edit Commands...
  2. In the list of commands, expand JavaScript
  3. Click the + sign found under the list, select New Command
  4. type the name "jslint"
  5. Replace the contents of the Command(s) text field with
    java org.mozilla.javascript.tools.shell.Main ~/Library/JSLint/jslint.js "$TM_FILEPATH"
  6. In the Input: dropdown select "Entire Document", in the Output: "Show as Tool Tip" or "Show as HTML"
  7. In Activation, click on Key Equivalent and then select a key combination you like, for example Command + L (L for Lint)
  8. And this is it, refer to the screenshot below to compare with what you just did. Close the bundle editor window and you're done

textmate bundle editor

Now test your new shiny tool. Open a javascript file and press Command+L. Here's a sample output:

JSLint results in TextMate

And after fixing the missing semi-colon:

jslint fixed

 

Installing Rhino on Mac

Friday, February 20th, 2009

To quote http://www.mozilla.org/rhino/:

Rhino is an open-source implementation of JavaScript written entirely in Java. It is typically embedded into Java applications to provide scripting to end users.

Rhino allows you to use JavaScript:

  • on the server-side, so you can ditch RoR, Perl, PH... well, keep PHP :) ... in favor of JavaScript
  • on the command line, so you can shell scripts

Let's see how you can install Rhino on OSX.

Step 1 - download and unzip

Download the binary from the Rhino site and unzip to a temporary directory, say /tmp. On the command-line:

$ curl ftp://ftp.mozilla.org/pub/mozilla.org/js/rhino1_7R1.zip > /tmp/rhino.zip
$ cd /tmp
$ unzip rhino.zip

Now you have the file /tmp/rhino1_7R1/js.jar

Step 2: move js.jar where Java can find it

Your default Java install (comes "free" with OSX) will look for class libraries in a predefined directory ~/Library/Java/Extensions. This directory may not exists, so create it and move the js.jar there.

$ mkdir ~/Library/Java
$ mkdir ~/Library/Java/Extensions
$ mv /tmp/rhino1_7R1/js.jar ~/Library/Java/Extensions/

Step 3: Done! Now test it

That's all there is, your Rhino install is ready to use. To launch and test the Rhino shell try:

$ java org.mozilla.javascript.tools.shell.Main
Rhino 1.7 release 1 2008 03 06
js> print('hello!')
hello!
js> parseInt('123abc')
123
js> encodeURI('hola LA!')
hola%20LA!
js> for (var i = 0; i < 5; i++)
  > print('i is now ' + i)
i is now 0
i is now 1
i is now 2
i is now 3
i is now 4
js> quit()

Last example - create a script that reads the HTML source of my blog:

$ echo "print(readUrl('http://phpied.com'))" > read.js

now you have a script called read.js, let's run it:

$ java org.mozilla.javascript.tools.shell.Main read.js

Thanks for reading!

And happy JS-scripting!