Archive for the 'mobile' Category

Lazy HTML evaluation

Wednesday, June 8th, 2011

#7 This post is part of the Velocity countdown series. Stay tuned for the articles to come.

Some time ago Google talked about using a sort of lazy JavaScript evaluation which especially helps mobile devices. The idea was to comment out a chunk of JavaScript you don't need right away and serve it this way. Later, when you need it, you get the content of the commented code and eval() it. More here and here.

At the last Fronteers conference I had the pleasure of chatting with Sergey Chikuyonok, who is so great and (among other things) is responsible for coming up with zen coding and writing a bunch of deep articles on image optimization for Smashing Magazine. So he told me he experimented with similar lazy HTML evaluation and it proved to be incredibly helpful for mobile devices. Not only the overall experience is faster but the initial rendering happens sooner and we all know how important that is.

Sergey is a busy person and chances of him writing about his experiment in English seemed pretty low at the time, so I decided to do an experiment on my own and see what happens. Meanwhile he did write about it so I forgot all about my findings, but here they are now.

Long document

I took one big HTML document - The adventures of Sherlock Holmes, which is half a megabyte or about 200K gzipped. Page A is the document as-is, plus some JS for measurements.

Page B (lazy) is the same page but with about 95% of its content commented out. The remaining 5% is a whole chapter so there's plenty of time to deal with the rest while the user is reading. After onload and a 0-timeout I take the commented markup (conveniently placed in <div id="lazy-daze">) and strip the comments. Then take the "unwrapped" time after another 0-timeout to let the browser repaint the DOM and regain control.

The overall skeleton of the lazy page is like so:

<!doctype html>
 
<html>
<body>
  <h1>THE ADVENTURES OF<br/>
  SHERLOCK HOLMES</h1>
  ...
  ... to chat this little matter over with you.</p>
 
  <div id="lazy-daze">
  <!--
    <p>II.</p>
    <p>
    At three o’clock precisely ... 
    ... she has met with considerable success.</p>
  -->
  </div>
 
<script>
 
 
window.onload = function () {
 
    setTimeout(function(){
 
        var daze = document.getElementById('lazy-daze'),
            inner = daze.innerHTML;
 
        daze.innerHTML = inner.substring(4, inner.length - 4);
    
        setTimeout(function(){
            // take end time... 
        }, 0);
                
    }, 0);
};
 
</script>
</body></html>

Experiment

All the files are here:
http://www.phpied.com/files/lazyhtml/

We have the plain normal document - http://www.phpied.com/files/lazyhtml/sherlock-plain.html
And the lazy one - http://www.phpied.com/files/lazyhtml/sherlock-lazy.html

In order to run the experiment you just go to
http://www.phpied.com/files/lazyhtml/start.html
And click "Go nuts". This will load each of the two documents 20 times and take a few time measurements. "Go nuts" again and you'll get 20 more data points.

The time measurements I take are:

  • "plain" - unload to onload of the base version
  • "lazy" - unload to onload of the lazy version NOT including unwrapping it. This should be quicker than the plain version
  • "unwrapped" - unload to onload plus time to unwrap and rerender - this is expected to be bigger than "plain" because the browser has to render twice and is therefore doing more work
  • DOM loaded "plain" - unload to DOMContentLoaded instead of onload
  • DOM loaded "lazy"

Then I take the same 5 measurements but instead of starting at unload of the previous page, it starts at the top of the documents, as soon as a timestamp can be taken with JavaScript. This will exclude DNS, establishing connection, time to first byte...

Results

Here are the results from back when I did the experiment originally last year, using iPhone 2 (with iOS 3.2 or thereabouts)

I ran this experiment over Wifi and again over 3G.

First striking thing - it takes the about the same time to load the plain old page over Wifi and over 3G. For the smaller, "lazy" document, there is a difference, but there's virtually none for the plain base page. The guess here is that the rendering and its cost in terms of memory and CPU is far greater than the actual download time. In other words it takes longer to render than it does to download an HTML. At least in this class of phones. This guess is confirmed when you look at the time from the top of the documents, when the request overhead is removed:

With or without the request time - it's all pretty much the same.

The next striking thing - and how about that lazy document! It renders 3-4 times faster than the whole plain document. Not bad.

And one more surprise - lazy+unwrap time is less than the plain old document. Now that's interesting. It appears faster to split the task into two and do the whole double-rendering, which should've been slower because it's extra work. I guess that poor phone really chokes on the long document.

The same I found is true in Firefox, but almost the difference is negligible.

iPhone 4

I repeated the experiment tonight on iPhone 4 and wifi. And boy, is there a difference. What used to take 13 seconds is now under 3s.

The lazy + unwrap time is more than the plain time, which was to be expected.

Rendering that initial lazy document is still 2-3 times faster that waiting for the whole document.

The numbers:

  • 2765 plain (2014 DOM)
  • 1268 lazy
  • 2995 lazy+unwrap

Ignoring the request overhead:

  • 2200 plain (1421 DOM)
  • 715 lazy
  • 2423 lazy+unwrap

And one last run/observation - on the 3G and iPhone 4 there isn't much benefit of lazy-evaluation and empty cache. The request seems much more expensive. unload to onload 4.9s where document top to onload is 2.5. When the request overhead is out of the picture than lazy eval wins again - 1.7s compared to 2.5s

Parting words

  • Lazy HTML FTW?
  • Who the heck loads an entire book in a page?! Well it may happen. It may not be a whole book, but just a lot of markup. The entire book gzipped was 219K. A hefty document, but have you seen some of those news sites?
  • Possible use case - blog comments. Lots and lots of blog comments. Or posts.
  • If you're going to lazy-load something and get it with an ajax request, why not save yourself the request and ship with another chunk of html
  • This was a simple layout task. Just a bunch of text. I'm guessing there could be much more complicated pages and layouts to render. And rendering is what takes the time it seems.
  • Drawbacks a plenty because of the hidden content - accessibility, SEO.

Thoughts? Anyone want to run the test on Android or any other phone/device/tab/pad/whathaveyou? The guess is that the newer/powerful the device the smaller the difference. But it will be nice to know.

Posted in (x)HTML(5), JavaScript, mobile, performance | 20 Comments »

 

Audio sprites

Wednesday, April 13th, 2011

Another "brilliant" idea that I had recently - how about combining audio files into a single file to reduce HTTP requests, just like we do with CSS sprites? Then use the audio APIs to play only selected parts of the audio. Unlike pretty much all brilliant ideas I have, I decided to search for this one before I dive in. Turned out Remy Sharp has already talked about this. So I knew it was possible and wanted to check the server-side or things. (Remy is amazing, by the way, and I was happy to have him as a reviewer of "JavaScript Patterns")

Here's the demo - parts of Voodoo Chile covered by yours truly.

Playing separate files

Markup is a few audio elements:

<audio id="in">
  <source src="in.mp3">
  <source src="in.ogg" type="video/ogg">
</audio>
<audio id="1">
  <source src="1.mp3">
  <source src="1.ogg" type="video/ogg">
</audio>
<audio>
  ...

I have the files:

  1. in.mp3 - intro
  2. 1.mp3 - figure 1
  3. 2.mp3 - figure 2
  4. out.mp3 - the end

1 and 2 repeat a few times.

I play these files in JavaScript using a next() iterator function, which contains (in a private closure) the melody (which file after which) and a pointer to the current file being played. After play()-ing each audio, I subscribe to the "ended" event and play the next() file.

var thing = 'the thing';
var next = (function() {  
    //log('#: file');
    //log('-------');
    var these = ['in', '1', '2', '1', '2', '1', '2', '1', 'out'],
        current = 0;
    return function() {
        thing = document.getElementById(these[current]);
        //log(current + ': ' + these[current] + ' ' + thing.currentSrc);
        thing.play();
        
        if (current < these.length - 1) {
            thing.addEventListener('ended', next, false);
            current++;
        } else {
            current = 0;
        }        
    }
}());

That was easy enough. And worked (except on iPhone, see later). But we should be able to do better with sprites.

Sprite

For the sprite I have this audio:

<audio id="sprite">
  <source src="combo.mp3">
  <source src="combo.ogg" type="video/ogg">
</audio>

There's only one file - combo.mp3 which contains all the other four files played one after the other.

So we need to know the start and the length of each piece of audio. There are two parts to playing the sprite. First is knowing the lenghts and the "song" (meaning the succession of audios) and starting to play:

var sprites = {
      // id: [start, length]
       'in': [0, 3.07],
        '1': [3.07,  2.68],
        '2': [3.07 + 2.68,  2.68],
        out: [3.07 + 2.68 + 2.68, 11.79]
    },
    song = ['in', '1', '2', '1', '2', '1', '2', '1', 'out'],
    current = 0,
    id = song[current],
    start = 0,
    end = sprites[id][1],
    interval;
 
thing = document.getElementById('sprite');
thing.play();

Next is "listening" and stopping when one audio should be stopped, then seeking through the file and playing another part of it. This is done with a setInterval(), I couldn't find a better audio event to listen to.

// change
interval = setInterval(function() {
    if (thing.currentTime > end) {
        thing.pause();
        if (current === song.length - 1) {
            clearInterval(interval);
            return;
        }
        current++;
        
        id = song[current];
        start = sprites[id][0];
        end = start + sprites[id][1]
        thing.currentTime = start;
        thing.play();
        log(id + ': start: ' + sprites[id].join(', length: '));
    }
}, 10);

And this is it. The property currentTime is read/write - you can figure out where we are and also fast-forward or rewind to where you want to go.

Results

  • Sprites play fine in FF, Chrome, O, Safari, iPhone's mobile webkit.
  • I haven't tested IE9.
  • All the browsers played y stuff off by a few milliseconds, I think some early, some late. This is probably due to unreliable setTimeout(). Also I didn't cut the audio pieces very well, so that might have someting to do. Also adding a few milliseconds of silence between the sprites may help. A follow up experiment will be to have a piano of sorts and see how timely the audio is played after a click/button press.
  • iPhone didn't play properly the non-sprited verison - I believe because it won't let you autoplay unless there's a user action. There might be a workaround, but I only cared about the sprites and they are fine!

Server side

I was imagining the whole thing as a combo service like YUI's JS/CSS combo handler. The browser says: i need these 5 files, the server then creates a new audio file and sends it back. In this case it should also somehow send the data about start/length of each audio, so maybe a JSONP thing. I was mostly curious about those file formats and how the stitching would work.

In terms of file formats, it's not that bad, turns out all I need is MP3 and OGG in order to support all these browsers (I was prepared for worse).

(I could also probably support IE3? and above with a <bgsound> and a WAV, but the WAV is too big to be practical. So any IE (before 9) enthusiasm should probably end up in Flash.)

I recorded my audio pieces in Garage Band and exported as MP3.

ffmpeg is teh tool! It's like imagemagick for audio/video.

Cutting out extra 4-5 seconds Garage Band adds to each file you export:

$ ffmpeg -i in.mp3 -ss 0 -t 2.43 in-ok.mp3

(I didn't do that very precisely I think)

Converting MP3 to OGG is like:

$ ffmpeg -i in.mp3 -acodec vorbis -aq 60 -strict experimental in.ogg

Then the stitching.

MP3 files can actually be concatenated together just like JS/CSS, provided they have the same bitrate. I've done it in the past.

You can also combine by reading the files and piping them into ffmpeg. That somehow feels better:

$ cat in.mp3 1.mp3 2.mp3 out.mp3 | ffmpeg -i - combo.mp3

You can also consider putting a bit of silence between the separate audios.

In order to get the length info to return it to the client, you can use ffmpeg -i filename.mp3 (I haven't done that part)

OGGs cannot be concatenated like MP3, so the combo service should `cat` the mp3s as shown above, then convert to OGG (also shown :) )

Voila

You can now roll your own on-demand audio combo handler and use audio sprites to have fewer HTTP requests a more responsive app/game/html5 thing.

Posted in mobile, Music, performance | 13 Comments »