phpied.com

Background

• HTTPWatch (automation)
• ...with PHP (and again and again, and response)
• JavaScript shell scripting

I gave this short presentation at the recent Yahoo FE summit's open mic, here are the slides and some notes.

Slides and screencast vid

Screencast to see the thing in motion:

Notes

Here's the transcript of the slides as produced by slideshare.net. I've added some notes here and there to make it more readable when the presenter is missing.

JavaScript shell scripting - Presentation Transcript

1. JavaScript is everywhere #42:
         C:> WSH

2. Stoyan

   I do programming.

3. Programming

   There are many options to choose from when you decide top practice the
   art and craft of programming.

4. JavaScript

   ... is a very good one. Simply because JavaScript...

5. ... is everywhere

6. On the server

   node.js, asp.net

7. Mobile

   html5, phonegap, titanium

8. Desktop

   XULRunner lets you create cross-OS GUI apps

9. Browser extensions

   FF, Chrome, bookmarklets...

10. Photoshop

    yep, that too
    Several Adobe products actually let you script common/uncommon/programmable tasks

11. Form validation too!

    (this was supposed to be funny)

12. Shell scripting

    let's talk about shell scripting with JavaScript

13. In Windows

14. WSH: Windows Scripting Host

    All reasonable Windows machines (at least as old as Win2000) have this Windows Scripting Host in there.
    You can write VBScript or JavaScript (OK, JScript) to ... well, script.
    How does it work?

15. C:>edit hello.js

    You create a file.

16. var a = "Hello",
        b = " WSH!",
        c = a + b;
    
    WScript.Echo(c);

    Put any old JavaScript in there and print out some results

17. C:>cscript hello.js
    Hello WSH!

    And this is how you run it.

    Or this:

    C:>wscript hello.js

18. Open apps

    In addition to regular sysadmin tasks (copy, write files, move) you can open and script applications too.

19. var ie = new ActiveXObject("InternetExplorer.Application");
    ie.Visible = true;
    ie.navigate(yahoo.com);

    This is how you open IE and point it to a page.

    Notice something familiar? ActiveXObject - the thing we used in IE before it got native XMLHttpRequest

20. Firefox?

    Can you also open FF?

    Not really, as it doesn't have COM interface (whatever that is).

    But there's an easy solution

21. HTTPWatch

    Finally we come to the topic of the talk.

22. Speed

    Performance is critical for the success of any web app.

    Really, it is.

    When talking about improving speed there are two main steps:

23. 1. Fix with YSlow

    Take a slow page, run YSlow, do what it says.

    Voila - a fast(er) page.

24. 2. No regressions

    The second step is to not allow regressions.

    Whatever you fix in step 1 will be slow in the next few months.

    Even less than months the bigger the team or the rate of changes.

    So to prevent regressions, you need to constantly...

25. Monitor

26. Set limits

    The simplest way to prevent regressions is to set some limits.

    If you go over the limits, an email is sent, an alarm sounds, panic instills and you've got to fix whatever cause it was.

27. e.g.
    max 2 scripts
    max 2 styles
    max 9 images
    max 0 redirects

28. Scripting HTTPWatch

    Watching for violations of the limits manually every day is not a job anyone would want.

    So automating it will help a great way towards employee satisfaction

29. var http = new ActiveXObject("HTTPWatch.Controller"),
        ie = http.IE.New(),
        ff = http.Firefox.New();

    This is how you open IE and FF with HTTPWatch's help.

    FF - yey!

30. // browser cache
    ie.clearCache();
    
    // show HTTPWatch
    ie.OpenWindow(false);

    Examples of stuff you can do with the HTTPWatch API.

    You can for example hit the page with empty cache and then again with full cache.

    Best of all - these are the real browsers with their sometimes kinky behaviors.
    Actually if you setup several machines for the monitoring (or somehow do multiple IEs)
    you can test with different versions of the browsers. Nothing synthetic!

31. ie.Record();
    ie.GotoUrl("yahoo.com");
    http.Wait(ie, -1);
    ie.Stop();

    Start monitoring, go to a page, stop monitoring after the page "settles" meaning some time after onload.

    ie.CloseBrowser();

32. new HTTPWatch()
          http://github.com/stoyan/etc/

    I did this JavaScript thingie to make everything a little easier.

33. var http = new HTTPWatch(ff);
    http.go(search.yahoo.com);
    http.done();

    Example usage.

34. [video]

35. var har = http.toHAR();
    har = eval(( + har + ));
    
    print(har.log.browser.name);
    print(har.log.browser.version);
    print(# requests: );
    print(har.log.entries.length);

    Opening and navigating browsers is cool. But we need some data back.

    HTTPWatch can export a HAR (HTTP Archive) file. I have this toHAR() method.
    It writes the file, than reads and returns it.
    You can than eval() it because it's just a JSON string.
    And you get the data back in convenient JS objects and arrays.

36. Internet Explorer 6.0.29...
    # requests: 10
    
    Firefox 3.5.6
    # requests: 15

    Result of running the above.

37. [video]

38. var comps = http.getComponentsByType();
    
    for (var i in comps) {
      print(i);
      print(comps[i].length);
    }

    Another method I thought would be useful is getComponentsByType()

39. redirect: 1
    text/html: 3
    image/gif: 4
    image/png: 3
    text/javascript: 1

    Results of the code above.

40. But wait...

    There's more

41. What about DOM?

    So far we only talked about HTTP traffic inspection - headers and such.

    Good news is that you can also inspect the DOM (in IE only) for any potential red flags.

    For example having the number of DOM elements sharply increase.

42. [video]

43. var http = new HTTPWatch();
    http.go(search.yahoo.com);
    
    var d = http.watch.container.document;
    
    print(d.getElementsByTagName(*).length);
    print(d.documentElement.innerHTML);

    That works!

    All your DOM voodoo skillz are suddenly reusable.

44. require(statz.js);
    
    var doc = http.watch.container.document;
    var html = http.har.log.entries[0].response.content.text;
    
    var out = statz(document, html);
    print(out.join("\n"));

    This is me repurposing two old bookmarklets that gather some interesting stats (one of them was even featured on Ajaxian, remember?).

    It was pretty easy to repurpose the bookmarklets, because it's just JavaScript.

    The stats thingie can inspect both raw HTML that went over the wire, as well as innerHTML that was the result of any additional DOM manipulations.

45. JS attributes (e.g. onclick): 1207 bytes
    CSS style attributes: 883
    Inline JS: 5243
    Inline CSS: 5015
    All innerHTML: 17283
    # DOM elements: 134
    
    Total size: 14124 bytes
    Content size: 401 bytes
    Content-to-markup ratio: 0.03

    Sample results.

46. To summarize...

47. JavaScript
    WSH
    HTTPWatch
    Monitor
    DOM and HTTP
    IE and Firefox

48. Thanks
    
    phpied.com

A few months ago I gave a 5 minute Ignite talk at the Velocity conference. (My previous post talks about the Ignite experience.) I thought they recorded the video and wanted to share but seems like it's not happening. So below are the slides from slideshare.

The ignite talk rules are: 20 slides that change automatically after 15 seconds. But I took the liberty of hacking the format a little bit. For one, the 18th (and 19th) slide which was my main point is duplicated. And then I had some transitions, like in the first two slides for example, where different text or pictures change after a timeout. I expanded those transitions for slideshare, so the slide count is 40-something, but at the presentation it was still total of 20 slides in 5 minutes.

If you follow the slides on slideshare, be sure to click the Notes tab which has what I said, more or less, at each slide. The full script is below.

And yes, if you like it, please vote for my South by Southwest panel proposal. It has the same name "Image Weight Loss Clinic" but will be a much longer version of this 5 minute Iginite and what is more - Derek Tonn of graphicsoptimization.com will share the stage with me. Pretty exciting, so cast you votes, leave a comment and help me get to SxSW.

The Script

Here's what I said, more or less:

1. Let’s talk a little bit about cutting back on that bandwidth bill. Saving some money. And without touching the code. How do we do that? One way is…
2. Images. Ever since the dawn of times people have been using images. Cavemen scraping on their cave walls… and today… Today we have websites. And LOLcats.
3. Half of the average web page weight today is images. And most of these images are too fat. They could use a diet and lose some weight but without losing quality, not even a single pixel.
4. So, welcome to the Image Weight Loss Clinic. Our first patient today is... some random page found on the Internets. Hmm, what have we got here… Header graphic, speaker photos, logos, rotating banners – the usual stuff.
5. First problem - there are some GIFs. We shouldn’t be using GIFs. PNG is the format for graphics - buttons, icons, and so on. The palette type of PNG, also known as PNG8 is good as a GIF and actually better. And PNG filesizes are smaller. Well, except for really tiny GIFs, but let’s ignore these, they don’t matter much.
6. You should convert all your GIFs to PNG8. There’s a number of tools to do this in a batch. You can use ImageMagick for example. “Instant results or your money back”. Especially since ImageMagick is free.
7. For your JPEG photos, use JPEGTran. It’s free, cross-platform, command-line tool, easy to script. And it’s lossless, you keep the photo quality. If you run it without setting any options, that’s good enough. Run with the –optimize and you’ll get even better results.
8. Option –progressive makes JPEGs that render in the browser from low resolution to high, as opposed to top-to-bottom. If the JPEG is bigger than 10K (which is most JPEGs), the progressive encoding has a good shot at giving you a smaller image. You can always brute force – run with both options and take the smaller result.
9. Option –copy none strips all comments and meta data - camera model, geo location, everything. If you own the image, strip the meta data. You’ll be surprised how much bloat is in there. If you don’t own the image, check with the owner. Option –copy all keeps all meta data.
10. Next – PNG. There’s a lot of options to optimize PNGs - all lossless and scriptable on the command line. Some even have nice graphical UIs built on top of them, you know, for our command-line-challenged friends.
11. There’s even browser-based UIs, built on top of some of the tools. For example, smush.it, which is now part of YSlow runs PNGcrush. Google’s Page Speed runs OptiPNG.
12. With so many PNG tools, how do you decide which one to pick? They are all good tools and they do their best, so if you’re in a hurry, just pick one and go with it, script it. You’ll be pleasantly surprised by the outcome.
13. Now if you’re serious about PNG weight loss, you can run all the tools you can lay your hands on. Run the next tool on the result of the previous, try with their different options switches. They all do different things, and results may vary from one image to the next. Run them all in sequence and get an even smaller file at the end.
14. Or, if you’re hardcore – run PNGSlim. PNGSlim is actually a Windows batch file that runs most of the other tools. It’s very slow, it can take hours to go through a handful of files. But at the end, it will give you the leanest, most optimized PiNG.
15. Let’s see some results. That page has a total page weight of over 800K, including scripts, styles, everything. Over 80% of the weight is images. And what happens after optimization?
16. We can strip quite a bit, over 200K of unnecessary image information. 200K, that’s over 30% if the image weight. The page looks exactly the same, pixel by pixel. There was no manual work involved, just running some tools. I think 30% is pretty impressive.
17. To summarize, don’t use GIFs, run JPEGtran on your photos, and for the graphics - any PNG or all PNG tools.
18. So the tools are available out there, now all you need is a process that runs the tools before your images go live. Make image optimization part of the push process.
19. These are all lossless operations that don’t require you to look at the images to verify they’re OK. So you can only win! And often you’ll see some amazing and surprising results. Even if you normally do a good job with image sizes, there’s always a few that slip as you race against the deadline. The thing is – setup a process, so you don’t even need to think about it.
20. Thanks very much, everybody! I’m Stoyan from Yahoo! Search. Enjoy Velocity, it's a great conference!

I was asked to give feedback on what it's like to give an ignite talk (for Scott Berkun's new book) and I thought why not share in a blog post.

I gave an ignite talk at this year's Velocity conference and it was a wonderful experience. I don't think the video was ever released so I'm thinking of doing the talk at home in a screencast, so stay tuned. Meanwhile - the slides and the script.

If you're not familiar with the format - it's a 5 minute talk and you have 20 slides that change automatically after 15 seconds. So you don't have time to think or you'll miss three slides.

To give you an idea of how cool the event is, speaking to someone from the crowd about last year's experience at the same ignite event they said: "it was fun. I don't remember anything, but I remember it was fun"

The good thing about the speaker at Ignite is that the format makes you:

1. practice
2. edit

These are two things you never get around when doing a "normal" 1 hour tech talk. The "normal" way of giving tech talks in my experience is: work on the slides till 3am the night before. Give as much content as possible. And the result is:

• you're nervous (you don't know what's on the next slide),
• you talk fast (lots of content),
• you talk to the slides (because you don't remember what you put there),
• audience dozes off.

It's amazing how much difference it makes when you prepare. Someone said that not practicing means practicing failure and I find it very true.

I'm thinking this: preparing a talk takes time, even if you don't practice. Since you're spending all this time preparing anyway, it makes sense to spend a little more and be a rock star, instead of a failure. The difference is small but significant. Anything worth doing is worth doing well. Otherwise just not do it, it's best for you and the audience, and the talk host.

I usually don't practice, but I practiced for my ignite talk. Not only that but I scripted the talk, since it's so short. I even used Mac's say utility to read my script and I took note of the length of the audio file result. Also made it say the Gettysburg address which is considered a by-the-book example of short and excellent speech. So after measuring the time I started editing, cutting, cutting, cutting, because 15 seconds is about 2 sentences and I had originally planned to say much more. Then I practiced again and edited again. Ad infinitum.

And the results of all the practice were pretty good. People who have seen me speak (and even shared the stage with me) were pleasantly surprised and couldn't believe it's me up there shining on that stage

Since that Ignite talk I decided I should always practice. It's worth it. Do it right or don't do it all! But then time came to do another tech talk, and I ended up with 90+ slides for one hour, working on the slides till 3 a.m... and it was the usual disaster. We, the humans, are weird, never seem to learn from our mistakes

I'm back from the most excellent JSConf (JavaScript Conference) in Washington D.C. I'm tired and need sleep but the conference was, hands down, the best conference I've ever attended. It was all about the community, it was inexpensive, with parties all around, both speakers and attendees were treated exceptionally well, in fact there wasn't a big difference since attendees presented just as good content on the B-track as did the A-track.

Here are my slides, "High Performance Kick Ass Web Apps (the JavaScript edition)". I changed the slides on the night before the presentation (and after the first night's party). The thing is that I was halfway through the slides and approaching 90 slides and I realized there's no way I will deliver so much content. One day... I'll prepare my slides way in advance and will actually practice delivering the talk...

Thanks to everyone who attended my image optimization talk at the php|works + PyWorks conference in Atlanta. And thanks for all the questions! I love questions, feels more natural - just geeks talking to geeks - as opposed to one guy sitting on a podium and talking.

And the slides: