HTTPWatch automation with JavaScript / Stoyan's phpied.com

Background

HTTPWatch (automation)
...with PHP (and again and again, and response)
JavaScript shell scripting

I gave this short presentation at the recent Yahoo FE summit's open mic, here are the slides and some notes.

Slides and screencast vid

JavaScript shell scripting

View more presentations from Stoyan Stefanov

Screencast to see the thing in motion:

Notes

Here's the transcript of the slides as produced by slideshare.net. I've added some notes here and there to make it more readable when the presenter is missing.

JavaScript shell scripting - Presentation Transcript

JavaScript is everywhere #42:
      C:> WSH

```
Stoyan
```
I do programming.
```
Programming
```
There are many options to choose from when you decide top practice the
art and craft of programming.
```
JavaScript
```
... is a very good one. Simply because JavaScript...
```
... is everywhere
```
```
On the server
```
node.js, asp.net
```
Mobile
```
html5, phonegap, titanium
```
Desktop
```
XULRunner lets you create cross-OS GUI apps
```
Browser extensions
```
FF, Chrome, bookmarklets...
```
Photoshop
```
yep, that too
Several Adobe products actually let you script common/uncommon/programmable tasks
```
Form validation too!
```
(this was supposed to be funny)
```
Shell scripting
```
let's talk about shell scripting with JavaScript
```
In Windows
```
```
WSH: Windows Scripting Host
```
All reasonable Windows machines (at least as old as Win2000) have this Windows Scripting Host in there.
You can write VBScript or JavaScript (OK, JScript) to ... well, script.
How does it work?
```
C:>edit hello.js
```
You create a file.

var a = "Hello",
    b = " WSH!",
    c = a + b;
        
WScript.Echo(c);

Put any old JavaScript in there and print out some results

C:>cscript hello.js
Hello WSH!

And this is how you run it.

Or this:

C:>wscript hello.js

```
Open apps
```
In addition to regular sysadmin tasks (copy, write files, move) you can open and script applications too.
```
var ie = new ActiveXObject("InternetExplorer.Application");
ie.Visible = true;
ie.navigate(yahoo.com);
```
This is how you open IE and point it to a page.

Notice something familiar? ActiveXObject - the thing we used in IE before it got native XMLHttpRequest
```
Firefox?
```
Can you also open FF?

Not really, as it doesn't have COM interface (whatever that is).

But there's an easy solution
```
HTTPWatch
```
Finally we come to the topic of the talk.
```
Speed
```
Performance is critical for the success of any web app.

Really, it is.

When talking about improving speed there are two main steps:
```
1. Fix with YSlow
```
Take a slow page, run YSlow, do what it says.

Voila - a fast(er) page.
```
2. No regressions
```
The second step is to not allow regressions.

Whatever you fix in step 1 will be slow in the next few months.

Even less than months the bigger the team or the rate of changes.

So to prevent regressions, you need to constantly...
```
Monitor
```
```
Set limits
```
The simplest way to prevent regressions is to set some limits.

If you go over the limits, an email is sent, an alarm sounds, panic instills and you've got to fix whatever cause it was.

e.g.
max 2 scripts
max 2 styles
max 9 images
max 0 redirects

```
Scripting HTTPWatch
```
Watching for violations of the limits manually every day is not a job anyone would want.

So automating it will help a great way towards employee satisfaction 🙂

var http = new ActiveXObject("HTTPWatch.Controller"),
    ie = http.IE.New(),
    ff = http.Firefox.New();

This is how you open IE and FF with HTTPWatch's help.

FF - yey!

```
// browser cache
ie.clearCache();

// show HTTPWatch
ie.OpenWindow(false);
```
Examples of stuff you can do with the HTTPWatch API.

You can for example hit the page with empty cache and then again with full cache.

Best of all - these are the real browsers with their sometimes kinky behaviors.
Actually if you setup several machines for the monitoring (or somehow do multiple IEs)
you can test with different versions of the browsers. Nothing synthetic!
```
ie.Record();
ie.GotoUrl("yahoo.com");
http.Wait(ie, -1);
ie.Stop();
```
Start monitoring, go to a page, stop monitoring after the page "settles" meaning some time after onload.
```
ie.CloseBrowser();
```
```
new HTTPWatch()
      http://github.com/stoyan/etc/
```
I did this JavaScript thingie to make everything a little easier.

var http = new HTTPWatch(ff);
http.go(search.yahoo.com);
http.done();

Example usage.

```
var har = http.toHAR();
har = eval(( + har + ));

print(har.log.browser.name);
print(har.log.browser.version);
print(# requests: );
print(har.log.entries.length);
```
Opening and navigating browsers is cool. But we need some data back.

HTTPWatch can export a HAR (HTTP Archive) file. I have this toHAR() method.
It writes the file, than reads and returns it.
You can than eval() it because it's just a JSON string.
And you get the data back in convenient JS objects and arrays.

Internet Explorer 6.0.29...
# requests: 10

Firefox 3.5.6
# requests: 15

Result of running the above.

var comps = http.getComponentsByType();
      
for (var i in comps) {
  print(i);
  print(comps[i].length);
}

Another method I thought would be useful is getComponentsByType()

redirect: 1
text/html: 3
image/gif: 4
image/png: 3
text/javascript: 1

Results of the code above.

```
But wait...
```
There's more 🙂
```
What about DOM?
```
So far we only talked about HTTP traffic inspection - headers and such.

Good news is that you can also inspect the DOM (in IE only) for any potential red flags.

For example having the number of DOM elements sharply increase.

var http = new HTTPWatch();
http.go(search.yahoo.com);

var d = http.watch.container.document;

print(d.getElementsByTagName(*).length);
print(d.documentElement.innerHTML);

That works!

All your DOM voodoo skillz are suddenly reusable.

```
require(statz.js);
      
var doc = http.watch.container.document;
var html = http.har.log.entries[0].response.content.text;

var out = statz(document, html);
print(out.join("\n"));
```
This is me repurposing two old bookmarklets that gather some interesting stats (one of them was even featured on Ajaxian, remember?).

It was pretty easy to repurpose the bookmarklets, because it's just JavaScript.

The stats thingie can inspect both raw HTML that went over the wire, as well as innerHTML that was the result of any additional DOM manipulations.

JS attributes (e.g. onclick): 1207 bytes
CSS style attributes: 883
Inline JS: 5243
Inline CSS: 5015
All innerHTML: 17283
# DOM elements: 134

Total size: 14124 bytes
Content size: 401 bytes
Content-to-markup ratio: 0.03

Sample results.

```
To summarize...
```

JavaScript
WSH
HTTPWatch
Monitor
DOM and HTTP
IE and Firefox

```
Thanks
      
phpied.com
```

Tell your friends about this post on Facebook and Twitter

Sorry, comments disabled and hidden due to excessive spam.

Meanwhile, hit me up on twitter @stoyanstefanov