phpied.com

… just updated.

http://developer.yahoo.com/performance/rules.html

The list (used to be 13 items, then 14, now 34) is becoming too overwhelming so it's now split into categories:

• content
• server
• cookie
• css
• javascript
• images
• mobile

Act now to improve you page loading speed, your user will thank you later

The slides from my PHP Quebec presentation in Montreal are up on slideshare. Roughly the content is divided into:

1. a quick review/update to Yahoo's existing 14 best practices for improving performance, and
2. a discussion of the 20 new ones

Enjoy responsibly and don't hesitate to send back questions and comments. And how about this slide #11: "The life of Mr.Page" 2.0

| View | Upload your own

The basic idea of a Web 2.0-style "mashup" is that you consume data from several services, often from different providers and combine them in interesting ways. This means you often need to do more than one HTTP request to a service or services. In PHP if you use something like file_get_contents() this means all the requests will be synchronous: a new one is fired only after the previous has completed. If you need to make three HTTP requests and each call takes a second, your app is delayed at least three seconds.

Solution

An improvement of course is to cache responses as much as possible, but at one point or another you still need to make those requests.

Using the curl_multi* family of cURL functions you can make those requests simultaneously. This way your app is as slow as the slowest request, as opposed to the sum of all requests. And that's something.

A function

Here's a little function I coded that will allow you do multi requests.

<?php

function multiRequest($data, $options = array()) {

  // array of curl handles
  $curly = array();
  // data to be returned
  $result = array();

  // multi handle
  $mh = curl_multi_init();

  // loop through $data and create curl handles
  // then add them to the multi-handle
  foreach ($data as $id => $d) {

    $curly[$id] = curl_init();

    $url = (is_array($d) && !empty($d['url'])) ? $d['url'] : $d;
    curl_setopt($curly[$id], CURLOPT_URL,            $url);
    curl_setopt($curly[$id], CURLOPT_HEADER,         0);
    curl_setopt($curly[$id], CURLOPT_RETURNTRANSFER, 1);

    // post?
    if (is_array($d)) {
      if (!empty($d['post'])) {
        curl_setopt($curly[$id], CURLOPT_POST,       1);
        curl_setopt($curly[$id], CURLOPT_POSTFIELDS, $d['post']);
      }
    }

    // extra options?
    if (!empty($options)) {
      curl_setopt_array($curly[$id], $options);
    }

    curl_multi_add_handle($mh, $curly[$id]);
  }

  // execute the handles
  $running = null;
  do {
    curl_multi_exec($mh, $running);
  } while($running > 0);

  // get content and remove handles
  foreach($curly as $id => $c) {
    $result[$id] = curl_multi_getcontent($c);
    curl_multi_remove_handle($mh, $c);
  }

  // all done
  curl_multi_close($mh);

  return $result;
}

?>

Consuming

The function accepts an array of URLs to hit and optionally an array of cURL options if you need to pass any. The first array can be a simple indexed array or URLs or it can be an array of arrays where the second has a key named "url". If you use the second way and you also have a key called "post", the function will do a post request.

The function returns an array of responses as strings. The keys in the result array match the keys in the input.

A GET example

Let's say you want to use some Yahoo search web services (consult YDN) to create a music artist band-o-pedia kind of mashup. Here's how you can search audio, video and images at the same time:

<?php

$data = array(
  'http://search.yahooapis.com/VideoSearchService/V1/videoSearch?appid=YahooDemo&query=Pearl+Jam&output=json',
  'http://search.yahooapis.com/ImageSearchService/V1/imageSearch?appid=YahooDemo&query=Pearl+Jam&output=json',
  'http://search.yahooapis.com/AudioSearchService/V1/artistSearch?appid=YahooDemo&artist=Pearl+Jam&output=json'
);
$r = multiRequest($data);

echo '<pre>';
print_r($r);

?>

This will print something like:

Array
(
    [0] => {"ResultSet":{"totalResultsAvailable":"633","totalResultsReturned":...
    [1] => {"ResultSet":{"totalResultsAvailable":"105342","totalResultsReturned":...
    [2] => {"ResultSet":{"totalResultsAvailable":10,"totalResultsReturned":...
)

A POST example

There's an interesting Yahoo search service called term extraction which analyses content. It accepts POST requests. Here's how to consume this service with the function above, making two simultaneous requests:

<?php
$data = array(array(),array());

$data[0]['url']  = 'http://search.yahooapis.com/ContentAnalysisService/V1/termExtraction';
$data[0]['post'] = array();
$data[0]['post']['appid']   = 'YahooDemo';
$data[0]['post']['output']  = 'php';
$data[0]['post']['context'] = 'Now I lay me down to sleep,
                               I pray the Lord my soul to keep;
                               And if I die before I wake,
                               I pray the Lord my soul to take.';

$data[1]['url']  = 'http://search.yahooapis.com/ContentAnalysisService/V1/termExtraction';
$data[1]['post'] = array();
$data[1]['post']['appid']   = 'YahooDemo';
$data[1]['post']['output']  = 'php';
$data[1]['post']['context'] = 'Now I lay me down to sleep,
                               I pray the funk will make me freak;
                               If I should die before I waked,
                               Allow me Lord to rock out naked.';

$r = multiRequest($data);

print_r($r);
?>

And the result:

Array
(
    [0] => a:1:{s:9:"ResultSet";a:1:{s:6:"Result";s:5:"sleep";}}
    [1] => a:1:{s:9:"ResultSet";a:1:{s:6:"Result";a:3:{i:0;s:5:"freak";i:1;s:5:"sleep";i:2;s:4:"funk";}}}
)

Update Jan 30, 2008: updated code based on comments and code from Carl

Here's the scenario: you have a page that links to some .mp3 files. You add a line of code in your page and lo and behold, there's a nice media player embedded into the page. Your visitors no longer have to download the MP3s, they can just stream it right there. All the mp3s on the page become part of a playlist.

The line in question you need to add to your page is:

<script src = "http://mediaplayer.yahoo.com/js"></script>

More about the player here - yahoomediaplayer.wikia.com

For examples of sites using this player in the wild, try aurgasm.us

A bookmarklet

Now, here's what you can do if you want to use the player, but the web site owner haven't incorporated it yet. Take the player with you. Run my bookmarklet that will simply insert the required javascript into the page.

So here goes in two easy steps:

1. Grab the bookmarklet:

   Right-click, add to favorites

   Y! player
2. Go to any page that links to MP3s and click your new shiny bookmarklet

   Enjoy!

Source

The readable source code of the bookmarklet:

javascript:(function(){

  var start = function(){YAHOO.music.WebPlayer.asyncLoadPlayer()};

  var script = document.createElement('script');
  script.src = 'http://mediaplayer.yahoo.com/js';

  if(script.addEventListener){
    script.addEventListener("load", start, false);
  } else{
    script.onreadystatechange = function(){
      if(this.readyState=="complete"){
        start();
        script = null;
      }
    }
  }
  script.type="text/javascript";
  document.getElementsByTagName('head')[0].appendChild(script);

})();

One of the hidden perks about working at Yahoo! is that you get to interact with a lot of smart people, and even some celebrities in the web dev profession. Rasmus Lerdorf, Douglas Crockford, YUI guys, the list is way too long… Those couple of weeks I had the pleasure of working closely with Steve Souders, chief performance architect, creator of YSlow and author of High-Performance Web Sites. We're in the same team (although he's in the Sunnyvale HQ and I'm in Santa Monica) and this time the project was the new YSlow release. I had a lot of fun, this was one of my first exposures to YSlow's code and to working on a Firefox extension in general.

I've blogged before on YSlow, but here's a word in case you're not familiar with this ultra-useful tool for speeding up your web site. YSlow inspects a page you give it for compliance with Yahoo's rules for front-end performance (my SitePoint article on the subject) and hints you how you can speed up your page, using a convenient action-oriented list. In addition to that there is a full list of the page components being inspected and also some other useful tools.

So what's new?

(more…)

click

Steve Souders, performance architect at Yahoo, announced today the public release of YSlow.

What's YSlow?

It's an extension to Firebug (yes, correct, Firebug, not Firefox) that helps with performance optimization efforts. It scores your page on the scale A to F, based on compliance with Yahoo's performance rules. It's a tool that has been used internally at Yahoo and is now released to the world.

The score

Here's how YSlow scores Yahoo's homepage, gives it an A with 93 points out of 100

You can see on the screenshot how YSlow is just another panel within Firebug, and when you select the panel it gives you a few features. The main one is Performance (shown on the screenshot).

You get a list of 13 things YSlow has evaluated (they are based on the performance rule) and if you don't get an A, there is an arrow that gives you more explanations why. Every one of the 13 items on the list is linked to online documentation of the rule so you can figure out right then what could be improved.

Other features

Besides Performance, there's also the Stats tab which gives you comparison of how your page size for visitors coming with an empty cache vs the ones that have previously visited the page.

The other tab is Components which lists every component on the page along with some information, relevant to performance, such as if the component as gzipped, what was the ETag if any, component size and expiration date.

In the tools section you'll find a nice surprise - integration with the JSLint tool, the unforgiving JavaScript verifier by Douglas Crockford.

The score (revisited)

OK, I'm sure you'll find the tool invaluable, but you may frown upon the score. Well, the scoring system is made so that it suits Yahoo's purposes, but you can modify it so that it fits your specific needs. In order to customize the points system you can go about:config in Firebug and search for yslow. There you can specify points for the score. In addition to that you can find the file called yslowcontext.js in your Firefox extensions folder (should be somewhere in Documents and Settings/Application Data/Mozilla/extensions/steve@yahoo/, path abbreviated), if you can't find it, just search for it. In this file, there is an array that defines the weight of each of the 13 rules in the overall score, so you can tweak that as well. To find the array, search for lintweights

Have fun!

And happy performance tunning!

Links

• YSlow
• YSlow on Mozilla Add-ons
• Performance rules
• Performance research articles by Tenni
• … and my own phpBB optimization effort

So there is this group of local LA web developers that meet every month or so to meet and discuss what's up. More about/join the group here.

This month Yahoo will be hosting the meetup in the Santa Monica office (my workplace), it's actually tomorrow, so if you're in LA, don't miss the opportunity for beer, pizza and meeting fellow web devs. RSVP here.

On Yahoo's side, Jim Bumgardner, a.k.a. krazydad will be demoing the Facebook app he did that allows you to find music videos and discover artists similar to the ones you like. The app, Jim talking about it, Yahoo Developers Network posting.

Sounds like it would be fun, and also a chance for a local web dev to see what Yahoo's office looks like, meet some of the people that work here, and in a way to try-before-you-apply

Previously I've mentioned how I do image searches with my bigger daughter, hunting for images of Cinderella, Ariel the little mermaid, and other equally beautiful princesses. So I thought why not build a custom little app for the little kid to do this, I mean, her, being the good (and beautiful) girl that she is, totally deserves it. Here are the very basics of such an app, my idea was to do a little demo how you can consume Y! services to do web and image searches, using nothing but JavaScript. Look ma, no server!

» demo

JS alone

The cool think about Y! web services is that they provide you different options for the output format of the results they return. You can get XML, php serialized string or JSON. (No SOAP, thank you very much!). I think more companies should consider such an offer, I mean since you get your service to work, the output format is just a … a format, nothing special.

How do you make the request? With JavaScript includes, meaning you use DOM methods to sreate a new <script> tag and add it to the head of your page. The src attribute of the new script tag points to the Y! service, a longish kind of URL with a bunch of parameters.

When making your request, you say that the output format should be JSON. Then you provide a callback function name. What Y! service does is it calls the function you gave, passing the JSON output as a parameter. Let's say you made a request to:
http://…./…&output=json&callback=myfunction
and we assume the result of your request is an array:
[1,2,3]
The Y! service will write out:
myfunction([1,2,3])
At the end, your function gets called, receiving the response from the call to the Y! service. It's now up to this function to decide what to do with the result.

Implementation

It all starts with a simple form and a placeholder for the result:

<body>
<form
    action=""
    method="get"
    onsubmit="ysearch.doSearch(
                this[0].value,
                'content',
                this[1].checked
              );
              return false;">
  <input />
  <input type="checkbox" />Image search
  <input type="submit" value="go!"/>
</form>
<div id="content">&nbsp; </div>
</body>

Submitting this form calls the doSearch() method of the ysearch class.

ysearch class has a few properties:

// url of the image search service
this.image_url = "http://api.search.yahoo.com/ImageSearchService/V1/imageSearch?";
// url of the web search service
this.web_url = "http://api.search.yahoo.com/WebSearchService/V1/webSearch?";

// ID of element that will
// display the result
this.where = '';
// is this an image search?
this.image_search = false;

// parameters for the search
this.url = "output=json"+
           "&callback=ysearch.process" +
           "&appid=YahooDemo" +
           "&start=1" +
           "&results=5" +
           "&query=";

// reference to the last included script
this.last_include = false;

Then comes the doSearch method. It does the following:

1. Checks input parameters to see if this is a request for a web search or an image search. The two types af searches use a different web service with a different URL
2. Appends the search query to the URL
3. Creates a new script tag with SRC that was just figured out
4. Saves a reference to the new script tag, so that it can be cleaned up with the next search

Here's the actual code for the doSearch() method:

this.doSearch = function(q, where, web_image) {

  this.where = where;

  var script_filename = "";
  if (web_image) {
      script_filename = this.image_url;
      this.image_search = true;
  } else {
      script_filename = this.web_url;
      this.image_search = false;
  }

  script_filename += this.url + escape(q);
  var html_doc = document.getElementsByTagName('head').item(0);
  // remove the last included script, we don't need it anymore
  if (this.last_include) {
      html_doc.removeChild(this.last_include);
  }
  var js = document.createElement('script');
  js.setAttribute('type', 'text/javascript');
  js.setAttribute('src', script_filename);
  html_doc.appendChild(js);

  // set the pointer to the last include
  this.last_include = js;

};

Becuase we passed the callback function name ysearch.process to the service, the new Y! script will call it, passing the search results as an object. Let's see what process() does, really nothing but looping through the results object (description of the object here and here) and spitting out HTML using the politically incorrect innerHTML. Here's the code:

this.process = function(resp) {

  if (resp.ResultSet && resp.ResultSet.totalResultsReturned > 0) {
    // loop through the results
    var out = '';
    for (var i = 0,
     max = resp.ResultSet.Result.length;
     i < max;
     i++) {

    var item = resp.ResultSet.Result[i];
    if (this.image_search) {
      out += '<p><a href="' + item.Url + '">'
      out += '<img src="' + item.Thumbnail.Url + '" /></a>';
      out += '<br />' + item.Summary + '</p>';
    } else {
      out += '<p><a href="' + item.Url + '">' + item.Title + '</a>';
      out += '<br />' + item.Summary + '</p>';
    }

    }
    document.getElementById(this.where).innerHTML = out;
  } else {
    alert('no results');
  }
}

Motivation

Inspired by this article on SitePoint that shows how to find the Google ranking for a specific page and a search query, I decided to do the same, but for the Yahoo! ranking. The fun part is that my script is a JavaScript and requires nothing but a browser in order to run.

How mine is different

In the article above you need to use Google's SOAP service, so if you're not lucky enough to be running PHP5, you'll probably need something along the lines of PEAR SOAP or NuSOAP. That implies you also need a web server, running PHP. Then you need a Google API key and you need to download stuff and upload it to your server.

Nothing even close to that in terms of requirements if you opt in for the Yahoo! web service. All you need is a browser and JavaScript enabled, which shouldn't be a big deal, I don't think

About the Yahoo! JSON web service

Yahoo's web service can return XML as everybody else, but it can also return serialized PHP and also JSON. Using the JSON option you can make a simple XMLHTTPRequest and get all the content JavaScript-ready, without the headaches of getELementsByTagName() or other DOMmy methods to wrestle that XML tree. The problem here is that you're requesting a file from a different domain, so the browser won't allow it. Workaround - a simple PHP script to serve as a proxy. Oooor (as we said we don't need no stinkin' server) you can use the dynamic JavaScript includes (discussed here) to do the request. As a result you get a working solution with JS only.

By the way, if you're wondering about the beauty of JSON, try this eye-opener.

Demo

Ah, yes, the demo is here.

Enter a/ your URL, or part of it, and b/ a search query. Then the script will tell you where in the first 1000 results is your URL to be found. If it is found.

How it works

Check the source for the details, it's reasonably well commented, but the big picture:

• You make a request (in yjsonrank.makeRequest()) by appending a new SCRIPT element to the HEAD of your HTML. The URL of the script element (the SRC attribute) points to the Y! web service and also passes the search query and a function to be called once the script is included. This function happens to be yjsonrank.process()
• The yjsonrank.process() function receives JSON data returned by the service, assigned to the resp variable.
• We loop through resp.ResultSet, checking every resp.ResultSet.Result if its Url property contains our URL. If yes - we're done! If not, we make another request this time for the next 50 results. (50 is randomly chosen, feel free to modify). We continue until we reach 1000th result, which is the max that Y! will be willing to give.

And that's pretty much it, the rest is just fluff and beautifications

More Y! info

• The JSON description
• The Web Search API page, listing all the additional parameters you can add to your search request and what format would have the result

Thanks for reading!