Archive for the 'PEAR' Category

IE9 and JPEG-XR: first impressions

Monday, April 5th, 2010

One of the new features in IE9 is the support for the JPEG-XR format, which reportedly has a better compression. Is it something we should dive into ASAP?

JPEG-XR

The wikipedia article is here. This format is developed and patented (red flag!) by Microsoft (yellow flag! :) ), it replaces the suggested JPEG-2000 format and is an official standard as of mid '09. It's formerly known as "HD photo" and "Microsoft something something" and is heavily used in Vista and Windows 7 - two OSes I'm yet to experience. Anyway.

The wikipedia article says that the format has a better compression, so I had to take a look at it!

Software support

That's where the problems start. The list of software that supports the format is not too long and most of the software just reads it, like IE9. IE9 doesn't run on XP, so I couldn't actually test it.

What I managed to try was:

  • a plugin for Paint.NET that read/writes JPEG-XR
  • a 60-day trial version of MS' Expression Design 3 that reads/writes - this required an upgrade of my .Net platform, so the whole process took forever on the poor XP VMWare (but hey, I have a book to finish, so no distraction is too long!)
  • a plugin for IrfanView that reads the format

Nothing for Mac though.

There's rumors all over the place that MS have a beta version of a Photoshop plugin that works on the Mac, but even the MS' press release linked to a 404. (which redirected to a bing search - here's the secret to gaining search market share!)

Actually before I digged into these programs, my first instinct was to check if ImageMagick supports this format. Turns out no. IMagick used libjpeg like pretty much all image programs out there and curiously enough here's what libjpeg's README has to say:

FILE FORMAT WARS
================

The ISO JPEG standards committee actually promotes different formats like
JPEG-2000 or JPEG-XR which are incompatible with original DCT-based JPEG
and which are based on faulty technologies.  IJG therefore does not and
will not support such momentary mistakes (see REFERENCES).
We have little or no sympathy for the promotion of these formats.  Indeed,
one of the original reasons for developing this free software was to help
force convergence on common, interoperable format standards for JPEG files.
Don't use an incompatible file format!
(In any case, our decoder will remain capable of reading existing JPEG
image files indefinitely.)

Sounds like another red flag to me. IDG (Independent JPEG Group) are the creators of libjpeg. If libjpeg doesn't sounds like it will support it JPEG-XR, that means adoption can be really slow if not feasible at all. But even if IE is the only browser under the sun that supports the format and the format is so much better, then there might be cases where it could be beneficial to browser-sniff and send different image versions, as far-fetched as that may sound.

Test in Paint.NET

I started with Paint.NET because it was the easiest. I took a photo I've taken with the iPhone, keeping the use case real, and resized to a 600x450px which sounds like a normal thing to do in a blog post for example. I used IrfanView and PNG, so that the original is lossless (click the thumb for the actual source image).

I converted the photo with Paint.NET to JPEG and to JPEG-XR (also called WDP/HD Photo). In both cases I used quality of 80%. There was also an option for WDP which was 32bit image by default, which I changed to 24 bits because the image was smaller filesize.

WDP export in Paint.NET

The results were - 45K for XR/WDP and 24K for JPEG. So the good old JPEG was smaller - the exact opposite of what should've happened. Additionally JPEGTran shaved off another 1.3K from the file. Seemed like JPEG-XR is not that good after all. But as I said I had a book to write so I kept going with the distractions, determined to avoid writing for as long as I can.

Test with Expression Design

Expression Design produced the exact same WDP/HD Photo/JPEG-XR file - 45K. And this is not surprising actually, since there is an image framework from MS, called WIC, part of .Net, which is probably what Paint.NET and Expression Design both use. But surprisingly enough the JPEG outcome from Expression Design was significantly bigger - 57K. What?!

Then I looked at the visual quality and the number of colors and it turned out the JPEGs were pretty different, although they were converted from the same PNG and using 80% in both programs.

Software/Format JPEG JPEG-XR aka WDP aka HD Photo
Paint.NET 24K (50 000+ colors) 45K (104 000+ colors)
Expression Design 57K (54 000+ colors)

Visually the JPEG from Paint.NET is clearly lower quality than the one from Expression Design and from the WDP format. Interestingly, IrfanView produced an pretty much identical file when converting the PNG to JPEG with quality 80. So Expression Design seems to be doing something differently.

Using IrfanView I increased the quality of the JPEG until the file size reaches the file size of the WDP. (After all, all I want to know is which format has the smaller filesize). The quality of 93 resulted in a JPEG that was about the same file size as quality 80 JPEG-XR. Then I tried so look at the visual quality and although I'm not a designer, it seemed to me that the two images are pretty identical and XR is maybe just a little better. But that's a little subjective.

Here's the two files for comparison. Let me know which one you think is better. In this case they are both losslessly converted to PNG, so all browsers can see the WDP.

Here's also an image diff (from ImageMagick's compare) - it shows that technically the two images are very different (the white dots are pixels with the exact same color values)

One other thing about Expression Design - when exporting WDR, it has a "transparency" checkbox ON. This results is bigger images, so make sure you turn it off when using, it makes no sense for photos.
Expression Design options

Batch conversion?

My motivation in this experiment was to see if there's a way (and a reason) to do a batch conversion of all JPEG imagery to JPEG-XR. This would be my favorite performance optimization - you run one script and wake up to a 5-10% less image bandwidth.

Looks like JPEG-XR could probably look better for the same filesize, meaning maybe a smaller filesize for the same quality. But it's not easy to decide when it comes to quality and certainly even harder for a machine (a simple batch conversion script) to tell. I was hoping that there's a way to losslessly convert to JPEG-XR. From what I can see, there isn't. JPEG-XR does have a lossless option but it creates huge files (like 250K instead of 45), so the lossless versions are not meant to be on the web. BTW, the lossless option is the same as 100 quality (which is not the same in normal JPEG, where even 100% is lossy).

So, in conclusion - JPEG-XR may look promising but is currently unusable for practical purposes, because of

  • the extremely limited support in browsers (browser, actually),
  • very limited choices of creation software
  • the benefits are hard to distinguish
  • not possible to batch-and-forget process all old JPEGs

And there's the other turn off - patents. Although Microsoft has promised to promise not to sue people around for implementing JPEG-XR, a patent is a patent and all software patents must die on general principle :)

 

Reducing the payload: compression, minification, 204s

Friday, December 11th, 2009

2010 update:
Lo, the Web Performance Advent Calendar hath moved

Dec 11 This post is part of the 2009 performance advent calendar experiment. Stay tuned for the next articles.

After removing all the extra HTTP requests you possibly can from your waterfall, it's time to make sure that those that are left are as small as they can be. Not only this makes your pages load faster, but it also helps you save on the bandwidth bill. Your weapons for fighting overweight component include: compression and minification of text-based files such as scripts and styles, recompression of some downloadable files, and zero-body components. (A follow-up post will talk about optimizing images.)

Gzipping plain text components

Hands down the easiest and at the same time quite effective optimization - turning on gzipping for all plain text components. It's almost a crime if you don't do it. Doesn't "cost" any development time, just a simple flip of a switch in Apache configuration. And the results could be surprisingly pleasant.

When Bill Scott joined Netflix, he noticed that gzip is not on. So they turned it on. And here's the result - the day they enabled it, the outbound traffic pretty much dropped in half (slides)

Netflix traffic after turning on gzipping

Gzip FAQ

How much improvement can you expect from gzip?
On average - 70% reduction of the file size!
Any drawbacks?
Well, there's a certain cost associated with the server compressing the response and the browser uncompressing it, but it's negligible compared to the benefits you get
Any browser quirks?
Sure, IE6, of course. But only in IE6 service pack 1 and fixed for after that. You can boldly ignore this edge case, but if you're extra paranoid you can disable gzip for this user agent
How to tell if it's on?
Run YSlow/PageSpeed and they'll will warn you if it's not on. If you don't have any of those tools just look at the HTTP headers with any other tool, e.g. Firebug, webpagetest.org. You should see the header:

Content-Encoding: gzip

provided, of course, that your browser claimed it supports compression by sending the header:

Accept-Encoding: gzip, deflate
What types of components should you gzip?
All text components:

  • javascripts
  • css
  • plain text
  • html, xml, including any other XML-based format such as SVG, also IE's .htc
  • JSON responses from web service calls
  • anything that's not a binary file...

You should also gzip @font-files like EOT, TTF, OTF, with the exception of WOFF. Average about 40% to be won there with font files.

How-to turn on gzipping

Ideally you need control over the Apache configuration. If not full control, at least most hosting providers will offer you ability to tweak configuration via .htaccess. If your host doesn't, well, change the host.

So just add this to .htaccess:

AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/javascript application/json

If you're on Apache before version 2 or your unfriendly host don't allow any access to configuration, not all is lost. You can make PHP do the gzipping for you. It's not ideal but the gzip benefits are so pronounced that it's worth the try. This article describes a number of different options for gzipping when dealing with uncooperative hosts.

Rezipping

As Billy Hoffman discovered, there's potential for file size reduction with common downloadable files, which are actually zip files in disguise. Such files include:

  • Newer MS Office documents - DOCX, XLSX, PPTX
  • Open Office documents - ODT, ODP, ODS
  • JARs (Java Applets, anyone?)
  • XPI Firefox extensions
  • XAP - Silverlight applications

These ZIP files in disguise are usually not compressed with the maximum compression. If you allow such downloads from your website, consider recompressing them beforehand with maximum compression.

There could be anywhere from 1 to 30% size reduction to be won, definitely worth the try, especially since you can do it all on the command line, as part of the build process, etc. (re)Compress once, save bandwidth and offer faster downloads every time ;)

15% uncompressed traffic

Tony Gentilcore of Google reported his findings that a significant chunk of their traffic is still sent uncompressed. Digging into it he realized there's a number of anti-virus software and firewalls that will mingle with the browser's Accept-Encoding header changing into the likes of:

Accept-Encoding: xxxx, deflxxx
Accept-Enxoding: gzip, deflate

Since this is an invalid header, the server will decide that the browser doesn't support gzip and send uncompressed response. And why would the retarded anti-virus program do it? Because it doesn't want to deal with decompression in order to examine the content. Probably not to slow down the experience? In doing so it actually hurts the user to a greater extend.

So compression is important, but unfortunately it's not always present. That's why minification helps - not only because compressing minified responses is even smaller, but because sometimes there is no compression despite your best efforts.

Minification

Minification means striping extra code from your programs that is not essential for execution. The code in question is comments, whitespace, etc from styles and scripts, but also renaming variables with shorter names, and various other optimizations.

This is best done with a tool, of course, and luckily there a number of tools to help.

Minifying JavaScript

Some of the tools to minify JavaScript include:

How much size reduction can you expect from minification? To answer that I ran jQuery 1.3.2. through all the tools mentioned above (using hosted versions) and compared the sizes before/after and with/without gzipping the result of minification.

The table below lists the results. All the % figures are % of the original, so smaller is better. 29% means the file was reduced to 29% of its original version, or a saving of 71%

File original size size, gzipped % of original gzip, % of original
original 120619 35088 100.00% 29.09%
closure-advanced 49638 17583 41.15% 14.58%
closure 55320 18657 45.86% 15.47%
jsmin 73690 21198 61.09% 17.57%
packer 39246 18659 32.54% 15.47%
shrinksafe 69516 22105 57.63% 18.33%
yui 57256 19677 47.47% 16.31%

As you can see gzipping alone gives you about 70% savings, minification alone cuts script sizes with more than half and both combined (minifying then gzipping) can make your scripts 85% leaner. Verdict: do it. The concrete tool you use probably doesn't really matter all that much, pick anything you're comfortable with to run before deployment (or best, automatically during a build process)

Minifying CSS

In addition to the usual stripping of comments and whitespaces, more advanced CSS minification could include for example:

// before
#mhm {padding: 0px 0px 0px 0px;}
// after
#mhm{padding:0}

// before
#ha{background: #ff00ff;}
// after 
#ha{background:#f0f}
//...

A CSS minifier is much less powerful than a JS minifier, it cannot rename properties or reorganize them, because the order matters and for example text-decoration:underline cannot get any shorter than that.

There's not a lot of CSS minifiers, but here's a few I tested:

  • YUI compressor - yes, the same YUI compressor that does JavaScript minification. I've actually ported the CSS minification part of it to JavaScript (it's in Java otherwise) some time ago. There's even an online form you can paste into to test. The CSS minifier is regular expression based
  • Minify is a PHP based JS/CSS minification utility started by Ryan Grove. The CSS minifier part is also with regular expressions, I have the feeling it's also based on YUICompressor, at least initially
  • CSSTidy - a parser and an optimizer written in PHP, but also with C version for desktop executable. There's also a hosted version. It's probably the most advanced optimizer in the list, being a parser it has a deeper understanding of the structure of the styleshets
  • HTML_CSS from PEAR - not exactly an optimizer but more of a general purpose library for creating and updating stylesheets server-side in PHP. It can be used as a minifier, by simply reading, then printing the parsed structure, which strips spaces and comments as a side effect.

Trying to get an average figure of the potential benefits, I ran these tools on all stylesheets from csszengarden.com, collected simply like:

<?php
$urlt = "http://csszengarden.com/%s/%s.css";
for ($i = 1; $i < 214; $i++) {
  $id = str_pad($i, 3, "0", STR_PAD_LEFT);
  $url = sprintf($urlt, $id, $id);
  file_put_contents("$id.css", file_get_contents($url));
}
?>

3 files gave a 404, so I ran the tools above on the rest 210 files. CSSTidy ran twice - once with its safest settings (which even keep comments in) and then with the most aggressive. The "safe" way to use CSSTidy is like so:

<?php
// dependencies, instance
include 'class.csstidy.php';
$css = new csstidy();
 
// options
$css->set_cfg('preserve_css',true);
$css->load_template('high_compression');
 
// parse
$css->parse($source_css_code);
 
// result
$min = $css->print->plain();
?>

The aggressive minification is the same only without setting the preserve_css option.

Running Minify is simple:

<?php
// dependencies, instance
require 'CSS.php';
$minifier = new Minify_CSS();
 
// minify in one shot
$min = $minifier->minify($source_css_string_or_url);

As for PEAR::HTML_CSS, since it's not a minifier, you only need to parse the input and print the output.

<?php
require 'HTML/CSS.php';
 
$options = array(
    'xhtml' => false,
    'tab' => 0,
    'oneline' => true,
    'groupsfirst' => false,
    'allowduplicates' => true,
);
 
$css = new HTML_CSS($options);
$css->parseFile($input_filename);
$css->toFile($output_filename);
// ... or alternatively if you want the result as a string
// $minified = $css->toString();

So I ran those tools on the CSSZenGarden 200+ files and the full table of results is here, below are just the averages:

  Original YUI Minify CSSTidy-safe CSSTidy-small PEAR
raw 100% 68.18% 68.66% 84.44% 63.29% 74.60%
gzipped 30.36% 19.89% 20.74% 28.36% 19.44% 20.20%

Again, the numbers are percentage of the original, so smaller is better. As you can see, on average gzip alone gives you 70% size reduction. The minification is not so successful as with JavaScript. Here even the best tool cannot reach 40% reduction (for JS it was usually over 50%). But nevertheless, gzip+minification on average gives you a reduction of 80% or more. Verdict: do it!

An important note here is that in CSS we deal with a lot of hacks. Since the browsers have parsing issues (which is what hacks often exploit), what about a poor minifier? How safe are the minifiers? Well, that's a subject for a separate study, but I know I can at least trust the YUICompressor, after all it's used by hundreds of Yahoo! developers daily and probably thousands non-Yahoos around the world. PEAR's HTML_CSS library also looks pretty safe because it has a simple parser that seems to tolerate all kinds of hacks. CSSTidy also claims to tolerate a lot of hacks, but given that the last version is two years old (maybe new hacks have surfaced meanwhile) and the fact that it's the most intelligent optimizer (knows about values, colors and so on) it should be approached with care.

204

Let's wrap up this lengthy posting with an honorable mention of the 204 No Content response (blogged before). It's the world's smallest componet, the one that has no body and a Content-Length of 0.

Often people use 1x1 GIFs for logging and tracking purposes and other types of requests that don't need a response. If you do this, you can return a 204 status code and no response body, only headers. Look no further that Google search results with your HTTP sniffer ON to see examples of 204 responses.

The way to send a 204 response from PHP is simply:

header("HTTP/1.0 204 No Content");

A 204 response saves just a little bit but, hey, every little bit helps.

And remember the mantra: every extra bit is a disservice to the user :)

Thank you for reading!

Stay tuned for the next article continuing the topic of reducing the component sizes as much as possible.

 

PEAR site minor redesign

Sunday, January 6th, 2008

pear-icon.png Just noticed there have been some changes on the PEAR site. Check it out - no nav menu on the left, spacier for the main content...

http://pear.php.net/

 

Text_Highlighter 0.7.1 and hiliteme.com updates

Tuesday, January 1st, 2008

In today's busy schedules there's less and less time to give love to our favorite open source projects. Luckily though, there's January 1st when you're supposed to relax (or suffer the consequences of partying on Dec 31st). Anyway, today I found the time to fix two bugs in Text_Highlighter and also include the patch from Daniel Fruzynski that adds support for highlighting VBScript. Wo-hoo!

Also updated hiliteme.com so you can try the package or simply highlight some code before posting it to your blog.

Luckily for me, the PEAR site uses GMT timestamps, so the release date of Text_Highlighter 0.7.1. is Jan 2nd. This way I hope it doesn't look like I need a life that badly.

 

Image_Text 0.6 beta is out

Thursday, April 19th, 2007

» Download here

This is my first PEAR release and I was actually surprised how easy it is to package and roll out a release.

So you have your local copy of the CVS repository that contains the scripts you want to release as part of the package. In order to release, you need package.xml, a configuration file, which you can either create yourself or have a script (which uses PEAR_PackageFileManager) to create the xml file for you.

The pear command line tool does all the rest.

  1. pear convert - creates package2.xml based on your package.xml. (package2 is the newer improved version of package2.xml. You can actually use PEAR_PackageFileManager2 instead and skip this step)
  2. pear package - creates the package archive which then you upload to pear.php.net
  3. pear cvstag package2.xml - tags the cvs repository with a tag figured out from the package2. In my case the tag was RELEASE_0_6_0beta

Thanks!

As stated in the change log notes, many thanks go to Christian Weiske and James Pic for helping out with this release!

 

Performance tunning with PEAR::DB

Tuesday, January 16th, 2007

If you use PEAR::MDB2, you can set a custom debug handler and collect all the queries you execute for debugging and performance tunning purposes, as shown before. But what if you're using PEAR::DB? Well, since PEAR::DB doesn't allow you such a functionality out of the box, you can hack it a bit to get similar results.

Simple app

Let's say you have a simple app:

<?php 
require_once 'DB.php';
 
$dsn = 'mysql://root@localhost/test';
$db =& DB::connect($dsn);
$db->setFetchMode(DB_FETCHMODE_ASSOC);
 
$sql = 'SELECT * FROM zipcodes';
$result = $db->getAll($sql);
$result = $db->getOne($sql);
$result = $db->getCol($sql);
$result = $db->getAll($sql);
$sql = 'SELECT zipcode FROM zipcodes';
$result = $db->getAll($sql);
$result = $db->getAll($sql);
$sql = 'SELECT CONCAT(zipcode, " - ", city) FROM zipcodes';
$result = $db->getAll($sql);
?>

Of course, this is an oversimplified example, usually you have more included files, class libraries and such, and it's not difficult to lose track of the database work as your app grows in complexity and size.

Now let's debug this app to see what type of database work it does.

Hacking PEAR::DB

In my case, I'm using MySQL, so I need to find the DB/mysql.php file in my PEAR directory. I open that file and find the simpleQuery() method. That's where all queries and up, sooner or later. I find this piece of code:

<?php
if (!$this->options['result_buffering']) {
    $result = @mysql_unbuffered_query($query, $this->connection);
} else {
    $result = @mysql_query($query, $this->connection);
}
?>

Then I hack this piece of code, adidng some lines before and after it. The result:

<?php
// start
$start_time = array_sum(explode(' ',microtime()));
// end
 
if (!$this->options['result_buffering']) {
    $result = @mysql_unbuffered_query($query, $this->connection);
} else {
    $result = @mysql_query($query, $this->connection);
}
 
// start
$query_took = array_sum(explode(' ',microtime())) - $start_time;
@$GLOBALS['global_query_counter']++;
@$GLOBALS['all_the_queries'][$GLOBALS['global_query_counter'] . ' - ' . $query] = $query_took;
//end
?>

Now as my app's pages are executed, I'll collect invaluable DB information.

Reporting

Let's see what we've collected.

You can add different types of reports in the footer of your application, or better yet, you can register a shutdown function to do the same. Here are some reporting ideas:

<?php
// report 1.
echo "<pre>All the queries, by the order they are executed:\\n";
print_r($GLOBALS['all_the_queries']);
echo '</pre>';
 
// report 2.
echo "<pre>All the queries, ordered by the time they took, descending:\\n";
arsort($GLOBALS['all_the_queries']);
print_r($GLOBALS['all_the_queries']);
echo '</pre>';
 
// report 3.
$sum = 0;
foreach ($GLOBALS['all_the_queries'] AS $t) {
    $sum += $t;
}
echo '<pre>';
echo 'Total number of queries:   ' . $GLOBALS['global_query_counter'] . "\\n";
echo 'Total time spend querying: ' . $sum;
echo '</pre>';
 
 
// report 4.
$distinct = array();
foreach ($GLOBALS['all_the_queries'] AS $q=>$t) {
    $parts = explode(' - ', $q);
    unset($parts[0]);
    $query = implode(' - ', $parts);
    @$distinct[$query]++;
}
echo "<pre>How many duplications:\\n";
arsort($distinct);
print_r($distinct);
echo '</pre>';
?>

Report results

Let's see what these reports gives us.

All the queries, by the order they are executed:
Array
(
    [1 - SELECT * FROM zipcodes] => 0.00626707077026
    [2 - SELECT * FROM zipcodes] => 0.00730204582214
    [3 - SELECT * FROM zipcodes] => 0.00796985626221
    [4 - SELECT * FROM zipcodes] => 0.00654602050781
    [5 - SELECT zipcode FROM zipcodes] => 0.0058650970459
    [6 - SELECT zipcode FROM zipcodes] => 0.0239379405975
    [7 - SELECT CONCAT(zipcode, " - ", city) FROM zipcodes] => 0.00581502914429
)

All the queries, ordered by the time they took, descending:
Array
(
    [6 - SELECT zipcode FROM zipcodes] => 0.0239379405975
    [3 - SELECT * FROM zipcodes] => 0.00796985626221
    [2 - SELECT * FROM zipcodes] => 0.00730204582214
    [4 - SELECT * FROM zipcodes] => 0.00654602050781
    [1 - SELECT * FROM zipcodes] => 0.00626707077026
    [5 - SELECT zipcode FROM zipcodes] => 0.0058650970459
    [7 - SELECT CONCAT(zipcode, " - ", city) FROM zipcodes] => 0.00581502914429
)

Total number of queries:   7
Total time spend querying: 0.0637030601501

How many duplications:
Array
(
    [SELECT * FROM zipcodes] => 4
    [SELECT zipcode FROM zipcodes] => 2
    [SELECT CONCAT(zipcode, " - ", city) FROM zipcodes] => 1
)

Thanks for reading!

Any comments or suggestions are very welcome!

 

DB-2-MDB2 in Portuguese

Tuesday, January 16th, 2007

Through a trackback I found out that Walter Cruz has translated my DB-2-MDB2 article in a language I was led to believe is Brazilian Portuguese.

Thanks very much Walter, this is very flattering!

Thanks to my buddy Isidoro who enlightened me that the language was Portugeese!

 

Using PEAR and AWS to keep an eye on Amazon

Wednesday, January 10th, 2007

What could possibly be better for a writer's ego other than being read and being praised? Hmm...

So I wanted to have a page that shows the books I've written, together with their Amazon sales rank and the average customer rating and number of reviews. It's really easy. I took one example out of the PEAR book and slightly modified it.

The result is here.

Implementation at a glance

When making a request, you need to say what type of request it is and what type of response you want. It's documented here. For the type of request, look under "API reference -> Operations" and for the response type, look under "API reference -> Response Groups"

The code

<?php
 
// include the PEAR package
require_once 'Services/AmazonECS4.php';
 
// Your AWS subscription id
$subscriptionId = '1WQDAES5PQ**********';
 
// create a new client by supplying
// subscription id
$amazon  = new Services_AmazonECS4($subscriptionId);
$amazon->setLocale('US');
 
// output options
// what do you need returned?
$options = array();
$options['ResponseGroup'] = 'SalesRank,ItemAttributes,Reviews';
 
// for which books
// comma-delimited list of ISBNs
$items = '1904811795,1904811914,1904811132';
 
// do the request
$result = $amazon->ItemLookup($items, $options);
 
// check for errors
if (PEAR::isError($result)) {
    print "An error occured<br/>";
    print $result->getMessage() . "<br/>";
    exit();
}
 
 
// some spaghetti to display HTML response
echo '<ul>';
foreach ($result['Item'] as $book) { // loop the books
    // URL
    echo '<li><a href="'. $book['DetailPageURL'] .'">';
    // book title
    echo $book['ItemAttributes']['Title'], '</a><br />';
    // authors, comma-delimited
    echo implode(', ',$book['ItemAttributes']['Author']);
    // sales rank
    echo '<br />Sales rank: ', $book['SalesRank'];
    // average rating
    if (!empty($book['CustomerReviews'])) {
        echo '<br />Rating: ';
        echo $book['CustomerReviews']['AverageRating'];
        echo ', based on ';
        echo $book['CustomerReviews']['TotalReviews'], ' reviews';
 
    }
    echo '</li>';
}
echo '</ul>';
?>
 
<a
  href="#"
  onclick="javascript:document.getElementById('response').style.display='block';"
  >Show complete response</a>
<pre id="response" style="display: none">
<?php print_r($result); ?>
</pre>
 

Reusing an existing database connection with MDB2

Thursday, January 4th, 2007

This is a follow up to a question posted by Sam in my DB-2-MDB2 post. The question was if you can reuse an exisitng database connection you've already established and not have MDB2 creating a second connection.

When using a non-persistent connection

No worries in this case. No new connection will be established. As the PHP manual states:

If a second call is made to mysql_connect() with the same arguments, no new link will be established, but instead, the link identifier of the already opened link will be returned.

That is, if you don't set the fourth parameter to mysql_connect() to true. This parameter forces a new connection. BTW, in MDB2 if you do want to force a new connection, you have to set new_link in the DSN string to true

Bottom line, if you don't do anything special, the existing connection will be reused by MDB2. You can always verify that this is the case by calling phpinfo(INFO_MODULES); and looking in the "mysql" section.

When using a persistent connection

When using a persistent connection you have to do some additional steps to ensure that the same persistent connection is used by MDB2.

  • Tell MDB2 that you want a persistent connection - $mdb2->setOption('persistent', true);
  • Tell MDB2 which connection you want to use - $mdb2->connection = $link;, where $link is your existing connection
  • Set $mdb2->opened_persistent = true;

Here's an example:

<?php
// somewhere you've established a connection
$link = mysql_pconnect('localhost', 'root', '');
mysql_select_db('test', $link);
echo $link; // e.g. Resource id #5
 
// Create MDB2 object
require_once 'MDB2.php';
$dsn = 'mysql://root@localhost/test';
$mdb2 =& MDB2::factory($dsn);
 
// reuse your connection
$mdb2->setOption('persistent', true);
$mdb2->opened_persistent = true;
$mdb2->connection = $link;
 
// connect
$mdb2->connect();
echo $mdb2->connection; // Resource id #5
 
// check the "mysql" part to be sure
phpinfo(INFO_MODULES);
?>
 

Laziest image resize in PHP

Wednesday, December 13th, 2006

Today I saw a post at digg.com on image resizing with PHP and there was quite a discussion. Let me share the laziest way (that I know of) how to do it - PEAR::Image_Transform is all it takes. Here goes:

<?php
require_once 'Image/Transform.php';
$i =& Image_Transform::factory('');
 
$i->load('test.jpg');
$i->fit(100,100);
$i->save('resized.png', 'png');
?>

In addition, the Image_Transform library offers diffferent ways (to skin the old cat) to resize an image - by given pixel value, only on the X axis, on Y, scalling in percentage and so on. And, of course, the library can do much more than resizing, as you can see in the API docs.

It supports all kinds of image manipulation extensions - GD, GD1, ImageMagick, NetPBM, Imlib... If you want to use a specific one, you set as a parameter to the factory() method. In the example above I passed an empty string, so it will try to figure out what's available in my PHP setup and use it, trying Imagick2 first, then GD, then Imlib.

You have setOption() and setOptions() methods if you want to play around with the image quality and those sort of things.

 

Performance tuning with MDB2

Saturday, December 9th, 2006

This is a follow-up to Lars' comment about the PEAR book. In the MDB2 chapter I showed an example how you can create custom debug handlers in MDB2 and then gave a suggestion about a useful application of this functionality for performance tuning. Basically the idea is that your custom debug handler collects all queries that are executed during the life of a given script. Then, once the script finishes execution, the debug handler reports the stats that it has collected. In the book, the example is how you count the number of times each distinct query is executed, this way you can spot problems caused by the OO abstraction. For example, say you have a come class Users that has a method loadUser(), which abstracts the database work. While debugging with the custom error handler, you might figure out that without noticing, you're calling this method in a few places and it makes the same repeating query(queries) over and over again. So you can now optimize/cache results and so on.

The suggestion I made in the book is that in addition to counting, you might want to try executing all SELECTs again, just to see how much time they take and you can execute them once again, prepending them with EXPLAIN to get some details on possible room for improvement.

Now here's one solution to this suggestion. What you can see in this script is:

  • Setting up MDB2
  • Declaring a custom debug handler class
  • "Attaching" it to the MDB2 instance
  • Registering it for execution at the end of each script
  • Testing it (creating a DB, table, some queries)

I hope you like it and try it out.

Here's the result of executing this script, you can see what you get back.

Room for improvement

Obviously, the method dumpInfo() can be improved. First, it can print out a nice table, instead of lazy print_r(). Then, it can include some logic, my idea is for it to "understand" the EXPLAIN results and to give you a hint by using colors, for exampe green background for queries that are OK, yellow for warnings and red for queries that definitelly need some work. Could be nice, no?

Test script

Kinda longish, but I hope I added enough comments. I also hope I didn't introduce any syntax errors while formatting it for posting here, chopping long lines, etc.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
<?php
 
// PEAR error handling setup
require_once 'PEAR.php';
function pearError ($e)
{
  echo '<pre>';
  echo $e->getMessage().': '.$e->getUserinfo();
  echo '</pre>';
}
PEAR::setErrorHandling(
  PEAR_ERROR_CALLBACK,
  'pearError'
);
 
// creating MDB2 instance
require_once 'MDB2.php';
$dsn = 'mysql://root:test@localhost';
$mdb2 =& MDB2::factory($dsn);
$mdb2->setFetchMode(MDB2_FETCHMODE_ASSOC);
 
// The custom error handler
//
// It will collect all the queries being executed
// in the script, the collection is done by the
// collectInfo() method.
// Once the script finishes executing, we'll call
// the method executeAndExplain() which will
// execute all unique SELECTs once again
// in order to give us an info of how much time
// each query takes.
// Then executeAndExplain() will execute again
// all SELECTs, this time prepending an EXPLAIN
// so that we can get valuable
// optimization-related information
// Not only that but instead of simple EXPLAIN,
// we can use EXPLAIN EXTENDED and after that
// we can call SHOW WARNINGS -
// this will give us even more optimization hints
//
// http://dev.mysql.com/doc/refman/5.1/en/explain.html
// http://dev.mysql.com/doc/refman/5.1/en/show-warnings.html
//
class Explain_Queries
{
  // how many queries were executed
  var $query_count = 0;
  // which queries and their count
  var $queries = array();
  // results of EXPLAIN-ed SELECTs
  var $explains = array();
  // the MDB2 instance
  var $db = false;
 
  // constructor that accepts MDB2 reference
  function Explain_Queries(&$db) {
    $this->db = $db;
  }
 
  // this method is called on every query
  function collectInfo(
    &$db,
    $scope,
    $message,
    $is_manip = null)
  {
    // increment the total number of queries
    $this->query_count++;
    // the SQL is a key in the queries array
    // the value will be the count of how
    // many times each query was executed
    @$this->queries[$message]++;
  }
 
  // print the debug information
  function dumpInfo()
  {
    echo '<h3>Queries on this page</h3>';
    echo '<pre>';
    print_r($this->queries);
    echo '</pre>';
    echo '<h3>EXPLAIN-ed SELECTs</h3>';
    echo '<pre>';
    print_r($this->explains);
    echo '</pre>';
  }
 
  // the method that will execute all SELECTs
  // with and without an EXPLAIN and will
  // create $this->explains array of debug
  // information
  // SHOW WARNINGS will be called after each
  // EXPLAIN for more information
  function executeAndExplain() {
 
    // at this point, stop debugging
    $this->db->setOption('debug', 0);
    $this->db->loadModule('Extended');
 
    // take the SQL for all the unique queries
    $queries = array_keys($this->queries);
    foreach ($queries AS $sql) {
 
      // for all SELECTs…
      $sql = trim($sql);
      if (stristr($sql,"SELECT") !== false){
        // note the start time
        $start_time = array_sum(
            explode(" ", microtime())
        );
        // execute query
        $this->db->query($sql);
        // note the end time
        $end_time = array_sum(
            explode(" ", microtime())
        );
        // the time the query took
        $total_time = $end_time - $start_time;
 
        // now execute the same query with
        // EXPLAIN EXTENDED prepended
        $explain = $this->db->getAll(
          'EXPLAIN EXTENDED ' . $sql
        );
 
        $this->explains[$sql] = array();
        // update the debug array with the
        // new data from
        // EXPLAIN and SHOW WARNINGS
        if (!PEAR::isError($explain)) {
          $this->explains[$sql]['explain'] = $explain;
          $this->explains[$sql]['warnings'] =
               $this->db->getAll('SHOW WARNINGS');
        }
 
        // update the debug array with the
        // count and time
        $this->explains[$sql]['time'] = $total_time;
      }
    }
  }
}
 
// instance of the custom debug handler
$my_debug_handler = new Explain_Queries($mdb2);
// set debug option
$mdb2->setOption('debug', 1);
// set debug handler to the method that
// collects all queries
$mdb2->setOption(
  'debug_handler',
  array($my_debug_handler, 'collectInfo')
);
// register functions to be executed on shut down
// after the script has finished execution.
// Now that the show's over, it's the time to
// report what happened in this script db-access-wise
// First shutdown function executes the
// SELECTs again, the other one prints the results
register_shutdown_function(
  array($my_debug_handler, 'executeAndExplain')
);
register_shutdown_function(
  array($my_debug_handler, 'dumpInfo')
);
 
 
//
//
// At this point all MDB2 setup is done,
// time for the actual script to do something
//
//
 
// load the DB manager module
$mdb2->loadModule('Manager');
 
// drop database if it exists
// temporarily change the PEAR error handling
PEAR::pushErrorHandling(PEAR_ERROR_RETURN);
$mdb2->dropDatabase('test_db_explain');
PEAR::popErrorHandling();
 
// create and set a new database
$mdb2->createDatabase('test_db_explain');
$mdb2->setDatabase('test_db_explain');
 
// create table "events" from a definition array
// the table has event ID, name and date/time
$definition = array (
  'id' => array (
    'type' => 'integer',
    'unsigned' => 1,
    'notnull' => 1,
    'default' => 0,
  ),
  'name' => array (
    'type' => 'text',
    'length' => 255
  ),
  'datetime' => array (
    'type' => 'timestamp'
  )
);
 
$mdb2->createTable('events', $definition);
 
// create a primary key - the ID field
$definition = array (
  'primary' => true,
  'fields' => array (
    'id' => array()
  )
);
$mdb2->createConstraint(
  'events',
  'myprimekey',
  $definition
);
 
// load the class that has some static helper
// functions to work with MDB2's cross-RDBMS
// date format
MDB2::loadFile('Date');
 
// INSERT
// some data to insert into the events table
$data = array(
  // using MDB2-managed sequences
  'id'     => $mdb2->nextId('events'),
  'name'     => "Breakfast a Tiffany's",
  'datetime'   => MDB2_Date::unix2Mdbstamp(
    strtotime('Jan 15, 2007')
  )
);
// The "datetime" value shows how you can use
// any date format you wish as long as you're
// able to get a unix timestamp out of it
// In this case I'm using strtotime()
// Then there is a call to MDB2's date helper
// to get the MDB2 timestamp
 
// auto insert
// for the autoExecute() method we need to
// load the Extended module
$mdb2->loadModule('Extended');
$result = $mdb2->autoExecute(
  'events',
  $data,
  MDB2_AUTOQUERY_INSERT
);
 
//
// Time to SELECT something
//
// Using date helpers again
$start_date = MDB2_Date::date2Mdbstamp(0,0,0,12,31,1980);
$end_date   = MDB2_Date::date2Mdbstamp(0,0,0,12,31,2020);
$sql = 'SELECT * FROM %s WHERE %s > %s AND %s < %s';
$sql = sprintf(
  $sql,
  $mdb2->quoteIdentifier('events'),   // quote table name
  $mdb2->quoteIdentifier('datetime'), // quote field name
  $mdb2->quote($start_date, 'date'),  // quote data as date
  $mdb2->quoteIdentifier('datetime'), // quote field name
  $mdb2->quote($end_date,   'date')   // quote data as date
);
$res = $mdb2->getAll($sql); // execute
 
//
// * Bad practice code follows *
// Just some more inserts and selects, bad
// practice because these queries are not
// necessarily portable accross various RDBMS,
// lacking proper quoting and preparation
for ($i = 2; $i < 31; $i++) {
  $mdb2->query(
    'INSERT INTO events VALUES('
    . $i
    . ', "test event", "2005-05-05 00:00:00")'
  );
}
$res = $mdb2->getCol(
  'SELECT DISTINCT datetime FROM events'
);
$res = $mdb2->getRow(
  'SELECT * FROM events WHERE id
  IN (SELECT id FROM events WHERE id > 1)'
);
?>
 

Personal news update Nov/06

Wednesday, November 1st, 2006

So what I've been up to recently? Having a bit of a break, I guess. First, I'm not currently writing a book, after the last one for which I completed my chapter back in June. That's some free time (There is a very exciting book project on the horizon though, we'll see). Then, I'm not working so much on our new house. Now (August) that my family is back from Bulgaria, and the two little princesses are running around, it's next to impossible to do any construction work, however small.

I changed my job not so long ago, I'm working for SAP now, here in good old Old Montreal. Meeting smart people every day and learning new things, SAP has quite a bit going on and there's always something to learn. Take the proprietary ABAP programing language for one.

As always a PHP junkie, I looked into what's possible in terms of integration of PHP and SAP. Turns out it's possible and it's fun. There is this open-source SAPRFC PHP extension, that allows you to use PHP to connect to an SAP system and do stuff. Out of this interest a few things happened:

  • I published an article at the International PHP Magazine about a tool (or more like a collection of tools), called Scripting In A Box which is developed at SAP. It's one big archive (as in ZIP) which you unpackage to your C:\ drive and you get Apache, MySQL, PHP(+SAPRFC), Perl, Ruby/ROR, Python, Eclispse(+PHPEclipse), all pre-configured and running together. So you can start scripting in minutes. This tool actually gave me a chance to try out and love PHPEclipse, something I've missed, being so attached to my TextPad and ignoring any other way to do PHP. Now I can highly recommend PHPEclipse as a PHP IDE.
  • Next, some folks at SAP (Thanks Craig, André!) recognized my PHP experience and asked me to do a little demo of SAP+PHP at SAP's big event, called TechEd in Las Vegas. This was quite an experience! Las Vegas is one different place and the conference itself was pretty big with 5000 people, I think. I had a chance to meet guys from SDN (SAP Developers Network), which is quite a vivid community with something like half a million members. You know what's the everage response time when you post a question on the SDN forums? 7 minutes.
  • Then, I wrote another article for IPM, which described the demo I did at TechEd. (I'll add the source code and some screenshots at the bottom of this post.)
  • I also contributed one new container for the PEAR::Auth package, it allows you to authenticate users against an SAP system in your PHP app.
  • Another contribution to PEAR was the ABAP language definition for the Text_Highlighter package
  • Finally, I did a little ABAP console, but I'll blog about it seperately and will share the code, of course.
  • ... and my first posting on SDN was published today. The next one will be a sort of a cross-post here and on SDN about the ABAP console thingie.

I think that's about it for SAP. Otherwise, as usual, I get easily excited by different things, so I've been doing this and that, here and there, on my own terms, relaxing, without any deadlines preasure.

On the pipeline, I have a bit of stuff to do, again, small little things I enjoy doing, like helping with one article for the PEAR::MDB2 manual, assembling an extra intro chapter for the PEAR book, helping out with Text_Highlighter (I'm this package's official helper since a few days ago), also doing some work for the Image_Text PEAR package, as well as anything else that comes into the radar any given day. Yeah, this is how I understand relaxation, doing whatever you're passionate about, even if this means less sleep at night. Ah, and I have the test for Canadian citizenship tomorrow, so I should be reading as opposed to writing now (I always do this, the busier I am, the more interesting things I "shoehorn")

SAPRFC/PHP demo files

Yeah, the demo app uses YUI and a bit of AJAX and animation to make it a bit sexier.

 

JSON renderer for Text_Highlight

Friday, October 27th, 2006

Text_Highlighter is one of my favourite PEAR packages, seems like I'm addicted to highlighting source code. After adding BB code and simple HTML renderers and an ABAP code syntax definition, today I played with adding a JSON renderer. Useful in case you want to get highlighted source code in your new shiny AJAX app.

Array renderer

After I did the JSON renderer, I said, OK, what if I want to tweak the JSON output just a bit (or the output from any renderer for that matter)? Add more options? Nah, I had a better idea, I scrapped the whole thing and did an Array renderer first. If you have the array output from the renderer, it's trivial to format it as JSON, or XML, or HTML, or anything. I believe even the exisitng Text_Highlighter renderers should be rewritten, to extend an Array renderer. Anyway, back to JSON.

Demo

To see the JSON renderer in action, you can go to my hiliteme.com site and check the JSON tab.

Source

The source code is available here - JSON.phps which extends Array.phps. To test, you need to add the two renderers to your PEAR repository under Text/Highlighter/Renderer

Example

So let's say you need to highlight the PHP code

<?php 
    echo "Hello Highlighted World!"; 
?>

You create an instance of Text_Highlighter and Text_Highlighter_Renderer_JSON and call the highlight() method, assuming that the code you need highlighted is in $source

<?php
// dependencies
require_once 'Text/Highlighter.php';
require_once 'Text/Highlighter/Renderer/JSON.php';
 
// instance
$json_options = array();
$json_options['tabsize'] = 4;
$json_options['enumerated'] = true;
$renderer =& new Text_Highlighter_Renderer_JSON($json_options);
$highlighter =& Text_Highlighter::factory($_POST['language']);
$highlighter->setRenderer($renderer);
 
// do the highlighting
$json_result = $highlighter->highlight($source);
?>

Now $json_result will look like:

[["inlinetags","&lt;?php"],["code"," \\n    "],["reserved","echo"],["code"," "],["quotes","&quot;"],["string","Hello Highlighted World!"],["quotes","&quot;"],["code","; \\n"],["inlinetags","?&gt;"]]

As you see the JSON output is an array, one element per highlighted keyword, and in this array there is a sub array - class/keyword. If you want to display this in your page (let's say you got it from an AJAX call), you can do a loop through the array and surround the keywords with span tags of the selected style:

// say your ajax call returned var src 
// var src = xmlhttp.responseText;
var data = eval(src); 
 
var res = ''; 
for (var i in data) {
    res += '<span class="hl-' + data[i][0] + '">';
    res += data[i][1];
    res += '</span>';
}
 
var el = document.getElementById('some-div').
el.innerHTML = '<pre>' + res + '</pre>';

Here I used innerHTML, you can also use DOM, but in this case you need a special case for the "\n" so that you can create a br element to address IE's habit of ignoring line feeds in a DOM-generated pre tag.

BTW, if you don't set the enumerated option to true, you'll get objects inside the main array, check hiliteme.com's JSON tab for an idea of how this would look like.

 

The PEAR book

Sunday, October 15th, 2006

PEAR bookIn case you've missed it - the PEAR book hit the streets! The exact title is "PHP Programming with PEAR" and it's co-written by Stephan Schmidt, Carsten Lucke, Aaron Wormus and yours truly. Aaron also put up a Wiki for book and PEAR-related updates, it's at thepearbook.com

I tried to put up a list of the packages and classes covered in the book, I've probably missed some classes, especially Date_* and Calendar_* ones, but I hope I got all the packages. Here goes (alphabetically) :

  • Calendar
  • Date
  • Date_Holidays
  • Date_Span
  • Date_Timezone
  • File_PDF
  • HTML_Table
  • HTML_Table_Matrix
  • HTTP_Request
  • MDB2
  • MDB2_Schema
  • Services_AmazonESC4
  • Services_Ebay
  • Services_Google
  • Services_Technorati
  • Services_Webservice
  • Services_Yahoo_Search
  • Spreadsheet_Excel_Writer
  • Structures_DataGrid
  • Structures_DataGrid_Column
  • Structures_DataGrid_DataSource
  • XML_Beautifier (mention)
  • XML_FastCreate
  • XML_Parser
  • XML_RPC
  • XML_RPC_Client
  • XML_RPC_Message
  • XML_RPC_Response
  • XML_RPC_Server
  • XML_RPC_Value
  • XML_RSS
  • XML_Serializer
  • XML_Util
  • XML_XUL

For more info on a package, you can consult the PEAR site and manual. Did you know that you can access a package's page by typing its name (case insensitive) in the URL after pear.php.net, like http://pear.php.net/mdb2_SCHEMA for example?

 

ABAP code syntax highlighting

Tuesday, September 19th, 2006

Just finished and eager to share - I added a new syntax definition to the Text_Highlighter PEAR package (see also here). It's for highlighting code written in the SAP's own ABAP programming language.

A live demo is available at the hiliteme.com site, just pick ABAP from the drop-down of programming languages. Any feedback is appreciated, because it's a brand new thing and may have bugs or incompletenesses (is that a word?). So feel free to highlight your ABAP code and post it to blogs or forums.

The implementation wasn't hard, the Text_Highlighter package is made to be extended and even provides the tools for that. All you need to do is create an XML file that contains keywords and other syntax rules, such as formats of the comments and so on. Then there is a command line tool that takes the XML file and generates a class out of it. The class is later on used when highlighting. Here's the XML file in case you want to improve on it and generate your own ABAP.php class:

 

The PEAR book is on it’s way

Tuesday, September 12th, 2006

PEAR book Here's the link to publisher's page dedicated to the PHP Programming with PEAR. Guess who wrote the chapter for MDB2? ;)

It's an honour to me to be in the company of the other authors, people who have done a lot for the PEAR community:

Here's what the book is about:

  • Accessing databases with MDB2
  • Displaying data in a range of formats (HTML, Excel spreadsheet, PDF)
  • Creating and parsing XML documents
  • Serializing PHP objects into XML, and unserializing XML documents to PHP objects
  • Consuming and offering web services
  • Accessing Web APIs including Google, Yahoo, Amazon, and Technorati
  • 250 pages of good stuff :) Get your copy or just spread the word!

    Update: In the rush to share the news I forgot to say a big "thank you!" to Lukas Smith, the man behind MDB2, who was responsive as always and was kind enough to review my chapter.

     

    HiLiteMe.com updated

    Saturday, September 9th, 2006

    Following from here, I'm proud to announce an update to HiLiteMe.com. With two custom renderers for Text_Highlighter, the service now offers you:
    - BB code for your source, so you can post beautiful code to bulletin boards and others that use BB code for formatting
    - simple HTML code - the formatting is using only the tags b, i and u, this is for devices such as iPod that can understand only tiny amount of HTML code. So you highlight some code and take it on the road, sweet.

    In addition to that there is Simple HTML preview, CSS styles that are used for the "rich" HTML highlighting and also the code for the rich HTML formatting, so no need to view->source->copy

    Other additions - tabsize setting (how many spaces for a tab, if you paste code with tabs in it) and line numbers setting (yes/no) to specify whether you want lines to be numbered in your code.

    Test drive?

    Note on the BB code highlighting - I tested on:
    - vBulletin - works beautifully and
    - phpBB - the current out-of-the-box version will not work, becuase it's not allowing [color] BB tags inside [code] tags. Buuut, there is a MOD to enable this

     

    HiLiteMe.com

    Sunday, September 3rd, 2006

    In case someone is wondering how do I highlight the code I post on this blog ... well, the lazy way. I don't. Some time ago I setup a free service, hiliteme.com to do it for me, then I just copy/paste the generated code. It's far from the best solution, but it's definitelly the laziest, without any spend-time-to-save-time effort on my end ;)

    So if you ever need to highlight source code - for a blog, word doc or whatever, you can use this free service, I'll be happy. HiLiteMe.com is using the PEAR Text_Highlighter package (I talk about it here)

    BTW, I think in general it's a good idea is to use JavaScript to do the code highlighting. After all, it's just presentation and if it can be done on the client, why loading you server with this task. I know at least two free scripts that do that, there's probably more. The one that looks very good is this one.

     

    SAP container for PEAR::Auth

    Friday, September 1st, 2006

    PEAR::Auth is a package that allows you to abstract the user authentication from the main part of your application and not worry about it. What is good about the package is that it comes with different "containers" that allows you to authenticate users against different storages, I mean you can store users data in a database and use the PEAR::DB or PEAR::MDB2 containers, or you can use flat files, IMAP servers, SOAP and what not. And the package is easily extensible. So I played around with creating an SAP container that allows you to check users against your company's SAP system and for example build a section of your Internet (or Extranet) page that is only accessible for people and partners that exist as users in the SAP system.

    In order to connect to an SAP system with PHP you need the SAPRFC PHP extension. Get it here. Then you use the function saprfc_open() (more docs here) to establish a connection. You provide some info about the SAP system as well as your username/password. Once connected, a so-called "SSO ticket" is generated for you. This is just a long string, like a session ID. For consecutive connections you can use this SSO ticket instead of providing username/password every time. BTW, SSO stands for Single Sign-On.

    Now, with my little SAP container you can benefit from the PEAR and PEAR::Auth infrastructure to do the logins. The way to do an authentication is simple (example stolen in parts from this PEAR manual entry). You pass the connection options (such as hostname). Then, once the user is authenticated, the container retrieves the SSO session ID and sticks into the Auth session data, so that it's reusable for consecutive connections within the same session. If you need to do more with the SAP system, apart from authenticating users, you can get back the updated connection options and just pass them to saprfc_open(). Here's an example:

    <?php
    // get Auth lib
    require_once "Auth.php";
     
    // SAP connection options
    $options = array (
        'ASHOST'    => 'hostname'
    );
     // create Auth object using the SAP container
    $a = new Auth("SAP", $options);
     
    
    $a->start();
     
    // check
    if ($a->checkAuth()) {
    
     
        // authorised! You can do the protected stuff here
     
        // For example open a connection to the SAP system
        // using the stored authentication data
        $rfc = saprfc_open($a->getAuthData('sap'));
     
        // show sapinfo if you will
        echo '<pre>';
        print_r(saprfc_attributes($rfc));
        echo '</pre>';
    
    }
    ?>

    And here's the actual Auth_Contaner_SAP class, should be placed in a file called SAP.php in your_pear_dir/Auth/Container/

    <?php
    require_once 'PEAR.php';
    require_once 'Auth/Container.php';
    /**
     * Performs authentication against an SAP system
     * using the SAPRFC PHP extension.
     *
     * When the option GETSSO2 is TRUE (default)
     * the Single Sign-On (SSO) ticket is retrieved
     * and stored as an Auth attribute called 'sap'
     * in order to be reused for consecutive connections.
     *
     * @author Stoyan Stefanov <ssttoo@gmail.com>
     * @package Auth
     * @see http://saprfc.sourceforge.net/
     */
    class Auth_Container_SAP extends Auth_Container {
        /**
         * @var array Default options
         */
        var $options = array(
            'CLIENT'    => '000',
            'LANG'      => 'EN',
            'GETSSO2'   => true,
        );
     
        /**
         * Class constructor. Checks that required options
         * are present and that the SAPRFC extension is loaded
         *
         * Options that can be passed and their defaults:
         * <pre>
         * array(
         *   'ASHOST' => "",
         *   'SYSNR'    => "",
         *   'CLIENT' => "000",
         *   'GWHOST' =>"",
         *   'GWSERV' =>"",
         *   'MSHOST' =>"",
         *   'R3NAME' =>"",
         *   'GROUP'    =>"",
         *   'LANG'     =>"EN",
         *   'TRACE'    =>"",
         *   'GETSSO2'=> true
         * )
         * </pre>
         *
         * @var array array of options.
         */
        function Auth_Container_SAP($options)
        {
            $saprfc_loaded = PEAR::loadExtension('saprfc');
            if (!$saprfc_loaded) {
                return PEAR::raiseError('Cannot use SAP authentication, '
                        .'SAPRFC extension not loaded!');
            }
            if (empty($options['R3NAME']) && empty($options['ASHOST'])) {
                return PEAR::raiseError('R3NAME or ASHOST required for authentication');
            }
            $this->options = array_merge($this->options, $options);
        }
    
        /**
         * Performs username and password check
         *
         * @var string Username
         * @var string Password
         * @return boolean TRUE on success (valid user), FALSE otherwise
         */        
        function fetchData($username, $password)
        {
            $connection_options = $this->options;
            $connection_options['USER'] = $username;
            $connection_options['PASSWD'] = $password;
            $rfc = saprfc_open($connection_options);
            if (!$rfc) {
                $message = "Couldn't connect to the SAP system.";
                $error = $this->getError();
                if ($error['message']) {
                    $message .= ': ' . $error['message'];
                }
                PEAR::raiseError($message, null, null, null, @$erorr['all']);
                return false;
            } else {
                if (!empty($this->options['GETSSO2'])) {
                    if ($ticket = @saprfc_get_ticket($rfc)) {
                        $this->options['MYSAPSSO2'] = $ticket;
                        unset($this->options['GETSSO2']);
                        $this->_auth_obj->setAuthData('sap', $this->options);
                    } else {
                        PEAR::raiseError("SSO ticket retrieval failed");
                    }
                }
                @saprfc_close($rfc);
                return true;
            }
    
         }
        /**
         * Retrieves the last error from the SAP connection
         * and returns it as an array.
         *
         * @return array Array of error information
         */
        function getError()
        {
    
            $error = array();
            $sap_error = saprfc_error();
            if (empty($err)) {
                return $error;
            }
            $err = explode("\n", $sap_error);
            foreach ($err AS $line) {
                $item = split(':', $line);
                $error[strtolower(trim($item[0]))] = trim($item[1]);
            }
            $error['all'] = $sap_error;
            return $error;
        }
    }
    ?>
     

    phpDoc clip library for TextPad

    Wednesday, August 30th, 2006

    Here's a little something I did to make it a bit easier to write API docs in TextPad, it's a clip library to save some typing when writing comments in the phpDoc format. I submitted it to the TextPad team, so at some point it will probably appear in the downloads section, but meanwhile you can get from here.

    Installation

    1. Download the file phpdoc.tcl
    2. Copy to your Samples directory in the TextPad folder
    3. Make sure the Clip Library panel is on (menu option View -> Clip Library)
    4. Choose phpDoc clip from the dropdown in the Clip Library panel

    Some screenshots

    Type in some class description, highlight and then select "Class comment"

    phpdoc1.PNG phpdoc2.PNG

    Example class property comment
    phpdoc3.PNG phpdoc4.PNG

     

    Bulgaria, IPM, quick update

    Monday, June 12th, 2006

    I'm currently on a vacation in my native land Bulgaria, the party's on (and so the soccer World Cup finals) so It wuould be quiet around here for a while. Meanwhile I've disabled comments, trackbacks and pingbacks, I appologize, it's just that I'm receiving a lot of spam and since I don't have the time to clean it, the spam will look as an insult to my readers.I cannot find the WordPress option to disable comments retroactively, so all comments will be held for moderation. I appologize once again.

    Meanwhile my article on DB and MDB2 was published in the International PHP Magazine, the TOC is here. This article is an extended and improved version of the original DB-to-MDB2 blog posting you can find here, plus I've added an intro part in case you've never used DB or MDB2.

    BTW, I'm enjoying writing this post on a dial-up and using IE5.5., this is an experience that is pretty ... interesting ;)

    l8r!

     

    DB-2-MDB2

    Saturday, February 4th, 2006

    Intro

    Recently I had to move an existing project from using PEAR::DB to PEAR::MDB2 - the new database abstraction layer. I took notes on the parts of the code I needed to change, I hope they can benefit someone who's doing the same. Many thanks go to Lukas Smith, the lead developer, he was always responding very fast to my reports and questions in the PEAR mailing list.

    One thing to notice in MDB2 is that it tries not to do any unnecessary work and does many things only on demand. For example when you create an object, that doesn't mean that a connection is established. It is established only when you make the first real database access, a SELECT for example.

    I assume you have an idea of PEAR::DB, since this posting illustrates a DB-to-MDB2 endeavour, but even if you don't, I hope the posting will still be useful as an intro to DB and MDB2.

    Including the libs

    So first off, including the libs (I assume you have PEAR on your machine).

    require_once 'DB.php';
    require_once 'MDB2.php';
    

    One thing to note here is that installing MDB2 doesn't install any of the database wrappers. So if you use MySQL for example, you'd need to install it separately:
    pear install MDB2_Driver_mysql-beta
    in addition to
    pear install MDB2-beta

    MDB2 is now a stable release! So you can now remove the "-beta" monkier when installing the packages.

    DSN

    Next - the DSN string. It's the same for MDB2 as for DB.

    $dsn = 'mysql://root@localhost/db2mdb2';
    

    BTW, MDB2 can also accept an array of all the connection details, as opposed to a DSN string. And so does DB (Thanks for the clarification, Justin!)

    Creating instances

    $db =& DB::connect($dsn);
    $mdb2 =& MDB2::factory($dsn);
    

    MDB2 provides a factory method to create an instance. At this time no database connection is yet established. MDB2 also provides a singleton() method to create an instance.

    Fetchmode

    It is the same in both DB and MDB2, just note the prefix of the constant.

    // set fetchmode
    $db->setFetchMode(DB_FETCHMODE_ASSOC);
    $mdb2->setFetchMode(MDB2_FETCHMODE_ASSOC);
    

    Simple SELECTs

    There are the methods to select one row, one column, one cell and a bunch of records. DB prefixes them with get, while MDB2 uses query.

    // select several records and shove them into an array
    $all = $db->getAll('SELECT * FROM people');
    $all = $mdb2->queryAll('SELECT * FROM people');
    
    
    // select one cell
    $one = $db->getOne('SELECT name FROM people WHERE id = 1');
    $one = $mdb2->queryOne('SELECT name FROM people WHERE id = 1');
    
    
    // one row
    $row = $db->getRow('SELECT * FROM people WHERE id = 1');
    $row = $mdb2->queryRow('SELECT * FROM people WHERE id = 1');
    
    
    // a column
    $col = $db->getCol('SELECT name FROM people');
    $col = $mdb2->queryCol('SELECT name FROM people');
    

    Quoting values

    In DB, the suggested method to quote is quoteSmart(). In MDB2 it's quote() and it accepts a second parameter, which tells the type of the value to be quoted. If the second parameter is omitted, MDB2 will try to guess the type.

    $one = $db->getOne(
             'SELECT name FROM people WHERE id = ' 
             . $db->quoteSmart(1)
           );
    
    
    $one = $mdb2->queryOne(
             'SELECT name FROM people WHERE id = ' 
             . $db->quote(1, 'integer')
           );
    

    Sequence tables

    If you use sequence tables, both libs will provide you with a nextId() method:

    echo $db->nextId('people_db');
    echo $mdb2->nextId('people_mdb2');
    

    The only difference is that when DB creates a sequence table (with one field and one value), the name of the field is id, where MDB2 will use sequence. If you're translating an existing project to MDB2 like me, and the sequence tables are already created by DB, you have the option of renaming this field in the database for all sequence tables, or you can set an MDB2 option and you're good to go.

    $mdb2->setOption('seqcol_name','id');
    

    Auto execute

    Say you have the data:

    $data = array('id' => 5, 'name' => 'Cameron');
    

    To auto-insert it using DB, you'd do:

    $db->autoExecute('people', $data, DB_AUTOQUERY_INSERT);
    

    For MDB2, the auto execution is probably not considered an often-used feature, so it's not in the base instance. You need to load an additional module to have access to it:

    $mdb2->loadModule('Extended');
    

    Now you can

    $mdb2->autoExecute('people', $data, MDB2_AUTOQUERY_INSERT);
    

    The above will work in PHP5 only. In PHP4, due to the limited support of object overloading (Thanks again to Lukas for clarifying this!), you'd need to do:

    $mdb2->extended->autoExecute('people', $data, MDB2_AUTOQUERY_INSERT);
    

    Note that the second way will also work in PHP5.

    Prepared statements

    In DB:

    $statement = $db->prepare('INSERT INTO people VALUES (?, ?)');
    $data = array(6, 'Chris');
    $db->execute($statement, $data);
    $db->freePrepared($statement);
    

    In MDB it's almost the same, only that the statement becomes an object and you call its (as opposed to MDB2 main object's) methods to execute and to release memory:

    $statement = $mdb2->prepare('INSERT INTO people VALUES (?, ?)');
    $data = array(7, 'Dave');
    $statement->execute($data);
    $statement->free();
    

    Execute multiple

    The same applies to executing a statement with with multiple "rows" of data from an array. executeMultiple() is in the Extended MDB2 module, so you need to load it:

    DB:

    $statement = $db->prepare('INSERT INTO people VALUES (?, ?)');
    $data = array(
        array(8, 'James'),
        array(9, 'Cliff')
    );
    
    $db->executeMultiple($statement, $data);
    $db->freePrepared($statement);
    

    MDB2:

    $statement = $mdb2->prepare('INSERT INTO people VALUES (?, ?)');
    $data = array(
        array(10, 'Kirk'),
        array(11, 'Lars')
    );
    
    $mdb2->loadModule('Extended');
    $mdb2->extended->executeMultiple($statement, $data);
    
    $statement->free();
    

    Transactions

    In DB:

    $db->autoCommit();
    $result = $db->query('DELETE people'); // will cause an error
    
    if (PEAR::isError($result)) {
        $db->rollback();     //echo 'rollback';
    } else {
        $db->commit();     //echo 'commit';
    
    }
    

    In MDB2 you have to check if transactions are supported in your RDBMS. Then during the transaction, you can always check "Am I in transaction?"

    if ($mdb2->supports('transactions')) {
        $mdb2->beginTransaction();
    
    }
    $result = $mdb2->query('DELETE people');
    if (PEAR::isError($result)) {
        if ($mdb2->in_transaction) {
            $mdb2->rollback();         // echo 'rollback';
        }
    } else {
        if ($mdb2->in_transaction) {
            $mdb2->commit();         // echo 'commit';
        }
    }
    

    Example script

    You can download a script that has the examples above and play with it. Here's also the sql file to recreate the database:

    Any questions or comments are welcome ;) Thanks for reading!

     

    Image fun

    Wednesday, September 14th, 2005

    Here are some examples of image manipulation using the GD library, more specifically these are pixel operations. Pixel operation meaning doing something to a pixel in an image with regards to this pixel only, not taking into account the neighbours.

    An example pixel operation is making a negative image. You take every pixel in an image and substitute it with its opposite color pixel.

    OK, so how is this thing working. Pretty simple. I take a PNG image, go through every pixel of this image and call a function passing the pixel as a parameter. The callback function returns a new pixel. I take all returned pixels and create a new image.

    Pixel class

    To get started I have a pixel class. It simply contains three integer values, the values of red, green and blue the pixel has.

    <?php
    class Pixel {
        function Pixel($r, $g, $b)
        {
            $this->r = ($r > 255) ? 255 : (($r < 0) ? 0 : (int)($r));
            $this->g = ($g > 255) ? 255 : (($g < 0) ? 0 : (int)($g));
            $this->b = ($b > 255) ? 255 : (($b < 0) ? 0 : (int)($b));
        }
    }
    ?>

    This class has only one method, the constructor of the class, which takes care to sanitize the RGB values.

    To create a red pixel, you simply do:

    <?php
    $red = new Pixel(255, 0, 0);
    ?>

    The pixel operations class and main method

    Then I have a class that will do the actual operations, I call the Image_PixelOperations. I didn't go through making a nice interface for reading and writing different file formats, I was thinking that this class can be developed further and based on PEAR's Image_Transform, which has tools for opening, validating, displaying, writing image files. What I needed the most was a simple method that opens a PNG, goes through every pixel, calls a function, gets a new pixel and writes the pixel to a new image. Hence, the pixelOperation method:

    <?php
    class Image_PixelOperations {
    
        function pixelOperation(
                $input_image,
                $output_image,
                $operation_callback,
                $factor = false
                )
        {
    
            $image = imagecreatefrompng($input_image);
            $x_dimension = imagesx($image);
            $y_dimension = imagesy($image);
            $new_image = imagecreatetruecolor($x_dimension, $y_dimension);
    
            if ($operation_callback == 'contrast') {
                $average_luminance = $this->getAverageLuminance($image);
            } else {
                $average_luminance = false;
            }
    
            for ($x = 0; $x < $x_dimension; $x++) {
                for ($y = 0; $y < $y_dimension; $y++) {
    
                    $rgb = imagecolorat($image, $x, $y);
                    $r = ($rgb >> 16) & 0xFF;
                    $g = ($rgb >> 8) & 0xFF;
                    $b = $rgb & 0xFF;
    
                    $pixel = new Pixel($r, $g, $b);
                    $pixel = call_user_func(
                        $operation_callback,
                        $pixel,
                        $factor,
                        $average_luminance
                    );
    
                    $color = imagecolorallocate(
                        $image,
                        $pixel->r,
                        $pixel->g,
                        $pixel->b
                    );
                    imagesetpixel($new_image, $x, $y, $color);
                }
    
            }
    
            imagepng($new_image, $output_image);
        }
    }
    
    ?>

    The method takes a filename, it is not doing any validation, it assumes it's a valid PNG file. The second parameter is the output filename. The third is the callback function that will be called on every pixel and the last parameter is any additional parameter we want to pass to the callback function.
    The average luminance is something specific to the "contrast" operation, so it's no too important for what pixelOrerations does.

    Adding noise

    Time for writing the first callback, the addNoise method. Adding noise to an image means adding a random value to each channel of a pixel. (If you're wondering, the value of red in a pixel is called a channel, and so are the blue and the green). Here goes the addNoise implementation.

    <?php
        function addNoise($pixel, $factor)
        {
            $random = mt_rand(-$factor, $factor);
            return new Pixel(
                        $pixel->r + $random,
                        $pixel->g + $random,
                        $pixel->b + $random
                    );
        }
    ?>

    What we have here is generation of a random value in a user-specified range and adding the random value to the pixel. User-specified range is a number between 0 and 255, where 0 means less noise and 255 means more noise. Well, 0 means no noise and 255 means a lot of noise, 255 is not a boundary, you can go above and the more you go the more you get noise only and pretty much nothing of the original image.

    Let's test! I have the simplest of HTML forms:

    <form method="get">
        <input name="image" />
        <input type="submit" />
    </form>

    I specify the image name in the form and submit.

    Then if something was submitted I create an object on the pixel operation class:

    <?php
    if (!empty($_GET['image'])) {
    
        $po =& new Image_PixelOperations();
    
    }
    ?>

    Then I display the original image, call the pixelOperation method and display the result:

    <?php
        echo 'Original: <br /><img src="'. $_GET['image'] .'" />';
        echo '<hr />';
        
        // noise
        $noise = 100;
        $po->pixelOperation($_GET['image'], 'result_noise.png', array($po, 'addNoise'), $noise);
        echo '<br />Add noise (factor '. $noise .'): <br /><img src="result_noise.png" />';
        echo '<hr />';
    
    ?>

    The result is:

    Adding more or less noise gives the following results where the first one has a noise factor of 20 and the second one, 500:

    and

    Brightness

    Next, adjusting brightness. This is nothing but adding the same integer to each channel of each and every pixel. If we add, we brighten the image, if we substract, we darken the image. The callback method is:

    <?php
        function adjustBrightness($pixel, $factor)
        {
    
            return new Pixel(
                        $pixel->r + $factor,
                        $pixel->g + $factor,
                        $pixel->b + $factor
                    );
        }
    ?>

    To test this, we do:

    <?php
        $brightness = 50;
        $po->pixelOperation($_GET['image'], 'result_bright.png', array($po, 'adjustBrightness'), $brightness);
        echo '<br />Brighten: <br /><img src="result_bright.png" />';
        $brightness = -50;
        $po->pixelOperation($_GET['image'], 'result_dark.png', array($po, 'adjustBrightness'), $brightness);
        echo '<br />Darken: <br /><img src="result_dark.png" />';
        echo '<hr />';
    ?>

    Which gives us:

    and

    Swap colors

    Next, colors swapping. This means for example take the amount of red in a pixel and replace it with the amount of blue in the same pixel. So there are these possibilities for swapping:

    • RGB to RBG
    • RGB to BGR
    • RGB to BRG
    • RGB to GBR
    • RGB to GRB

    The method definition is pretty simple:

    <?php
        function swapColors($pixel, $factor)
        {
    
            switch ($factor) {
                case 'rbg':
                    return new Pixel(
                                $pixel->r,
                                $pixel->b,
                                $pixel->g
                            );
                    break;
                case 'bgr':
                    return new Pixel(
                                $pixel->b,
                                $pixel->g,
                                $pixel->r
                            );
                    break;
                case 'brg':
                    return new Pixel(
                                $pixel->b,
                                $pixel->r,
                                $pixel->g
                            );
                    break;        
                case 'gbr':
                    return new Pixel(
                                $pixel->g,
                                $pixel->b,
                                $pixel->r
                            );
                    break;        
                case 'grb':
                    return new Pixel(
                                $pixel->g,
                                $pixel->r,
                                $pixel->b
                            );
                    break;        
                default:
                    return $pixel;
            }
    
        }
    
    ?>

    Testing the different possibilities gives us:
    RGB to RBG:

    RGB to BGR:

    RGB to BRG:

    RGB to GBR:

    RGB to GRB:

    Removing and boosting colors

    Next, we have two more methods. One that sets a channel to zero (for example "no red!") the other one maximizes a channel. Or two channels. So we have six options for each method:

    • Removing (or maximizing) red
    • Removing (or maximizing) green
    • Removing (or maximizing) blue
    • Removing (or maximizing) red and green at the same time
    • Removing (or maximizing) red and blue
    • Removing (or maximizing) green and blue

    (It doesn't make much sense to remove all three channels or to maximize them. Why?)

    The implementation:

    <?php
        function removeColor($pixel, $factor)
        {
    
            if ($factor == 'r' ) {
                $pixel->r = 0;
            }
            if ($factor == 'g' ) {
                $pixel->g = 0;
            }
            if ($factor == 'b' ) {
                $pixel->b = 0;
            }
            if ($factor == 'rb' || $factor == 'br') {
                $pixel->r = 0;
                $pixel->b = 0;
            }
            if ($factor == 'rg' || $factor == 'gr') {
                $pixel->r = 0;
                $pixel->g = 0;
            }
            if ($factor == 'bg' || $factor == 'gb') {
                $pixel->b = 0;
                $pixel->g = 0;
            }
    
            return $pixel;
        }
    
        function maxColor($pixel, $factor)
        {
    
            if ($factor == 'r' ) {
                $pixel->r = 255;
            }
            if ($factor == 'g' ) {
                $pixel->g = 255;
            }
            if ($factor == 'b' ) {
                $pixel->b = 255;
            }
            if ($factor == 'rb' || $factor == 'br') {
                $pixel->r = 255;
                $pixel->b = 255;
            }
            if ($factor == 'rg' || $factor == 'gr') {
                $pixel->r = 255;
                $pixel->g = 255;
            }
            if ($factor == 'bg' || $factor == 'gb') {
                $pixel->b = 255;
                $pixel->g = 255;
            }
    
            return $pixel;
        }
    
    ?>

    And the tests:
    Remove red:

    Remove green:

    Remove blue:

    Remove red and green:

    Remove green and blue:

    Remove red and blue:

    Maximize red:

    Maximize green:

    Maximize blue:

    Maximize red and green:

    Maximize green and blue:

    Maximize red and blue:

    Negative

    This one is pretty easy, negate the channel. The logic is - you have a lot of red? Nah, I'll use the opposite, less red.

    <?php
        function negative($pixel)
        {
            return new Pixel(
                        255 - $pixel->g,
                        255 - $pixel->r,
                        255 - $pixel->b
                    );
        }
    ?>

    The test gives us:

    Greyscale

    I don't know if you know it, but the gray is a color that has equal amounts of R, G and B. Darker shades of gray have a log of R, G and B, the lighter shades have less. To greyscale an image we take the average of the amounts of R, G and B and set the three channels to the average.

    <?php
        function greyscale($pixel)
        {
    
            $pixel_average = ($pixel->r + $pixel->g + $pixel->b) / 3;
    
            return new Pixel(
                        $pixel_average,
                        $pixel_average,
                        $pixel_average
                    );
        }
    ?>

    Test:

    Black and White

    Unlike greyscale that has shades, B&W has only two colors black (0, 0, 0) and white (255, 255, 255). We use a factor here to determine where the boundary would be, meaning what do you consider black and what white. The simplest logic is that we sum R+G+B and if it's closer to 255+255+255 than it is to 0 (0+0+0), then we call it white, otherwise it is black. Using a factor gives us some more flexibility of drawing the line between black and white (This reflects the subjectivity of real life ;) )

    <?php
        function blackAndWhite($pixel, $factor)
        {
            $pixel_total = ($pixel->r + $pixel->g + $pixel->b);
    
            if ($pixel_total > (((255 + $factor) / 2) * 3)) {
                // white
                $pixel->r = 255;
                $pixel->g = 255;
                $pixel->b = 255;
            } else {
                $pixel->r = 0;
                $pixel->g = 0;
                $pixel->b = 0;
            }
    
            return $pixel;
        }
    
    ?>

    Test with factor 20:

    Clip

    At this point I started hunting the web for ideas for more pixel manipulations (I'm still open, BTW, post a comment with anything you find). I found this clipping thing described somewhere. I'm not sure how useful it is (maybe I didn't get it right). It is in essence removing boundary values and replacing them with pure 0 or 255. So you have (5, 155, 250), this will become (0, 155, 255). Again, there is a factor to give you flexibility in drawing the line. I'm not convinced how useful this is, the only thing I can think of is decreased file size, because the new image uses less colors. Anyway, here's the implementation and the test.

    <?php
        function clip($pixel, $factor)
        {
            if ($pixel->r > 255 - $factor) {
                $pixel->r = 255;
            }
            if ($pixel->r < $factor) {
                $pixel->r = 0;
            }
            if ($pixel->g > 255 - $factor) {
                $pixel->g = 255;
            }
            if ($pixel->g < $factor) {
                $pixel->g = 0;
            }
            if ($pixel->b > 255 - $factor) {
                $pixel->b = 255;
            }
            if ($pixel->b < $factor) {
                $pixel->b = 0;
            }
    
            return $pixel;
        }
    ?>

    Clipping with factor 100:

    Adjusting contrast

    This is not a pure pixel operation because it takes into account all the pixels in an image in order to decide how to manipulate a given pixel. The contrast adjustment needs the so called average luminance. In order to calculate the average luminance, you need a formula, which I copied, so the implementation is:

    <?php
        function getAverageLuminance($image)
        {
    
            $luminance_running_sum = 0;
            
            $x_dimension = imagesx($image);
            $y_dimension = imagesy($image);
    
            for ($x = 0; $x < $x_dimension; $x++) {
                for ($y = 0; $y < $y_dimension; $y++) {
    
                    $rgb = imagecolorat($image, $x, $y);
                    $r = ($rgb >> 16) & 0xFF;
                    $g = ($rgb >> 8) & 0xFF;
                    $b = $rgb & 0xFF;
    
                    $luminance_running_sum += (0.30 * $r) + (0.59 * $g) + (0.11 * $b);
                    
                }
    
            }
           
            $total_pixels = $x_dimension * $y_dimension;
    
            return $luminance_running_sum / $total_pixels;
        }
    
    ?>

    The actual contrast callback is simple:

    <?php
        function contrast($pixel, $factor, $average_luminance)
        {
            
            return new Pixel(
                $pixel->r * $factor + (1 - $factor) * $average_luminance,
                $pixel->g * $factor + (1 - $factor) * $average_luminance,
                $pixel->b * $factor + (1 - $factor) * $average_luminance
                );
        }
    ?>

    Tests with decreased and increased contrast (factors 0.5 and 1.5):

    Salt and Pepper

    I got the idea from this page. It's basically sprinkling random white (salt) and black (pepper) pixels. The implementation:

    <?php
        function saltAndPepper($pixel, $factor)
        {
            
            $black = (int)($factor/2 + 1);
            $white = (int)($factor/2 - 1);
            
            $random = mt_rand(0, $factor);
            
            $new_channel = false;
            
            if ($random == $black) {
                $new_channel = 0;
            }
            if ($random == $white) {
                $new_channel = 255;
            }
            
            if (is_int($new_channel)) {
            
                return new Pixel($new_channel, $new_channel, $new_channel);
                
            } else {
                return $pixel;
            }
        }
    ?>

    Test with factor 20:

    Gamma correction

    <?php
        function gamma($pixel, $factor)
        {
            
            return new Pixel(
                    pow($pixel->r / 255, $factor) * 255,
                    pow($pixel->g / 255, $factor) * 255,
                    pow($pixel->b / 255, $factor) * 255
                );
        }
    ?>

    Test with factor 2.2:

    Randomize

    This is just picking a random function from the ones described above a calling it with a random factor. The result? Well, a scientific way to create worst (or better, it's all how you look at it) colorful noise ;)

    Next?

    I thought of two functions that might be useful. One of them is to snap a pixel to a predefined color. I mean if a pixel is close to something we want, make it closer. This might be useful when for example you want to change the color scheme of your website and you want to alter all images to make them pinkier for example. So far the experiments are going well, but the results ate not too promising ;)

    The other function would be to replace a color with another one, while also using proximity. For example take all very very light shades of gray and make them white.

    Then there are the filters to do, another set of operations which take into account not only the current pixel but also the neighbours. Examples - blur, edge detect, etc.

    BTW, the full source for this posting is here. Extending it is easy, you just need to define a callback function for a new effect and pass it in a call to pixelOperation. I'll be happy if you post the results here.

    -----------------

    Update form 2005-09-21:

    Thanks to Laurens Holst who posted this comment about the sensitivity of the human eye to the different colors! So I created a new greyscaling function using the formula he suggested. In my original function I just take the average of the R, G and B, while now these colors come each with a coeficient. Then I divide by the sum of the coeficients.

    The result is better!

    Here's the experiment with the same Wiki image (the second greyscale image is using the new formula)

    The new image looks just a bit brighter, nothing special. But I tried with another image I found on SitePoint and it looks like the second image is not only brighter but contains more human-eye-readable information. Take a lcoser look at what looks like a display on a cash register. The second greyscale image looks more "informative".

    Here's the source of the new greyscale function:

    <?php
    function greyscale2($pixel) 
    {
    
        $pixel_average = 
            ( 0.3  * $pixel->r 
            + 0.59 * $pixel->g 
            + 0.11 * $pixel->b) / (0.3 + 0.59 + 0.11);
    
        return new Pixel(
                    $pixel_average,
                    $pixel_average,
                    $pixel_average 
                );
    
    }
    ?>
     

    Proposal for PEAR::Structures_DataGrid column formatter

    Saturday, September 3rd, 2005

    This posting outlines a proposal for an addition to the PEAR package Structures_DataGrid. More specifically it concerns the formatter method of the Structures_DataGrid_Column class.

    First off, great package! Many thanks to the authors Andrew Nagy [Wishlist] and Olivier Guilyardi!

    I was trying to use the package to create a listing of database data (a list of users actually) , with a link to delete a record next to each data row. So I needed a custom column that doesn't exist in in the database and in order to create it I needed to use a callback function as a formatter of the column. Following the examples in the documentation I did:

    <?php
    $column =& new Structures_DataGrid_Column('&nbsp;');
    $column->formatter = 'getDeleteLink(link_text=Delete)';
    ?>

    This is a strange syntax to define a callback function, with the brackets and the way parameters are given. But I can live with it, having in mind all the benefits I gain from using the package.

    Next, I wanted the getDeleteLink function to be part of my User class, the same class that contained the listing() method, which is where I create an use the DataGrid object. So I tried something along the lines of

    <?php
    array($this, 'getDeleteLink')
    ?>

    This is the usual way to use the callback pseudo type in PHP when you've in a class whose method you want to call back. It didn't work, but I found another solution. If the callback method is static, I can use the syntax to call the method statically:

    <?php
    $column =& new Structures_DataGrid_Column('&nbsp;');
    $column->formatter = 'User::getDeleteLink(link_text=Delete)';
    ?>

    This is still not so intuitive way of defining a callback function but it worked and this is the important thing.

    OK, then for my one of the columns I wanted to call back htmlentities(), just in case someone really wants the username "I'm an <script>var XSSGuy;</script>" ;) I tried quite a few way to make it work, but I couldn't. In the end, the formatter() method always adds the current record as a parameter to the function being called back and htmlentities() won't accept an array. I guess I could wrap htmlentities() in another static method in my class and call the new method, but seemed like such a waste. In addition I saw a @todo comment that says the formatter() needs a revision, so I went ahead.

    What I ended up having is much shorter measured in lines of code that the original implementation. Also it is more flexible and intuitive, following the usual way a callback function is defined.

    Also with my implementation I could use the array($this, 'method') syntax.

    A step back. What are we looking for in the formatter method? It should give the ability to use:

    • A function with no parameters.
    • A function with one parameter which is the value of the current column of the data row (the most common use probably)
    • A function that accepts the whole data row (record) as a parameter
    • A function that accepts the whole data row as a parameter plus any number and type of other parameters

    OK, that's for the parameters. Now for the function itself. It could be:

    • A stand-alone, old-procedural-school function
    • A static method of a class
    • A non-static method of a class

    And finally we should be able to call the function inside and outside of a class.

    Follows a list of valid ways to define a formatter function with my implementation.

    <?php
    // This will call myFunction passing the
    // current column record as a parameter
    array('myFunction')
    // This is also the syntax to call a function
    // without any parameters.
    // It will happen in case the column is not mapped
    // to a table column
    // (the second parameter of the
    // Structures_DataGrid_Column constructor)
    // To be exact, this second use will call the
    // callback function with FALSE parameter
    // PHP allows passing parameters to functions
    // that don't expect any
    
    // This will pass the current record
    // as an array with key 'record'
    // The key 'record' is backwards-compatible
    // with the way it's currently working
    array('myFunction', true)
    
    
    // This allows for any number if parameters to be
    // passed in addition to the
    // current record, which is still
    // identified by its 'record' key
    array('myFunction',
            array(
                'param1' => 'value1',
                'param2' => 'value2'
            )
         )
    
    
    // For the call from within a class or
    // outside of it, this example shows all valid calls
    class myClass {
        
        function blah(){}
        
        function dg(){
            $column =& new Structures_DataGrid_Column('&nbsp;');
            
            // static
            $column->formatter(
                array(
                    array('myClass', 'blah')
                )
            );
            
            // non-static, using $this instance
            $column->formatter(
                array(
                    array($this, 'blah')
                )
            );
        
        }
    }
    
    // non-static using an instance of the class, outside of it
    $column =& new Structures_DataGrid_Column('&nbsp;');
    $myInstance = new MyClass();
    
    $column->formatter = array(array($myInstance, 'blah'));
    ?>

    The callback function is always defined as an array, this is in case the authors of the package are committed to the API and doesn't want to change it. Currently the formatter is a string, so a simple check is_array() can serve is a switch between the old and the new implementation.

    Finally, here's the actual implementation of the formatter() method. It may be disappointingly short, after this whole long posting ;)

    <?php
    class Structures_DataGrid_Column
    {
    // ...
        function formatter($record)
        {
            
            if (empty($this->formatter[0])) {
                return false;
            }
            
            $parameters = false;
            if (empty($this->formatter[1])) {
                if (!empty($this->fieldName)){
                    $parameters = $record[$this->fieldName];
                }
            } else {
                $parameters = array('record' => $record);
                if (!is_bool($this->formatter[1])) {
                    $parameters = array_merge(array('record' => $record), $this->formatter[1]);
                }
            }
            
            return call_user_func($this->formatter[0], $parameters);
        }
    
    ...
    }
    ?>