Easy to use Javascript Compression with PHP

Here is a class that I wrote to compress JavaScript. I’m going to describe how I did it so that you can write your own, or modify what I’ve done easily. If there is any real interest in this, I’ll put it on GitHub.

First, lets talk about best practices. To speed up downloading and make your site more efficient, all the javascript for a site should be served in a single, compressed, static .JS file that is separate from your HTML. That way your users can quickly download the file once, the first time they visit the site. There are several reasons why you might resist this at first, such as the difficulty dealing with debugging and refactoring compressed code, as well as dealing with third party libraries. I’ll address these as I go through how the compressor works, and I believe this class has allowed me the best of both world.

Another issue is how to deal with page-level code. You may have code you only want to run on a specific page. I always use the page’s name as a class for the html element, for instance:

<html class="contact_us">

I then have one JS file for with all the code that is specific and unique for that site or project, which I usually name after the project, generally the client name, for instance “acme_motors.js”.

$(function(){
    if ( $("html").hasClass('contact_us') ) {
        //run code here specific to the contact_us page.
    } else if ( $("html").hasClass('home') {
        // and so forth...
    }
});

Sometimes, these blocks of code might have a few pieces that are dynamically constructed by the server at the time of request. Usually, you can narrow this down to a list of variables, assembled into an object, and pass that to the script using a function call in your html. Basically, you need to separate you’re static script as much as possible from the dynamic properties in order to optimize your site. If you’re using PHP, that function call in your file might look something like this:

<script type="text/javascript">// <![CDATA[
	shoppingCart.setup({
		items: <?=json_encode($cart->get_items()) ?>
		total: <?=$cart->get_total() ?>
	});
// ]]></script>

There is one other case where you might have inline JavaScript as event handlers written directly into your HTML elements. Such as:

<a onmouseover="myMouseOverEffect(0)" href="javascript:doAjaxRequest('myAjaxPage.php');">
 Click Here!
</a>

This is just wrong. It prevents you from being able to do multiple things with the event and it makes refactoring impossible. Never do this. When you see this replace it with correct event listeners that are attached from the Javascript. (Also, every time I see a link labeled “Click Here” it makes me cringe.)

Now, on to the the actual JS_Compressor class.
I like to keep all my scripts provided as separate, uncompressed files while in development, so I can easily find the line number and read the code that is causing an error from the debugging console. This is also important when it comes to updating libraries. I also wanted to be able to try out different compression schemes and bench test them against each other.

HTML5 Boilerplate recommends running an ANT build script at deployment, but that’s an extra step and it doesn’t allow for you to use different compression schemes. You are are pretty much stuck with YUI, which appears to be inferior. They recommend loading jQuery from the Google CDN, but that can be a loosing strategy once the rest of your code gets up to around 200K. I’ll describe why in my findings at the end of this article.

Good JavaScript compression has 5 parts:

  1. Concatenation:
    All the javascript files and libraries are added together into one .JS file to minimize server requests.
  2. Minification:
    All the unnecessary white space and comments are removed.
  3. Obfuscation:
    All the variable names are replaced with the shortest possible names.
  4. Caching:
    The compressed file is cached to speed access to it.
  5. Binary Compression:
    The file is served to the user using gzip or deflate.

Concatenation

The concatenation is pretty simple. We retrieve the contents of all the JS files and concatenate them together. In this case, $jsfiles is an array containing the paths to the JS files. There is one catch here. The last statement in a JS file does not necessarily have to have a semicolon to execute, so we append a semicolon. It needs to be on its own line, just incase the last line of the file has an inline comment, “// comment;”. Thankfully, placing multiple semicolons between statements doesn’t cause any error in JS.

foreach ($js_files AS $js_filename)
{
  $jscode = @file_get_contents($js_filename);
  if ($jscode !== false)
  {
    $js .= $jscode . "\n ; \n";
  } else
  {
    $err .= $js_filename . '\n';
  }
}

Minification and Obfuscation
Minification is simply removing all unnecessary whitespace this just makes the code look alot smaller, but generally doesn’t save much space.
Obfuscation is more complex, but in it’s simplest form it means shortening variable name as much as possible. There are several schemes out there. This class can currently use five of them. All of these schemes do both minification and obfuscation:

  • YUI Compressor (YUI)
    This is the only “safe” compressor, so it’s the default. If you are using external resources, such as pulling jQuery from Google’s CDN you will need to use this scheme. It’s the least efficient though.
  • Minify (minify)
    I have to say that Minify has the most sophisticated caching and binary compression scheme. However, I am not currently taking advantage of it at this time.
  • Google’s Closure compiler (google)
    I started this little project by reading a tutorial titled How to Roll Your Own JavaScript Compressor with PHP and the Closure Compiler. The conclusion was pretty ridiculous. There was no caching in place, client or server side. So, every time a user accessed the file, it would have to make an extra round through the compiler on a remote server. This could only have the resulted in increase load times.
  • Dean Edwards’ Packer (packer)
    In my real-world bench test, this scheme proved to be the most effective by far. It lacks some of the sophistication of the other schemes, but in terms of shrinking text file sizes, it’s the most effective.

(The names in parenthesis are the accepted by my classes set_scheme() method)

In order to test which scheme is the most effective, I created a test() method. This method generates a cached version using each scheme, checks the of the file, and then deletes the cached file. It returns an associative array of the file sizes in bytes keyed by the names of the schemes. Here is the output from my project:

with jQuery:
[YUI]      => 465234
[minify] => 465800
[google]   => 342325
[packer]   => 292261
[none]     => 599866

without jQuery (loading from CDN):
[YUI]      => 327216
[minify] => 326090
[google]   => 258084
[packer]   => 213908
[none]     => 351626

jQuery alone:
Array
(
    [YUI] => 138017
    [minify] => 139709
    [google] => 83575
    [packer] => 76544
    [none] => 248240
)

So, as you can see here, the YUI compressor is the most inferior scheme (besides not using compression at all), but YUI is required if your are loading jQuery externally. The jQuery 1.7.1 weighs in around 76K when packed. In my tests, Packer compressed JS to around 60% the size of what YUI would output. So, if you’re site just requires a few lines of jQuery code, go ahead and load it from the CDN, because most users will already have it cached, so it’s free. However, it’s worth testing once your local JS code gets to be about 100K – 200K. If it’s over 200K you can be pretty certain that it’ll be faster to just use packer and include your local copy of jQuery.

Caching

Caching is done by checking running an MD5 hash on a string containing the meta data (file size, date modified and file name) for all the files. At first I was just doing a hash on the full concatenated code, but having the server read all the files on every request to inefficient to me. So, now I do it this way which saves about 20 milliseconds, and I can sleep a little easier at night. This works pretty well for now, but I’d eventually like to use Memcache instead of file based caching.

foreach ($js_files AS $js_file)
{
	$stats = stat($js_file);
	$meta_data .= "$js_file : {$stats['size']}-{$stats['mtime']} \n";
}
$filename = md5($meta_data);
$file_path = $this->dest_path . $filename . '.js';

if ( ! file_exists($file_path) OR ! $this->cache)
{
 // Some Magic happens
  //output the cached file
  $handle = fopen($file_path, "w");
  fwrite($handle, $compressed_js);
  fclose($handle);
}

Client-side Caching

Nothing is more infuriating for a developer than unknowingly having headers set that are caching your code on your ISP or having to constantly clear your cache to see changes. I always implement client side caching in a way so that it will only be used on the production server. In my scheme, aggressive caching is only set on files being served from the compressed directory.

Binary Compression

Client side caching and binary compression is where you are going to make the most optimization. Currently I’m using the technique described by Patrick Lin, GZip files with .htaccess and PHP. It’s pretty simplistic, though. In the future, I’d like to implement a system that will inspect the request headers to see what compression would work best and use Deflate if possible.

Below is the full code for the JS_Compressor class. You’ll need the appropriate compressor libraries in the same directory (or another include path) as well. Here is where you can find them.


<?php

/*
Deveploper: Robert Williams <rob@simplyinteractive.com>
*
* This class compiles, compresses and caches all of the javascript files list
* in the $scripts array.
*
* Usage: near the top of the PHP template execute
*
* $js_compressor = new JS_Compressor();
*
* If the $compressing property is TRUE it compiles all of the files into one compressed file
* and caches it in $dest_path. If $compress is FALSE it just outputs all the
* files as normally as individual extrnal scripts.
*
* Make sure there as a folder located at the $dest_path ('js/compressed/' by default) in
* production, and that it is writable by Apache/PHP.
*
* Now, you can add new scripts by calling set_scripts()
* with an array of strings .
* paths should be relative to the requested document.
*
*
* Scripts can be added to this array for specific sections:
* $js_compressor->add_script('js/script_for_this_page.js');
*
* Optionally, set the scheme. Default is 'YUI' as only the YUI compressor provides
* "safe" linkage to external resources, such as jQuery loaded from the google CDN.
* You can alternatively use 'google' for Google Closure, 'minify', or 'packer'.
*
* Near the bottom of the template, just before your </body> tag, execute:
* $js_compressor->render();
* This will output the appropriate script tags, and compress and cache the
* javascript if neccessary.
*/

class JS_Compressor
{

private $scripts = array();
private $dest_path = 'js/compressed/';
private $scheme = 'YUI';
private $compressing = TRUE;
private $caching = TRUE;

/**
*
* @param array $scripts
* @return JS_Compressor
*/
public function set_scripts($scripts)
{
$this->scripts = $scripts;
return $this;
}

/**
*
* @param string $script
* @return JS_Compressor
*/
public function add_script($script)
{
$this->scripts .= $script;
return $this;
}

/**
*
* @return array of strings
*/
public function get_scripts()
{
return $this->scripts;
}

/**
*
* @param string $dest_path
* @return JS_Compressor
*/
public function set_dest_path($dest_path)
{
$this->dest_path = $dest_path;
return $this;
}

/**
*
* @param string $scheme
* @return JS_Compressor
*/
public function set_scheme($scheme)
{
$this->scheme = $scheme;
return $this;
}

/**
*
* @param bool $compressing
* @return JS_Compressor
*/
public function set_compressing($compressing)
{
$this->compressing = $compressing;
return $this;
}

/**
*
* @param bool $caching
* @return JS_Compressor
*/
public function set_caching($caching)
{
$this->caching = $caching;
return $this;
}

public function render()
{
if (!$this->compress)
{
foreach ($this->scripts AS $script)
{
echo "<script src=\"$script\"></script>\n";
}
return false;
} else
{
$js_files = $this->scripts;

// fetch JavaScript files
// First get and test against headers first.
foreach ($js_files AS $js_file)
{
$stats = stat($js_file);
$meta_data .= "$js_file : {$stats['size']}-{$stats['mtime']} \n";
}
$filename = md5($meta_data);
$file_path = $this->dest_path . $filename . '.js';

if (!file_exists($file_path) OR !$this->cache)
{
$full_js = '';     // code to compress
$compressed_js = ''; // compressed JS
$err = '';  // error string
foreach ($js_files AS $js_filename)
{
$js_code = @file_get_contents($js_filename);
if ($js_code !== false)
{
$full_js .= $js_code . "\n ; \n";
} else
{
$err .= $js_filename . '\n';
}
}
if ($err != '')
{
// error: missing files
echo '<script>throw "The following JavaScript files \n\
could not be read by JS Compressor:\n' . $err . ' ";</script>';
}
if ($full_js != '')
{
// compress
switch ($this->scheme)
{
case "google": $compressed_js = $this->google_compress($full_js);
break;
case 'minify':
include_once('classes/helpers/min/lib/JSMin.php');
$compressed_js = JSMin::minify($full_js);
break;
case 'packer':
include_once 'JavaScriptPacker.php';
$packer = new JavaScriptPacker($full_js, 'Normal', true, false);
$compressed_js = $packer->pack();
break;
case 'YUI':
include_once('YUICompressor.php');
Minify_YUICompressor::$jarFile = $_SERVER['DOCUMENT_ROOT'] . 'classes/helpers/yuicompressor-2.4.6/build/yuicompressor-2.4.6.jar';
Minify_YUICompressor::$tempDir = '/tmp';
$compressed_js = Minify_YUICompressor::minifyJs(
$full_js
, array('nomunge' => true, 'line-break' => 1000)
);
break;
default:
$compressed_js = $full_js;
break;
}
//output the cached file
$handle = fopen($file_path, "w");
fwrite($handle, $compressed_js);
fclose($handle);
}
}
// output single script link.
echo '<script defer src="' . $file_path . '"></script>';

return $file_path;
}
}

//end render();

/**
* Uses google closure's compression through a rest API.
* This includes obfuscation, which breaks some dependencies.
*
* @param string $js
* @return string
*/
public function google_compress($js)
{
// REST API arguments
$apiArgs = array(
'compilation_level' => 'ADVANCED_OPTIMIZATIONS',
'output_format' => 'text',
'output_info' => 'compiled_code'
);
$args = 'js_code=' . urlencode($js);
foreach ($apiArgs as $key => $value)
{
$args .= '&' . $key . '=' . urlencode($value);
}
// API call using cURL
$call = curl_init();
curl_setopt_array($call, array(
CURLOPT_URL =>
'http://closure-compiler.appspot.com/compile',
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => $args,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_HEADER => 0,
CURLOPT_FOLLOWLOCATION => 0
));
$jscomp = curl_exec($call);
curl_close($call);
return $jscomp;
}

/**
* The function provides a way for benchmark testing. *Spoiler alert* Packer
* is the smallest.
*
* @return array
*/
public function test()
{
$schemes = array(
'YUI',
'minify',
'google',
'packer',
'none'
);
$output = array();
// turn off cache checking.
$this->compress = true;
$this->cache = false;
// Consume data.
ob_start();
foreach ($schemes AS $scheme)
{
$this->scheme = $scheme;
$file = $this->render();
$output[$scheme] = filesize($file);
unlink($file);
}
ob_end_clean();
return $output;
}

}

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>