YUI Compressor is a popular tool that allows web developers to easily and reliably minify / munge Javascript and CSS files. The one common complaint amongst users is that it doesn’t allow you to combine files into a consolidated package prior to minification. As I discussed a while back, JAWR takes care of this automatically for Java based web apps but automated solutions aren’t very efficient for small, script-based web apps. Hence, I wrote a small shell script that will combined a list of files into one and then run them through the yuicompressor.

Now, before you run the script, you either need to move the yuicompressor.jar to /usr/share/yui/yuicompressor.jar or just edit the first line of the script to point to the location of your yuicompressor jar file.

yuicompress.sh

# yuicompress
#
# combine files and then compress the result with yuicompressor
# note: yuicompressor jar must be at /usr/share/yui/yuicompressor.jar
#
# usage: yuicompress [-o outfile] [-t type] args

# check for yui compressor -----------------------------------------------------
yuipath="/usr/share/yui/yuicompressor.jar"
if ! test -f $yuipath
then
        echo "cannot find $yuipath"
        exit
fi

# get options ------------------------------------------------------------------
ofile="compressed.txt"
type="js"
preserveLicense=false
while getopts 'o:t:' OPTION
do
        case $OPTION in
        o)      ofile="$OPTARG"
                ;;
        t)      type="$OPTARG"
                ;;
        ?)      printf "Usage: %s: [-o outfile] [-t type] args\n" $(basename $0) >&2
                exit 2
                ;;
        esac
done
shift $(($OPTIND - 1))

# get files --------------------------------------------------------------------
if (( $# < 1 ))
then
        echo "no files specified"
        exit
fi
files=$*

# combine files ----------------------------------------------------------------
cat $files > $ofile

# compress results -------------------------------------------------------------
java -jar $yuipath --type $type -o $ofile $ofile

The usage is illustrated in the script itself but here are a few examples just to get you started:

./yuicompress -o lib.js lib/js/*.js
./yuicompress -o all.css -t css css/*.css
./yuicompress -o some.css -t css css/main.css css/misc.css

Downloaded the latest version of Maven and don’t know what to do with all of those files? Run this shell script (with sudo for the appropriate privileges) on the directory containing all of the maven files (usually called apache-maven-[version#]):

mvnupdate.sh

#check for directory argument --------------------------------------------------
if [ $# -lt 1 ]
then
        echo "no directory name specified"
        exit
fi
#check directory exists --------------------------------------------------------
if ! test -d $1
then
        echo "$1 is not a directory"
        exit
fi
#move previous version ---------------------------------------------------------
MVN="/usr/share/maven"
oldMVN=$MVN`date +"%Y%m%d%H%M%S"`
if test -d $MVN
then
        sudo mv $MVN $oldMVN
fi
#move new version --------------------------------------------------------------
sudo cp -r $1 $MVN

You know those ugly outlines that Firefox puts around links and buttons when you click on them? Yeah, I don’t get it either. From a developer’s standpoint, IE is by far the worst browser to support but Firefox also has its share of baffling behaviors and deviations from the standards.

So, how do you remove those ugly outlines from your site? Add this to your style sheet and it should do the trick.

*, *:hover, *:active, *:link { outline: none; }

It might be overkill but it takes care of most situations involving links. Unfortunately there’s no easy way to get rid of the outlines around buttons that are styled with images but this at least minimizes the number of visual abominations.

Recently I gave a presentation to colleagues on Jawr and the general merits of Javascript/CSS minification. Most of the material was pilfered from the official Jawr site but it may be easier to digest for those of us with shorter attention spans.

So, first an introduction:

Jawr

“Jawr is a tunable packaging solution for Javascript and CSS which allows for rapid development of resources in separate module files. Developers can work with a large set of split javascript files in development mode, then Jawr bundles all together into one or several files in a configurable way.”

And why should we care about any of this? Before answering this question, let's step back for a moment and discuss an important concept in web application development:

Minification

Minification (n), also called minimization, or minimisation (for those who speake the Queen's English): the process of removing all unnecessary characters from code without altering its functionality. This typically includes all non-critical whitespace and newlines that are not included in string literals and most comments.

Verb form: minify.

Why minify (and bundle)?

• Improves load times for the client

• Less data being transmitted by the server

How?

• Minification reduces the size of HTTP responses.

Why should we be sending unnecessary data? The spaces, tabs, and newlines are valuable only in development so we should keep them to ourselves.

• Bundling reduces the number of HTTP requests.

The HTTP protocol is fairly inefficient when it comes to retrieving lots of files. First a TCP connection must be established. This requires a three-way handshake. The client sends a connection request, the server responds, and then the client acknowledges the response.

Now the client sends the HTTP request itself. HTTP, Hypertext Transfer Protocol, is, as the name indicates, a text based protocol with all information being sent as human readable text. This means that the speed, small size, and efficiency of a rigid bit and byte based based protocol are sacrificed in favor of versatility and comprehensibility.

I'm not going to get into the details of the TCP/IP stack and talk about maximum transmission units or fragmentation but, for those of you unfamiliar with the specifics, I'll just stress that there's even more overhead in the transmission of these bloated requests and responses in addition to the initial handshake and large, text -based headers.

The response, with a larger set of headers, comes back from the server and, if this is an HTML document, the client begins sending requests for the other documents referenced therein. This includes scripts, style sheets, and pictures. All browsers have a limit on the number of concurrent connections allowed to a single domain per page and for a while this was a whopping 2 (this is still effectively the case for IE). Additionally, unless the Keep Alive flag is set, each request requires a new TCP connection to be negotiated.

Multiple script files, which halt the execution of the page due to the assumed requirement that they be loaded and executed in order, will make initial page load times uncomfortably long unless the files are cached. Therefore, reducing the number of files that need to be downloaded means fewer HTTP requests resulting in faster load times.

And why should we care about load times?

• Web applications and the culture of instant gratification

If we, as web application developers, truly want to compete with desktop applications then we strive to match their responsiveness. This is why the whole AJAX paradigm has become such a phenomenon (and poorly misused buzzword) over these past four years. When we perform an action within the context of a web page, we don't want to wait ages while the screen reloads. Even in cases where a full page transition is necessary, slow load times can mean the death of a project where alternatives are available.

• Mobile browsers

My iPhone is faster and has more storage space than the iMac I bought back in 1999 and a much better downstream than the 56K modem bundled within. It weighs as much as my wallet and enables me to check my email from places where laptops are socially forbidden. Mobile browsing is becoming ever more prevalent and it would be a serious mistake on the part of any web application developer to neglect making optimizations for their sake.

• Some people still use dial-up

…and some people still use AOL… and a lot of people still use IE. Strange, I know, particularly the last point, but whatever the reason for the circumstances may be, they may not have a choice but we certainly do have the choice to optimize.

So why don't we just use one big script file and one big style file?

• Development is much more efficient when using multiple, logically separated files and good organization is integral to good programming.

Editing or debugging large script files is a nightmare. Editing or debugging large, minified script files is impossible.

• Some components need to be included on every page whereas other components or groups of components need to be included in particular sections of the site.

Optimization doesn't mean simply getting rid of meaningless text, it also means excluding useless code. If I have an admin section of a site and a client section of a site, there is no need whatsoever for scripts specific to the admin to be sent to the client and vice-versa. In the case of styles, there could be CSS conflicts between these two sections so we certainly wouldn't want to bundle section specific stylesheets together.

Other options

• YUI compressor (no grouping) (http://developer.yahoo.com/yui/compressor/) Java app, minify and compress.
• Minify (PHP) http://code.google.com/p/minify/

Jawr

• Minifies
• Munges
• Bundles
• Compresses
• Enforces Caching
• Does all this on demand
• Has debug and production modes
• Built in JSP tag library for including the scripts and styles

Additionally

• Free and open source
• Easy to set up
• Easy to configure
• Plays nice with others (Spring MVC, JSF/Facelets, Grails, pure HTML)

Setup (web.xml)

Setup (jawr.properties)

Setup (JSP)

Result (debug)

Result (production)

File Sizes

	 16532 autocomplete.js
	  1176 binder.js
	   367 console-noop.js
	  1617 effects-ext.js
	 38986 effects.js
	  1543 form-highlighter.js
	 11809 global.js
	   822 modalbox-ext.js
	 22778 modalbox.js
	  2625 prototype-ext.js
	124000 prototype.js

	All files
		244 KB

	core.js (minified and bundled)
		156 KB

	core.js (minified, bundled, and gzipped)
		40 KB

Load times (in debug mode)

Load times (in production mode)

Load times (with caching)

Trying to import a load of UTF8 encoded data into a database or send it across a socket using PHP? Do you need to alter the data first using the built in string manipulation functions?

Functions such as utf8_encode won”t help you if the data you have is already utf8 encoded. For an isolated set of cases, the Multibyte String Functions will allow you to safely manipulate the encoded data but they must be explicitly called. This means another set of functions to memorize or, at the very least, look up repeatedly at php.net because, as can be expected, there is no direct mapping between the regular string functions and the multibyte string functions.

So, what if you want to use those regular, well-memorized string functions instead? Include the following and you should be set:

mb_internal_encoding("UTF-8");

Installing the MySQL ruby gem can be quite a pain but, by following these few steps in order, you might be fortunate enough to bypass the frustrating problems.

1. Install the os x developer tools
2. Install the 32 bit edition of MySQL.

3. sudo gem update --system

4. sudo env ARCHFLAGS="-arch i386" gem install mysql --
   		--with-mysql-config=/usr/local/mysql/bin/mysql_config

I am an avid Mac user, that’s no secret. I also like to make my daily life more efficient by scripting repetitive tasks. Again, no secret. As such, I have a complicated relationship with Apple’s Automator. With the ability to execute tasks with a single right/ctrl-click and some minor navigation through a contextual menu, there’s no denying the power and utility of this feature. I’ve compiled many Automator actions to take care of oft-repeated tasks but have occasionally spent entirely too long trying to debug a workflow that should be simple but, given the nature of the limited / limiting drag and drop interface, turned out to be annoyingly complex.

Anyone who had dealt with assigning and using multiple variables in a workflow knows how incredibly frustrating a task it can be. Now, I don’t read documentation much. I’m often far too eager to get to work to bother with a manual but perhaps I would have seen this gem earlier if I hadn’t rushed into the world of Automator unprepared:

You can run shell scripts in Automator, that’s all you need to know. Select some files/directories in the Finder, ctrl-click, select the script, and off you go. Sure beats dealing with paths, setting up aliases, and typing out long arguments in the command-line.

Following the Apache setup for OS X 10.5 in the previous post, one may want to activate the already-present php installation.

cd /private/etc/apache2
sudo pico httpd.conf

Now, uncomment the following line:

#LoadModule php5_module      libexec/apache2/libphp5.so

LoadModule php5_module      libexec/apache2/libphp5.so

Now we need to make sure the php.ini file is present by copying the php.ini.default file.

cd /private/etc
sudo cp php.ini.default php.ini

You can also use this shell script to accomplish the task:

activatephp.sh

#!/bin/bash
#-activatephp.sh-#
#httpd.conf --------------------------------------------------------------------
HTTPD="/private/etc/apache2/httpd.conf"
sudo cp $HTTPD "$HTTPD.old"

PHPMOD="LoadModule php5_module"
sudo sed s/"#$PHPMOD"/"$PHPMOD"/g "$HTTPD.old" > $HTTPD

#php.ini -----------------------------------------------------------------------
PHPINI="/private/etc/php.ini"
sudo cp "$PHPINI.default" $PHPINI

Copy the text into a file called activatephp.sh and run it as follows:

sudo ./activatephp.sh

Alter the php.ini file if needed or copy your own pre-configured php.ini file into /private/etc instead.

Restart Apache through the Sharing preference pane (again, to avoid accidentally starting conflicting instances).

The modern day Mac OS comes with a plethora of great tools pre-installed and (typically) pre-configured to run. From an SVN client to a GCC compiler to Ruby on Rails, there are precious few necessities a typical developer won’t find tucked away in the Terminal upon first boot.

Apache has been bundled with OS X since 10.0 and there’s even a way to enable it using the System Preferences through the Sharing preference pane… or at least that’s how it should work.

Recently I needed to enable a web server on a new MacBook Pro. Apparently checking the Web Sharing box in the Sharing preference pane isn’t enough. Now I think I ran into this problem long ago but I forgot the steps involved. So, for my own records at the very least, here is how you enable apache in OS X (10.5):

NOTE: I’m comfortable with pico as my quick text-editing terminal app. Use whatever you prefer.

For all accounts:

cd /private/etc/apache2/users
sudo pico local.conf

<Directory "/Users/*/Sites/">
	Options Indexes MultiViews
	AllowOverride None
	Order allow,deny
	Allow from all
</Directory>

For a specific user (called USER in this example) account:

cd /private/etc/apache2/users
sudo pico USER.conf

<Directory "/Users/USER/Sites/">
	Options Indexes MultiViews
	AllowOverride None
	Order allow,deny
	Allow from all
</Directory>

Here’s a shell script I wrote to take care of it:

enableapache.sh

#!/bin/bash
#-enableapache.sh-#
if [ $# -gt 0 ]
then
        FILE="$1.conf"
        USER=$1
else
        FILE="local.conf"
        USER="*"
fi

sudo echo "<Directory \"/Users/$USER/Sites/\">
        Options Indexes MultiViews
        AllowOverride None
        Order allow,deny
        Allow from all
</Directory>" > "/private/etc/apache2/users/$FILE"

Copy the text into a file called enableapache.sh and run it as follows:

sudo ./enableapache.sh		(for all accounts)
sudo ./enableapache.sh USER	(for a specific user called USER)

Now restart Apache using the Sharing preference pane. Doing so through the command line could result in two instances of Apache trying to run simultaneously.

After running into some issues configuring MySQL in a scriptable fashion recently and finding little documentation to support my needs online, I am bound by compulsion to share my solutions in order to help any subsequently frustrated developers.

Changing the MySQL root password from a blank / empty string using one single command line command:

mysqladmin -uroot --password="" password "myNewSecurePassword"

Granting global remote root access using one single command line command (this will work for any account specified after the -u flag):

mysql -uroot -pmyNewSecurePassword -e "GRANT ALL PRIVILEGES ON *.*
 TO 'root'@'%' IDENTIFIED BY 'password'"

NOTE: To restrict the access to a certain IP address, simply replace the % with the IP address. To restrict access to a range of IP addresses / subnet, replace the variable portion of the IP address with the % character. For example, ‘192.168.1.%’ will allow any IP from 192.168.1.1 to 192.168.1.254. The same can be done with domain names or named addresses. Examples:
‘user’@'mydomain.com’
‘user’@'%.mydomain.com’
‘user’@'myserver.mydomain.com’

Actually enabling the remote access on your server after you’ve discovered that you can’t even access it via telnet for some puzzling reason:

Find your my.cnf file. Add / edit the following line:

bind-address=YOUR_IP_OR_HOSTNAME

Make sure you can ping whatever you use for YOUR_IP_OR_HOSTNAME can be successfully pinged from the local machine. Using the loopback address will not work.

Also, if this line exists:

skip-networking

…then comment it out:

#skip-networking

Anything beyond that and you’re on your own.