On adding alpha channels to JPEG images

This requires formatting beyond my depository's capabilities. So it can live here in its own little page. If you feel compelled to comment you might do so back at the depository.

Summary

It is possible to embed an alpha channel in a JPEG in a way that modern browsers can render. On this page I show how, document the limitations, give you the code, and propose a standardized format.

Origin

On October 5th, thegrossman posted JPEGs with Alpha Channels?!? on Hacker News. This article talked about using CANVAS tags to composite JPEG images and PNG masks. In the HN comments relix suggested embedded the PNG inside the JPEG in the JPEG's metadata to make a monolithic file, then using Javascript to extract it into a data URI inside the browser, but declined to spend the "hour or two" to whip it up since he was going on a trip.

I have spent his "hour or two", though I decline to divulge the constant to convert relix hours to jws hours.

The JPEG Image Compression FAQ has this to say on the matter:

[12] Can I make a transparent JPEG?

No. JPEG does not support transparency and is not likely to do so any time soon. … The only real solution is to combine lossy JPEG storage of the image with lossless storage of a transparency mask using some other algorithm. Developing, standardizing, and popularizing a file format capable of doing that is not a small task. As far as I know, no serious work is being done on it; transparency doesn't seem worth that much effort.

Someone should think about updating that.

Solution

APP7 alpha0

The JPEG image format makes provision for application specific data elements. These are used to store EXIF information, color space data, and all sorts of vendor specific camera data. There are 16 available APPx markers and no apparent mechanism for allocating or arbitrating their use. I determined the APP7 was not widely used and hereby claim it, when the data begins with "alpha0" for embedding simple alpha channels in JPEG images. "alpha1" to "alpha9" can be used for successive refinements.

A valid alpha0 JPEG will have at least two APP7 markers begining with "alpha0". One will be the "content-type" and will be of a form similar to "alpha0content-type:image/png". This is case sensitive. It represents the MIME type of the mask.

The other required marker is an APP7 beginning "alpha0data:" followed by the binary representation of the mask image. Note that this limits mask images to 65523 bytes, but in practical terms browsers have smaller limits on data URIs, so a mechanism for concatenating chunks to form larger images is not a part of alpha0.

jpegapp

Not finding a suitable option, I wrote jpegapp to allow me to manipulate the APPx segments of JPEG images. It is 300 lines of C and no libraries so it should build easily just about anywhere.

$ jpegapp -h
	  Usage: jpegapp [options] [input file]
	  e.g.: jpegapp -i 7=alpha0content-type:image/png -i 7@mask.embed image.jpg

	  Options:
	  -i | --insert seg@file   insert the contents of file as APPx where x=seg
	       --insert seg=string insert the argument as APPx where x=seg
	  -r | --remove seg        remove all APPx segments where x=seg
	  -o | --output file       send output to file instead of stdout
	  -v | --verbose           Print a lot of debug messages
	  -h | --help              This help message

To generate the example on this page I use this Makefile. For any large scale use, I think keeping the source images as PNGs and generating the JPEG would be best, but I just need this spaceman for now. (Notice I create two APP7 segments, one using the literal notation for the content-type, and one using the file insertion notation for the data.)

all : astronaut-256-embed.jpg

	%.embed : %.png
	( echo -n "alpha0data:" ; cat $< ) > $@ 

	  astronaut-256-embed.jpg : astronaut-256-whitematte.jpg
          astronaut-256-mask.embed ../jpegapp -v -o $@ \
          -i 7=alpha0content-type:image/png \
          -i 7@astronaut-256-mask.embed \
          astronaut-256-whitematte.jpg

Source Code

You have already downloaded the Javascript involved with these HTML examples. The CSS is not important, other than to notice that I am driving image selection by class and data-* attributes on the IMG tags. You can also download the source for jpegapp (BSD license). 300+ lines, the bulk of which is silly stuff like option parsing and error handling. No special libraries are required.

Making Your Own Images

If you don't feel like compiling the source code (hint: make jpegapp, no makefile required) I slapped up a horrible web page to let you use my server to create your JPEG files. See: http://jim.studt.net/jpeg-alpha/jam.php.

Limitations

"Best" Practice

If deploying this, I recommend:

Future Directions

Going forward, people should consider doing the following:

Examples

Filesize: 42k GETs: 1

Regular old JPEG with a background. Quality set to 95, but since we downsampled a lot the smoothness still gives a small file.

Filesize: 90k GETs: 1

But we really want a transparent background, like this PNG. 90k seems a bit big.


Filesize: 13k GETs: 1

Here we have the alpha channel from the masked PNG extracted as its own little image. We are going to apply this to the JPEG file two different ways.

Filesize: 42k + 13k = 55k GETs: 2

We use some javascript to replace the IMG tag with a CANVAS in which we have composited our JPEG and the mask image. This falls back to an unclipped image, but we could also arrange for it to fall back to the larger PNG.

Filesize: 42k + 13k = 55k GETs: 2

Here we make use of webkit's mask-image CSS to do the work. Other browsers see the full image.


Filesize: 46k GETs: 1

Here the magic begins. We've take the orginal JPEG and whited out all the pixels we don't intend to see. We've also taken the black mask image (of which the alpha is important, not the black) and stuffed it inside an application specific data segment of the JPEG file. It is still a 100% legal JPEG file usable by all applications.

Filesize: 46k GETs: 1

Our same magic JPEG, but now we have asked Javascript to extract the mask image from the application specific data of the JPEG and replace the IMG with a CANVAS containing the composite image.

Filesize: 46k GETs: 1

Our same magic JPEG, but now we have asked Javascript to extract the mask image from the application specific data of the JPEG and use it as a webkit CSS mask.


Filesize: 42k + 21k = 63k GETs: 2

This is a quality control test. In this the mask image is red to make sure that our mask isn't bleeding into resulting image. A faint red halo here would be bad. Looks good for me in Safari.

Filesize: 42k + 7k = 50k GETs: 2

This is a visual alternative test. In this the mask image is binary, either transparent or opaque. It cuts the size of the mask about in half, but our space man now has sharp edges. Depending on your wishes binary masks can be a good idea. (Especially noticable in the cable coming out of his left ear in case you doubt your eyes.) Also note that with this masking and the embeded JPEG with white matted pixels, the composite image is actually smaller than the original.