Generating thumbnail images: AffineTransformOp gotchas
AffineTransformOp is a Java 2D filter that can scale (resize), rotate, and/or skew an image. Since I was generating thumbnails, I was only interested in its scaling ability.
To scale an image using AffineTransformOp, all you need to do is generate a new AffineTransform using the static method AffineTransform.getScaleInstance, then use it to construct an AffineTransformOp. When constructing an AffineTransformOp, you also need to pass a constant theat identifies the type of interpolation to use, either nearest neighbor, bilinear, or bicubic. Once you have the AffineTransformOp, you just call the filter method, passing in the original image and the destination image, which can be null, in which case AffineTransformOp will create a destination image for you.
After generating my first thumbnail image and saving it as a JPEG, I was able to view my generated thumbnail image in Preview, but it looked like crap. It looked blocky, as if AffineTransformOp was using the nearest neighbor interpolation method even though I had specified bicubic. I switched to nearest neighbor just to see if there was a difference, and there was--bicubic was definitely better--but the thumbnail looked bad.
Prior to now, I was generating thumbnail images in photoSIG using the Image.getScaledImage method. I looked for information on getScaledImage versus AffineTransformOp and found an article called The Perils of Image.getScaledImage that provided the answer. Bicubic interpolation doesn't work well when the scaling factor is very small, as it is when scaling from a 1,000-pixel-wide original image to a 125-pixel-wide thumbnail. The solution, according to the article, is to scale the image in steps, with each step scaling the image by no more than one-half.
I updated my thumbnail-generation code to use the multi-step method. I first created a AffineTransform to half the size of the image, specifying a scaling factor of 0.5 for both the height and width. As long as the width and/or height of the image was more than double the size of the thumbnail that I wanted to create, I applied the halving filter. Then, once I was within a factor of two of my desired thumbnail size, I created and applied a final, custom AffineTransform.
This seemed to work well enough, and the quality of my thumbnails was definitely better. But while it worked well for some images, others had a one-pixel-wide black line on either the bottom or the right side. Actually, I had noticed similar black lines on some of the other thumbnails that I had generated with the single AffineTransform, but I was so busy trying to figure out why the thumbnails looked like hell that I didn't pay much attention to them.
The black lines frustrated me for a while, and I started to think unflattering thoughts about Java 2D in general. Why can't Sun's stuff just work? But it turned out that Sun's stuff was working all too well. The answer came to me while I was in the bathroom at work. I don't know why I was thinking about it then, but I've often found that when I'm dealing with a vexing problem, my mind continues to work on it at while I'm doing other things, and eventually the answer comes to me. I almost always figure things out in the end. But sometimes it takes a while.
The black lines were caused by Java 2D trying to handle my request to scale the image by exactly 0.5--even when the width or hight of the image was odd. If the original image is 640x480, then no problem, AffineTransformOp returns a 320x240 image. But what if the image is 640x427? AffineTransformOp can't create a BufferedImage that's 213.5 pixels high, so it generates one that's 214 pixels high. I don't know exactly how it handles the bottom row, but I imagine that it uses the alpha channel to give the appearance of an image that is taller than 213 pixels but not quite 214 pixels. Then, when I convert the image to TYPE_INT_RGB, the bottom row of pixels becomes black.
I fixed the code so it generated a new AffineTransform each time through the halving loop with the exact scaling factors needed to make the resulting image an integer number pixels wide and high (which would have been 0.5, 0.49882903981 in the above case).
That fixed most of the problem, but strangely, I was still getting a black line every now and then. I finally figured out that these occasional black lines were due to my using floats to calculate the scaling factors rather than doubles. Why was I using floats? Because the Kernel used by ConvolveOp uses floats, and so I had gotten used to casting non-integer values to floats. Oops.
After doing all this work, I wound up getting rid of AffineTransformOp completely. What did I use instead? Graphics2D.drawImage. By passing a width and height to drawImage, I can make Java 2D scale the image without having to bother with AffineTransformOp. I can draw directly to an TYPE_INT_RGB image (instead of having to convert the TYPE_INT_ARGB I get from AffineTransformOp), and since I specify the target width and height directly, there are no scaling factors to miscalculate. If I'd used drawImage right from the start, I would have saved myself a lot of time. But then again, I wouldn't have learned as much!
To scale an image using AffineTransformOp, all you need to do is generate a new AffineTransform using the static method AffineTransform.getScaleInstance, then use it to construct an AffineTransformOp. When constructing an AffineTransformOp, you also need to pass a constant theat identifies the type of interpolation to use, either nearest neighbor, bilinear, or bicubic. Once you have the AffineTransformOp, you just call the filter method, passing in the original image and the destination image, which can be null, in which case AffineTransformOp will create a destination image for you.
After generating my first thumbnail image and saving it as a JPEG, I was able to view my generated thumbnail image in Preview, but it looked like crap. It looked blocky, as if AffineTransformOp was using the nearest neighbor interpolation method even though I had specified bicubic. I switched to nearest neighbor just to see if there was a difference, and there was--bicubic was definitely better--but the thumbnail looked bad.
Prior to now, I was generating thumbnail images in photoSIG using the Image.getScaledImage method. I looked for information on getScaledImage versus AffineTransformOp and found an article called The Perils of Image.getScaledImage that provided the answer. Bicubic interpolation doesn't work well when the scaling factor is very small, as it is when scaling from a 1,000-pixel-wide original image to a 125-pixel-wide thumbnail. The solution, according to the article, is to scale the image in steps, with each step scaling the image by no more than one-half.
I updated my thumbnail-generation code to use the multi-step method. I first created a AffineTransform to half the size of the image, specifying a scaling factor of 0.5 for both the height and width. As long as the width and/or height of the image was more than double the size of the thumbnail that I wanted to create, I applied the halving filter. Then, once I was within a factor of two of my desired thumbnail size, I created and applied a final, custom AffineTransform.
This seemed to work well enough, and the quality of my thumbnails was definitely better. But while it worked well for some images, others had a one-pixel-wide black line on either the bottom or the right side. Actually, I had noticed similar black lines on some of the other thumbnails that I had generated with the single AffineTransform, but I was so busy trying to figure out why the thumbnails looked like hell that I didn't pay much attention to them.
The black lines frustrated me for a while, and I started to think unflattering thoughts about Java 2D in general. Why can't Sun's stuff just work? But it turned out that Sun's stuff was working all too well. The answer came to me while I was in the bathroom at work. I don't know why I was thinking about it then, but I've often found that when I'm dealing with a vexing problem, my mind continues to work on it at while I'm doing other things, and eventually the answer comes to me. I almost always figure things out in the end. But sometimes it takes a while.
The black lines were caused by Java 2D trying to handle my request to scale the image by exactly 0.5--even when the width or hight of the image was odd. If the original image is 640x480, then no problem, AffineTransformOp returns a 320x240 image. But what if the image is 640x427? AffineTransformOp can't create a BufferedImage that's 213.5 pixels high, so it generates one that's 214 pixels high. I don't know exactly how it handles the bottom row, but I imagine that it uses the alpha channel to give the appearance of an image that is taller than 213 pixels but not quite 214 pixels. Then, when I convert the image to TYPE_INT_RGB, the bottom row of pixels becomes black.
I fixed the code so it generated a new AffineTransform each time through the halving loop with the exact scaling factors needed to make the resulting image an integer number pixels wide and high (which would have been 0.5, 0.49882903981 in the above case).
That fixed most of the problem, but strangely, I was still getting a black line every now and then. I finally figured out that these occasional black lines were due to my using floats to calculate the scaling factors rather than doubles. Why was I using floats? Because the Kernel used by ConvolveOp uses floats, and so I had gotten used to casting non-integer values to floats. Oops.
After doing all this work, I wound up getting rid of AffineTransformOp completely. What did I use instead? Graphics2D.drawImage. By passing a width and height to drawImage, I can make Java 2D scale the image without having to bother with AffineTransformOp. I can draw directly to an TYPE_INT_RGB image (instead of having to convert the TYPE_INT_ARGB I get from AffineTransformOp), and since I specify the target width and height directly, there are no scaling factors to miscalculate. If I'd used drawImage right from the start, I would have saved myself a lot of time. But then again, I wouldn't have learned as much!
Comments