# Comparative Study

*Comparative study of WebP, JPEG and JPEG 2000, September 2010*

**Note:** We had conducted this initial study with the first version of WebP
that we released, which used the `libvpx` library. In addition, the source
images were JPEG images that we crawled from the Web. Based on feedback from
the community, we have now undertaken a
[new study](https://developers.google.com/speed/webp/docs/webp_study) with version 0.1.2 of `libwebp`
starting with lossless images (PNG) as source. The new study supersedes this
study.

The WebP team performed a comparative study of three image compression methods
over a random set of 1 million images. The goals of the study were to measure
the compression achieved by those methods and to analyze the trade-offs
between image sizes and compression.

Three compression methods, WebP, JPEG 2000 and Re-JPEG, were applied to the
900,000 JPEG images contained in the data set. JPEG images were re-compressed
with Re-JPEG so that each was as close as possible to a target peak
signal-to-noise ratio (PSNR) value. The team's analysis was done on the images
after they had been compressed.

In addition to the large scale study, the team manually inspected 100 random
images and found that, in most cases, WebP provides acceptable visual quality.

## Conclusions

WebP achieves overall higher compression than either JPEG or JPEG 2000. Gains
in file size minimization are particularly high for smaller images which are
the most common ones found on the web.

## Data Set

The data set used in the study was a collection of 1 million images randomly
sampled from a repository of images crawled from the web. The following table
shows the distribution of different types of images in the data set.

| Type | Count |
|---|---|
| JPEG | \~900K |
| PNG | \~47K |
| GIF | \~47K |
| Other | \~6K |

## Experiment Design

The experiments were run in two stages:

1. The team ran WebP on the images with a target PSNR of 42. They tuned the
   quality parameter (QP) for the WebP encoder until they achieved a result
   that was as close to 42 as they could get for a given image. The actual
   PSNR achieved for each image, therefore, was not necessarily exactly 42.

2. The team used the PSNR value for the WebP-compressed images as the target
   PSNR for the JPEG 2000 and Re-JPEG images. They also performed a line
   search over the quality parameters of the JPEG 2000 and JPEG images in
   order to come as close as possible to the target PSNR.

The actual compression result was measured as a percentage using the following
formula:

    compression_percentage = 100 * (original_image.length - compressed_image.length) / original_image.length

## Results

The tests yielded the following results for negative and non-negative
compression gain:

### Non-negative Compression Gain

The team did not penalize a method if that method increased the image size
after compression. That is, the compression_percentage was considered to be 0
when compressed_image.length was greater than the original_image.length.

### Negative Compression Gain

The team allowed negative compression percentage results to get a better
understanding of the various compressors' raw performance.

Following are the average compression percentages that were achieved by each
of the three methods.

| Type | Avg PSNR Obtained | Avg Compression % (non-negative compression gain) | Avg Compression % (negative compression gain) |
|---|---|---|---|
| WebP | 39.38 | 41.30 | 39.80 |
| JPEG 2000 | 39.49 | 27.67 | 9.71 |
| Re-JPEG | 39.36 | 22.37 | 14.62 |

When negative compression gain was allowed, the average performance for JPEG
2000 was lower than that of JPEG. This may be due to high-frequency artifacts
present in the JPEG images; these artifacts can contribute to a
less-than-ideal compression situation.

The Kakadu implementation used in the experiment to convert to JPEG 2000
failed to convert approximately 240,000 images.

JPEG 2000 had lower compression percentages than Re-JPEG, primarily because
the JPEG 2000 result was negative compression for many images. For that
reason, JPEG images achieved a lower overall compression score.

The following figures provide distribution graphs for the three methods:

**Figure 1: Image Size Population Distribution**

![](https://developers.google.com/static/speed/webp/images/Plot1_image_sizes.png)

The graph shows that most of the images were small. The frequency of the
images with a size more than 500K is generally less than 100.

**Figure 2: Compression Percentages for WebP, JPEG, and JPEG 2000 Images**

![](https://developers.google.com/static/speed/webp/images/Plot2_vp8compr.png)

This graph demonstrates that the body of pictures compressed with WebP got
better compression than re-jpeg and jpeg 2000. Also, the pictures compressed
with WebP got more positive compression than the others.

**Figure 3: Comparison of Image Sizes and Compression Percentage**

![](https://developers.google.com/static/speed/webp/images/Plot3_cdfcompr.png)

This graph shows that WebP achieves better compression than the other formats,
particularly for smaller images.