Sony ARW distortion correction

2018-05-09 Permalink

Raw output of Sony cameras is saved in its proprietary ARW format. It is a TIFF-based format that stores a lot of undocumented and partially encrypted metadata. Some have already tried to reverse engineer the format[1][2], but most of it still remains unknown.

Here I summarize my findings of how the distortion correction is applied. The following is my best guesses based on the ILCE-7RM2 I have. The results were also checked against ILCE-7M2 and ZV-1[3] cameras.

Cropping

The raster in the ARW is bigger (8000×5320) than the in-camera JPEG (7952×5304). One source of discrepancy is the presence of an overscan along the right edge; these pixels simply repeat the rightmost Bayer cell multiple times:

Cropping the overscan yields a 7968×5320 raster. This corresponds to the Sony­Image­Width­Max and Sony­Image­Height­Max tags reported by ExifTool.

At this stage the radial corrections are applied as described below, with the optical center located at the center of this image (i.e. (3984, 2660) for my camera).

After the image is demosaiced, corrected and enhanced, the camera crops 7952×5304 pixels from the center to produce the final JPEG. This corresponds to Sony­Image­Width and Sony­Image­Height, whose values are also duplicated in the EXIF tags. Some cameras have explicit DefaultCropOrigin and DefaultCropSize that allow off-center cropping.

Undistortion

The distortion correction coefficients are stored in the Distortion­Corr­Params tag (in both SR2 directory and the RAW directory) as a sequence of 16-bit signed integers. The first in the sequence is the number of the following valid coefficients. Unlike some software that uses polynomial coefficients to describe lens distortion, Sony’s coefficients describe knots of a spline instead. For example:

DistortionCorrParams: 16 -5 -1 4 11 18 26 35 43 49 54 55 51 40 21 -11 -58

Graphing them gives a curve like this:

The knots of the spline are spread evenly between the center and the corner of the frame. Let C denote the center and r_max = ∥C∥ the distance to the corner.

The mapping from the corrected point to the distorted (captured) frame is given by the function:

where

f(r) is the spline defined by DistortionCorrParams, and n is the number of its knots.

Chromatic aberration

The chromatic aberration (CA) coefficients are stored in the Chromatic­Aberration­Corr­Params tag:

ChromaticAberrationCorrParams: 32 384 384 384 384 384 256 256 256 256 256 128 128 128 128 0 0 -256 -256 -256 -256 -256 -256 -256 -256 -256 -256 -128 -128 -128 -128 -128 0

Just like with the distortion curve, the first number in the sequence is the number of the following valid coefficients. However, this time there are two curves encoded within, for the red and blue corrections respectively. Their graphs look like:

The mapping is done in a manner similar to the distortion correction, however this time the multiplier is different and there’s no radial scaling factor (s=1):

I assume that CA is applied before the lens distortion, possibly prior to demosaicing.

APS-C mode

In the APS-C mode the camera reads a smaller portion of the sensor. Nevertheless, the process of cropping the overscan and undistorting is identical, except that there are less coefficients describing the splines. Spline knots are still distributed uniformly from the center to the corner of the cropped frame.

Same applies to cameras with smaller sensors.

Results

Each panel below contains the same cropped patches obtained through four different methods. Each column corresponds to a different radius from the center of the frame.

Conclusion

The process described above gives good results, but it does not exactly match the in-camera JPEGs. There is still a visible radial offset in some regions. It might be attributed to different spline interpolation methods, different r_max factors, different order of operations, or other unknown factors.

Footnotes

1. Dave Coffin of dcraw figured out the decryption of the SR2 subdirectory.
2. See Summary: Sony’s ARW (v2.3.1) embedded lens-correction data by variousphotography for previous lens-correction reverse engineering attempts.
3. Special thanks to Jim Solomon for providing ZV-1 ARWs.