Fisheye lens correction

Written by Paul Bourke
November 2016


A normal lens has pincushion or barrel distortion which can be corrected for to give a perfect perspective projection, a pin-hole camera. This is called "rectification" and is often applied before any warp/stitch/blending is applied, for example, in panoramic photography.

In the same way there is a perfect circular fisheye projection, that is, one in which the distance r from the center of the fisheye circle is linearly proportional to the latitude of the corresponding 3D vector. Such a fisheye lens is often referred to as a "tru-theta" lens and while such lenses can and have been manufactured, in real life and for lower cost lenses the relationship is non-linear. The non-linearity normally occurs towards the periphery of the fisheye and results in a compression artefact.

In many cases the lens manufacturer can supply data for the curve relating "r", the distance on the sensor or fisheye circular image, to latitude of the 3D vector corresponding to that radius. In some cases for high grade lenses the manufacturer will supply curves for the particular lens in question, an acknowledgement that lenses can vary typically by 5% in the manufacturing process. In the situation where no such data is available one needs to construct a rig which when photographed will allow the angles to be measured.

The approach to correcting for the non-linear relationship is to fit a suitable polynomial to the data points relating "r" to latitude. A general function for the latitude φ might be

φ(r) = a0 + a1r + a2r2 + ... + airi + ... + anrn

Since the fisheye is assumed to be radially symmetric and r=0 is the center of the fisheye corresponding to a latitude of 0, a0 is zero. In practice the highest order polynomial needed for the fitting is n=3. So the polynomial for a least squares fit is:

φ(r) = a1r + a2r2 + a3r3

The null case, where the fisheye already has a linear relationship would be a1 = φmax / rmax, a2 = 0, and a3 = 0. Where φmax is the half the circular fisheye angle and rmax the radius on the image or sensor corresponding to φmax. Noting that the r is often normalised to 1, in which case rmax = 1.

As an example, the following data relating distance on the sensor to latitude is illustrated below, black curve, for a 196 degree fisheye. The blue line shows the tru-theta relationship for a true fisheye, and the relationship after the correction is applied.

a1r + b2r2 + c3r3

Usage

Usage: fishcorrect [options] tgafile
Options
   -w n      sets the output image size, default: same as input fisheye
   -a n      sets antialiasing level, default: 2
   -s n      fisheye aperture (degrees), default: 180
   -c x y    fisheye center, default: center of image
   -r n      fisheye radius, default: half the fisheye image width
   -z n      roll angle, default: 0
   -p n n n  third order correction terms
   -d        debug mode

Example

The following image is captured with a 210 degree fisheye which quite significant compression towards the rim.

The following is the undistorted version, that is, radius on the fisheye images is now proportional to latitude.