Fisheye lens correctionWritten by Paul Bourke
Original November 2016, data being updated regularly
See also: fisheyerectify.pdf (March 2018)
The source code implementing the projections below is only available on request for a small fee. It includes a demo application and an invitation to convert an image of your choice to verify the code does what you seek. For more information please contact the author.
A normal lens has pincushion or barrel distortion which can be corrected for to give a perfect perspective projection, a pin-hole camera. This is called "rectification" and is often applied before any warp/stitch/blending is applied, for example, in panoramic photography.
In the same way there is a perfect circular fisheye projection, that is, one in which the distance r from the center of the fisheye circle is linearly proportional to the latitude of the corresponding 3D vector. In the following, along any radial line on the fisheye (left), two points separated by the same pixel distance, correspond to the same angular distance (right).
Such a fisheye lens is often referred to as a "tru-theta" lens and while such lenses can and have been manufactured, in real life and for lower cost lenses the relationship is non-linear. The non-linearity normally occurs towards the periphery of the fisheye and results in a compression artefact.
In many cases the lens manufacturer can supply data for the curve relating "r", the distance on the sensor or fisheye circular image, to latitude of the 3D vector corresponding to that radius. In some cases for high grade lenses the manufacturer will supply curves for the particular lens in question, an acknowledgement that lenses can vary typically by 5% in the manufacturing process. In the situation where no such data is available one needs to construct a rig which when photographed will allow the angles to be measured.
The approach to correcting for the non-linear relationship is to fit a suitable polynomial to the data points relating "r" to latitude. A general function for the latitude φ might be
Since the fisheye is assumed to be radially symmetric and r=0 is the center of the fisheye corresponding to a latitude of 0, a0 is zero. In practice the highest order polynomial needed for the fitting is n=3, in some extreme examples a 4th order (n=4) is required. So the polynomial for a least squares fit is:
The null case, where the fisheye already has a linear relationship would be a1 = φmax / rmax, a2 = 0, and a3 = 0. Where φmax is the half the circular fisheye angle and rmax the radius on the image or sensor corresponding to φmax. Noting that the r is often normalised to 1, in which case rmax = 1.
As an example, the following data relating distance on the sensor to latitude is illustrated below, black curve, for a 196 degree fisheye. The blue line shows the tru-theta relationship for a true fisheye, and the relationship after the correction is applied. The horizontal axis is field of view (field angle) in degrees and the vertical axis is normalised fisheye image coordinates.
Usage: fishcorrect [options] imagefile Options -w n sets the output fisheye width, default: same as source fisheye -a n sets antialiasing level, default: 2 -s n source fisheye field of view (degrees), default: 180 -t n output fisheye field of view (degrees), default: same as source -c x y source fisheye center, default: center of image -r n source fisheye radius, default: half the fisheye image width -x n tilt angle (degrees), default: 0 -y n roll angle (degrees), default: 0 -z n pan angle (degrees), default: 0 -ox n fisheye offset amount in x axis, default: 0 -oy n fisheye offset amount in y axis, default: 0 -oz n fisheye offset amount in z axis, default: 0 -p n n n n fourth order correction terms, default: no correction -o s output filename, default: derived from input file -d debug mode
The following image is captured with a 210 degree fisheye which has quite significant compression towards the rim.
The following is the undistorted version, that is, radius on the fisheye images is now proportional to latitude.
It was found that more lenses than expected did indeed require a 4th order correction and for numerical reasons it was better to map from radians rather than degrees. All the following examples follow this convention.
Entaniya M12 250 degree fisheye
This lens curve seems to need a 4'th order term in order to get a good fit at the extreme field of view angle of 125 degrees. Graph is radians horizontally, and normalised fisheye image (sensor) vertically.
Entaniya HAL 250 (3.0) degree fisheye
Entaniya M12 220 degree fisheye
Entaniya HAL 200 5.0 fisheye
Entaniya M12 280 degree fisheye
Entaniya RP L220 220 degree fisheye
Sigma f2.5, 4.5mm, 180 degrees
Sigma seems to be one of those companies who refuse to provide linearity data on their lenses. Totally unreasonable, this is a fundamental characteristic of a lens that an owner has every right to insist upon. Fortunately one can determine it with only a modest effort. The equipment required is shown below, the procedure is as follows
Sigma f3.5, 8mm, 180 degrees
Canon 8-15mm, 8mm and 12mm position
Canon is another company who behave badly and don't supply this fundamental information of their products. It is more complicated with this lens since the field of view and the curves change with different zoom values. So there is actually a whole family of curves required in order to deal with any particular zoom value, alternatively one could create a 3D surface fit. This could be automated given that the focal length is supplied in the exif data on the photograph. Two curves are provided, based upon the usual use of the lens by the author: fully zoomed out (8mm) and 12mm is the maximum zoom to fill full frame Canon5D Mk III sensor horizontally.
While this distortion can be corrected for, it should be noted that the more curvature the less effective resolution one achieves for the widest angles. For many applications this is exactly where one does want resolution, for example, for feature point detection between multiple camera rigs. One should also be aware that if you need 220 degrees then for highly curved functions the relatively small number of pixels for those last 10 or 20 degrees may effectivey mean you only really have a useful lower FOV lens. This can work in ones favour, for example if you only need a 180 degree lens then a 220 fisheye that is linear up to 180 may be a better choice than a 180 degree lens with curvature.Canon RF 5.2mm f/2.8L Dual Fisheye
Perhaps the most linear fisheye I've seen. Field angle equation is simply phi = r / (0.5*190*pi/180) = 0.6031 r where r is normalised radius of the fisheye circle.
iZugar MKX22, 220 degrees
iZugar MKX200, 200 degrees
iZugar MKX13, 185 degrees
iZugar MKX19, 190 degrees
Meike 6.5mm, 190 degree
DZO, VRCA 220 degree
Sunex miniature DSL315, 190 degree
Sunex miniature DSL239, 185 degree
Sunex miniature DSL415, 195 degree
Sunex Superfisheye DSLR01, 185 degree
Lensagon BF10M14522S118, 190 degrees
Lensagon BF16M220D, 220 degrees
Nikkor 8mm f2.8, 180 degrees
Omnitech ORIFL190-3, 190 degrees
Aico ACHIR01028B10M, 245 degree
Aico ACHIR01420B9M, 185 degree
Raynox DCR-CF187, 180 degrees
Laowa 4mm, 210 degrees
SMTEC SL-190, 180 degrees
Evetar E3307, 195 degrees
Evetar E3267A, 250 degrees
Evetar E3279, 190 degrees
ArduCam part number M25156H18, SKU number LN031, 180 degrees
Bosch Flexidome IP Panoramic 7000 MP, 180 degrees