Preparing video from the Canon RF 5.2mm f/2.8L Dual Fisheye for screen displayWritten by Paul BourkeApril 2025
The following presents the two pipelines used for preparing video footage from the "Canon RF 5.2mm f/2.8L Dual Fisheye" for cylindrical and dome displays. The first display considered is a stereoscopic enabled 360 degree LED cylinder, 8m diameter and 4m high. The second display is a projection based truncated dome, 4m in diameter. A key difference between screen based stereoscopy and head mounted displays (so called, VR180) is dealing with the alignment of zero parallax. The examples here are captured in January 2025 from Budh Gaya, Bihar, India. It is known for being where Gautama Buddha attained enlightenment. The footage is a simple walk around the main Mahabodhi temple including under the famous the Bodhi Tree. Preparing frames for the cylinder
While a fisheye projection is a precise thing, a fisheye lens on a camera is not. The lenses are almost always positioned differently on the sensor, and they can have slightly different fields of view due to tolerances in fisheye lens manufacturing. Physical fisheye lenses can additionally have various mappings from radius on sensor to field angle, fortunately the Canon RF 5.2mm f/2.8L Dual Fisheye lenses are extremely linear so this correction isn't required. The first stage then is to deal with all the imperfections of the recorded fisheye frames. The approach here is to map the fisheye into half an equirectangular projection. This can be performed in (at least) two ways. The first is to remap each fisheye after deriving the relevant parameters. The author estimates these by thresholding a frame in order to derive fisheye center and radius. The remapping is performed with the fish2sphere utility, noting one could remap directly to a panorama using fish2pano. 
An alternative approach is to use the Canon EOS VR Utility which determines the parameters and creates a single image with the left and right halves of the equirectangular. 
The final step is to map the half equirectangular projections into a cylindrical panorama as expected for the cylindrical display. Since this is an LED based cylinder there are no addition mappings required, for example, there is no image warping and blending as required for a projector based system. The author uses sphere2pano. The Canon RF 5.2mm f/2.8L Dual Fisheye lenses are mounted parallel and, as expected, the EOS VR Utility preserves this for head mounted display viewing. For screen based viewing the zero parallax distance needs to be adjusted, typically to the radius of the cylinder. sphere2pano can perform this by rotating the equirectangular about the up vector, it also scales to the final native resolution of the display and crops the result to 170 degrees to remove the lenses that are in shot. For example the command line might as follows, the vertical field of view of 53.3158 corresponds to the physical cylindrical display used in this exercise. sphere2pano -w 12816 -sc 85 -180 -v 53.3158 -z 0.36 sph_left/$1.jpg sphere2pano -w 12816 -sc 85 -180 -v 53.3158 -z -0.36 sph_right/$1.jpg The resulting left and right eye cylindrical panoramas are as shown below. Noting that the native resolution of the 360 degree LED cylinder is 12816 pixels, the result here will only occupy 1/2 the frame, and therefore half the cylinder. 
The result on the cylinder. 
Preparing frames for the hemispherical dome
The pipeline for mapping the fisheye to an iDome is similar to the above except the final image needs to be a normalised fisheye rather than a cylindrical panorama. One approach would be to deal with all the parameters and map directly into the final normalised fisheye. The authors fishcorrect can apply this in one step, that is, extract the normalised fisheye, crop to 170 degrees, scale to the desired resolution and apply the zero parallax correction. Alternatively one might map into the two half equirectangulars and subsequently create the fisheye images from there. The author prefers this approach as it results in a perfect projection from which any subsequent mappings can be generated without worrying about imperfect recording details. 
The half equirectangular projections mapped to the left and right eye normalised fisheye projections are as shown below. These also have the zero parallax distance adjusted to the 2m radius of this iDome. The projector and fisheye lens being used has a vertical extent of 2400 pixels. sphere2fish -w 2400 -t 170 -180 -z 0.45 sph_left/$1.jpg sphere2fish -w 2400 -t 170 -180 -z -0.45 sph_right/$1.jpg
Note that there is still a warping stage to present the fisheye pairs correctly on the dome, this is because the projector and fisheye are positioned away from the center of the dome. This is outside the scope of this document and is discussed in more detail here for a single elevated fisheye projection system, see case 3. 
The footage used here formed a test of the suitability of the Canon R5 and the Canon RF 5.2mm f/2.8L Dual Fisheye to be used to capture footage for these two display environments. Some final notes ...
|
