Omni Directional Stereoscopic Panorama using discrete images

Introduction

The following discusses the generation of ODSP (Omni Directional Stereoscopic Panoramas) using a small discrete set of photographs, and introduces a test camera rig for the same. An ODSP consists of a pair (left and right eye) of cylindrical (or equirectangular) panoramas where depth perception is valid at any horizontal viewing angle, specifically, when viewed in a 360 degree stereoscopic environment (such as a cylindrical display) the stereoscopic depth is valid for multiple participants each looking in different directions. Alternatively, when viewed in a head mounted display, the stereoscopic depth can be presented simply by extracting the correct perspective view given the direction the viewer is looking in.

The ideal/correct method for capturing an ODSP image pair is with techniques similar to a slit scan camera. The ultimate ODSP camera was built by Seitz, it consisted of two lenses and two rolls of film that were exposed continuously as the camera rotated. The shutter remained open during the rotation and the exposure time was controlled simply by varying the rate of rotation. This can be approximated by a pair of digital video cameras and taking a small but finite number of vertical slits from each frame of the movie as the camera rotates.

Seitz Roundshot camera

Digital slit scan rig

An earlier discussion of this and an experimental digital camera rig can be found here: Capturing high resolution stereoscopic panorama images. A variation using a single camera is discussed here: Capturing high resolution stereoscopic panorama images with a single camera. The main issues with the digital method are:

• The panorama quality is compromised by the need to operate the camera in video mode. The lossy compression results in a poorer quality than would be expected given the underlying resolution of the camera sensors.

• The camera + lens needs to have a maximum height approaching the human interocular distance of 65mm. Highly capable cameras are not typically designed for this, more compact cameras target markets without the resolution and performance requirements required.

• In order to attempt to capture modest movement in the scene being captured, one would like to rotate the cameras relatively fast. In order to limit motion blur, high frame rates are necessary. At the time of the first exploration the frame rate was limited to 25fps, for a 6K pixel high frame, this was required to match the resolution of the displays the content was being captured for. The second experimental rig supported 60fps but for rotation speeds required to deal with people walking around, frame rates in the order of 200fps are required. At the time of writing there do not appear to be cameras that can record at the required resolution and frame rate, while maintaining a body size close to human interocular distance.

Discrete solution

In the following, a discrete solution is built and tested. Instead of using video and a slit scan methodology, a smaller number of photographs are taken and stitched together. The exact number of photographs depends on the lens being used which depends on the required vertical field of view (for a cylindrical display), as well as the overlap required for stitching.

It should be noted that there are stereoscopic enabled 360 video cameras on the market, most with 6 or 8 lenses. Unfortunately, while they can be used to create ODSP still panoramas, their panorama image resolution of 8 to 12K is insufficient for the displays being targeted here. In the authors experience, the stitching is rarely satisfactory due to a combination of factors: low number of discrete camera locations per eye, the fisheye optics and the relatively high departure from zero parallax positioning of the lenses.

The advantages of the discrete approach are:

• The images are based on photographs resulting in significantly higher resolution than slit scan video, and the native camera raw images can be used.

• HDR (multiple exposures) is an option although it requires the scene it to totally stationary.

• Editing panoramas produced in this way can be easier than with slit scan. This is partly due to masking that can be applied to the photographs during the stitching process. Note though that is still primarily a technique for stationary scenes.

For best results this approach still requires a largely stationary scene, but some movement can be tolerated. Movement in the center of the frame is captured at identical times and, in the experience during testing, is rarely problematic. Movement towards the edges of the frames can often be removed with masking of the corresponding imagery in the adjacent frame. There are various settings and tools in the stitching software used (PtGui) to solve the issue of modest motion. Other motion related errors can often be repaired in PhotoShop, or similar image editing tools. Fast moving objects that can appear in multiple frames are more problematic but that has been a limiting factor to date with all ODSP rotating rigs.

The main risk with this approach arises from the two panoramas being stitching independently, so there is no assurance they will stitch identically. That is, to obtain a stitch, there may be local distortions applied differently between the left and right eye pair. These distortions may result in localised parallax separation of objects between the image pairs resulting in depth perception errors.

For this approach the desirable characteristics for candidate cameras and lens are:

• Small camera height. The cameras are typically mounted in portrait mode and the height of the body determines the interocular. The target for the real world scale scenes is 65mm. High quality full frame sensor cameras generally have large bodies.

• The same applies to lenses, diameters also targeted at 65mm, or at least no wider than the camera height.

• The cameras need to be triggered simultaneously. Perfect synchronisation isn't strictly required but will assist with any post production removal of moving objects.

• Lens that meets the vertical field of view of the final display system. Typically one wants to shoot slightly wider to give some flexibility for vertical cropping, but not so wide as to be wasteful of the available pixel resolution.

• Horizontal sensor resolution needs to exceed the vertical resolution of the intended display (camera mounted in portrait mode). In reality there are benefits to capturing double the final resolution to assist with sharpening and noise suppression when panoramas are scaled down to the target resolution.

Specifications of the test camera rig (August 2026)

• 75mm interocular. While 65mm would be desirable, given the intended real world scale content, this was the smallest compact camera found that met the other criteria. In practice, feature poor objects closer than 2m from the camera could have stitching issues.

• The camera sensor is 6000 pixels high by 4000 wide, this results in cylindrical panorama widths of 27,000 pixels. The intended display environment resolution range from 13K to 20K pixels around the 360 degrees.

• 16mm lens (full frame), 24mm for (1.5 crop) APS-C sensor, corresponding to about 70 degrees vertically. The highest cylindrical panorama display the content will be presented on is 53 degrees (8m diameter, 4m high).

• For this lens the horizontal field of view is around 50 degrees. For a 40% overlap the photographs were captured every 30 degrees, 12 steps around the 360 degrees.

• By the very nature of the ODSP, the cameras are not rotated about an axis through the zero parallax of the lens. However the cameras are offset such that the zero parallax offset is only along one axis, that is, the minimum distance of 37.5mm off the ideal axis. The practical upshot of this is stitching errors may occur for scene objects with few feature points that appear less than 2m from the cameras. Given the limits of comfortably stereoscopic viewing, this is not a significant limitation.

• The cameras are triggered simultenaously from the motorised rotator using splitter electronics, includes diode isolation between cameras.

Some of the examples captured as part of the testing process are presented in this document in top-bottom format, left eye on the top and right eye on the bottom. They are designed for screen presentation and as such the zero parallax point has been aligned to the radius of the cylindrical display (4m).