Using QuickTime to Present Stereoscopic Movies

Written by Paul Bourke
December 2003


Introduction

Playing back precomputed stereoscopic movies has generally required a custom written playback system, this was particularly so when the predominant stereoscopic format was frame sequential. A solution written by the author is described below, it works well although since it only employs relatively simple compression methods (including uncompressed), it requires fast disk IO for XGA size (1024x768) frame movies at 25fps. This movie player has recently been expanded to support 6 channel audio and was the movie player used for the opening shows of the VROOM exhibit.

What one would really like is to base stereoscopic playback on some standard. A number of compression codecs are required so that a range of content types can be compressed without undue degradation. For example, MPEG derived compression is not generally suited to high resolution computer generated content in visualisation applications. While a standard solution was never on the cards for frame sequential stereo, the increasingly popular passive approach is easier. The reason for this is passive stereo doesn't need special syncing on each left/right eye image. Passive stereoscopic hardware generally employs a dual display card, the left eye image is on the left display and the right eye image is on the right display. There are (at least) two approaches to playing back stereoscopic movies on a passive system, the first is have two separate movies, one on each display. The problem with this is synchronising the two movies to 1/60 of a second say (authors estimate for stressless stereoscopic viewing). A much easier approach, and the one discussed here, is to make a single movie that has the left eye on the left half and right eye on the right half of each frame. There are still a few details, the stereo pairs still need to be centered on each display. The solution is to play the movies full screen and generally create the frames so they are at the same resolution as the dual display. For example, an XGA size movie would now consist of frames that are 2048x768 pixels.

Left eye, 1024x768
Right eye, 1024x768
QuickTime

There a number of movie playback options for passive stereoscopic movies but the most standard is the QuickTime technology and player from Apple Computer. However, the QuickTime codecs have been primarily designed to create very small movie files, this is to support distribution over the internet and also to allow movies to playback from relatively slow consumer hard disks. The result is that QuickTime (up to recently) wasn't particularly suited to playing movies with large frame sizes, the decompression algorithms just weren't fast enough. This was particularly so for computer generated movies with fine structure where the encoding quality needs to be kept high. Indeed the only codec that could manage 2048x768 (at the time of writing) at 25fps was the graphics codec and that codec isn't suited to many types of content, for example, those with a wide range of colours.

PIXLET codec

Enter PIXLET, this codec was released in the last quarter of 2003 and is designed specifically for large frame and high quality movies. In all but it's highest quality modes it can readily playback 2048x768 movies with high quality audio on a dual processor G5. While intended for film style movies, it seems to provide sufficiently high quality results when applied to sharp computer generated animations.

Full Screen

The last remaining detail is a deficiency in the QuickTime player from Apple, namely that it cannot play full screen across two displays. Unbelievable but true! Since new workstation style Macs are all delivered with cards capable of dual displays one can only hope that this limitation will soon be relaxed. There are other reasons for wanting full screen movies across two displays and that is for driving a two projectors for a double width display. Note that the fullscreen option from in the Apple player is only available in the registered version. This has currently been solved by the local development of a custom QuickTime player.


Update and summary: April 2007

There are three (standard) ways of creating stereo images, these are independent of the actual presentation mode whether it is a monitor, active shutter glasses, passive polaroid, HMD (not discussed here), or Infitec.

I have used all 3 and the software I write supports them all. (2) is not available on the Mac, it is supported on MSWindows and Linux only. Only (1) works with monitors, Mac Pro supports this with the nVidia FX4500 card.

Discussion

Codecs

At the time of writing it would seem the Pixlet codec has not lived up to the hype. Currently the optimal codecs (at least those preferred by the author) for large frame size and high quality movies are PhotoJPEG, MotionJPEG, or the Apple Intermediate codec.

There seems to be a high performance hit on some combinations of hardware, OS, and graphics card when movies hit the 2048 pixel wide barrier when played with the QT Pro player. This seems not to occur with ATI cards, a comprehensive study of what combinations of nVidia cards and Apple hardware has not been performed. An option to solve this is to align the left and right eye views to and bottom, a custom movie player is required to split this across the two displays.

A useful trick is to retain the left and right eye movies as two separate QT streams. A third reference movie can be created with both streams and the left eye offset by the width of the movies. Saving this as a reference movie gives flexibility between left-right or top-bottom orientation. ps: the offset is supported in the QT Pro version of the player. This is particularly interesting for large movies played on a twin raid disk where the left and right eye segments are split across the disks.

Update: March 2008

My recommended side-by-side passive stereoscopic movie playback is now the warpplayer application. It can handle all stereo mode combinations, across single or dual displays, and movies that are side-by-side or arrange top-bottom (this often gives better decompression performance.




OpenGL Stereo-Pair Movie Player

Written by Paul Bourke
1999/2000


Introduction

The goal was to be able to present stereo-pair movies at 20+ frames per second. These movies would either be created by rendering packages (precomputed) or filmed using two digital cameras mounted on a custom built rigging. The playback hardware environment included a Dec Alpha (Tur64 Unix) with OpenGL hardware (Powerstorm 4D51T and 350 card), an NEC projector with fast green phosphor (for frame sequential stereo at 120Hz), and a 5+ meter wide projection screen in a room with seating for 30 adults.

Software Overview

Two pieces of software were written, a movie encoder and a movie player. The encoder takes a collection image files (all the same pixel dimensions and colour depth) and creates a single movie file, optionally applying various image filters. The player takes this movie file and presents it as fast as possible up to a maximum of 30 frames per second. The player makes standard calls to OpenGL and the associated glut library.

Encoder - buildmovie
Usage: buildmovie -x width -y height [-dout format] [-din depth] [-e eyes] [-flip] [-cmap filename] [-
Where  -dout is the format of the output movie frames
             It is one of  2 (RLE)
                           8 (Grey)
                           9 (YCC compressed, frame order)
                          90 (YCC compressed, row order)
                          16 (6,5,5)
                          24 (8,8,8) [default]
       -din  is the depth of the input image
             It is one of  8 (grey)
                          24 (r,g,b) [default]
       -o    Specify output file otherwise stdout
       -flip Flips the image vertically
       -cmap Use a colour map, only applicable to 8 bit input
             Expects a file name with ascii colour index + rgb
       -bgr  Interprets colour as (b,g,r) instead of (r,g,b)
       -e    Make a mono (1) or stereo movie (2) [Default: 1]
       -g    Does simple gamma correction [Default: 1]
             Expects a floating point number. Slow!
       -q n  Quantises the input image to every nth level
       -d    For test mode
The "filename" is a list of image files in either raw,
ppm, tga, tif, rle, or mtv format. tga is most supported.
A single column file for -e 1, double column file for -e 2

buildmovie expects a text file containing an ordered list of image file names. This file has a single column if buildmovie is building a non-stereo movie, it has two columns for a stereo movie, the columns contain the left and right eye image file names. buildmovie knows about a number of output formats, the most commonly used is the YCC format that encodes 24 bit colour frames in what amounts to 9 bits per pixel. For more information on this compression mode see this. Other compression modes can readily be implemented by adding the encoder to buildmovie and the decoder to playmovie.

Player - playmovie

Usage: playmovie [command line options] moviefile
Command line options
                        -h      this text
                        -s      active stereo mode
                        -ss     dual screen stereo
                        -i      show frame information
                        -f      full screen mode
                        -b      byte swap for other endian machines
                        -m      hide mouse
                        -a      start in play mode
                      -l n      loop type, 0 = rotate, 1 = circular, 
                                           2 = stop at end, 3 = exit at end
                    -fps n      target frames per second
Key Strokes and Menus:
                         g      toggle autoplay
                         l      toggle looping mode
                         m      toggle mouse display
                      h,f1      toggle help screen
                      i,f2      toggle info screen
                     q,esc      quit
         left,right arrows      backward, forward frame
                       -,+      set play direction
                         1      first frame

The sole purpose of playmovie is to present movies (stereo or not) to the user. The computer being used to play the movie is expected to have a reasonably high performing OpenGL hardware option. It is possible to play stereo movies in non-stereo, in which case the left eye image is used. In full screen the movie area is centered on the screen and the rest of the screen is blacked out.

Performance

There are a number of factors that determine how fast movies played this way can be presented. They are discussed briefly below, all numbers quoted relate to the PowerStorm 350 card in a Dec Alpha XP1000 running at 667 MHz, all the images were 800x600 pixels.

Latest modifications