Warp mesh patch for Quartz Composer
Intended primarily for
fulldome projection using a spherical mirror
Also used as a navigable movie player
Modelled after the warpplayer application
Written by Christopher Wright @ Kineme
Design by Paul Bourke
The Quartz Composer patch discussed here warps an image (or movie) in an arbitrary way
as specified by a user generated mesh stored in a simple text file. There are many
potential applications for this but the motivation for this capability arose from a
very precise requirement, namely the warping of images for projection into immersive
spaces by using a projector and a spherical mirror.
This document assumes the reader is familiar with the topics listed above, namely the
use of a spherical mirror as a low cost single projector solution for immersive displays.
And that this technique requires image warping such that the result appears correct on
the final surface. Additionally, the concept of navigable movies ... movies where each
frame consists of more than a limited perspective projection and the viewer may change
his/her view while the movie plays.
In all the above applications an image is warped (distorted) using a regular mesh
consisting of vertex coordinates (x,y), texture coordinates (u,v) and an intensity
mapping (i). The patch described here adds this warping support to Quartz Composer and
is a direct analog to the existing stand alone application "warpplayer"
(it uses the exact same warp mesh file format) except that
it adds the enormous additional functionality of Quartz Composer to create more
dynamic/interactive experiences. The rest of this document will concentrate on applications
intended by the original designer of the patch, every attempt has been made to keep it
general so that hopefully other uses will rise.
Description of inputs
Mesh file: file name of ASCII warp mesh, if the file name starts with "/" then
it is an absolute file path, if it starts with a "~" then it is relative to the users
home directory, otherwise relative to the current Quartz Composition directory. For
portability reasons the last option is usually the preferred one.
Texture control u, v: either clamp or wrap.
X,Y,Z position: provided for generality, not used for the purposes discussed here.
Width, Height: this controls the width and height of the panel the image/movie
is displayed on. For the purposes discussed here one is always running in full screen
mode, for the image/movie to fill the frame the width and height should be the inverse
of the aspect ratio. So for 4:3 aspect the width and height should be 0.75, for 16:9
it should be 0.5625.
Delta u,v: These modify the mesh texture coordinates of the warping mesh.
Not all adjustments make sense,
the most common for panoramic images/movies is to map the mouse movement to the delta u
input to enable horizontal panning.
Theta: This also modifies the texture coordinates of the warping mesh. For the
applications discussed here it is mostly used for rotating a fisheye projection for
Image: This is the input port for the image (or movie).
This is perhaps the most common type of application for
planetariums using the
spherical mirror projection technique. The input image is a fisheye and the only
sensible navigation is rotation about the center of the fisheye.
Note the use of the intensity mapping to compensate for the variable density of
pixels, and hence light level on parts of the dome.
Example: Moonlight courtesy Andrew Quinn
Sample mesh file: fisheye.data
This example also uses a spherical mirror and a single projector but it is for a dome
orientated 90 degrees to a planetarium dome.
The warp map below takes a full spherical projection (image or movie) and warps it
for a 16:9 projector and spherical mirror. Varying "Delta u" allows one to pan left/right
while the movie is playing ... hence a navigable movie.
Example: Hawkesbury River courtesy Volker Kuchelmeister using the Ladybug camera
Sample mesh file: spherical.data
The following warps a cylindrical panoramic projection (image or movie) to a perspective
projection. In the case of a movie made up of cylindrical panoramic frames this
is another example of a navigable movie. Varying "Delta u" also achieves the panning left and
right, adjusting "Delta v" is a vertical pan up or down.
Example: Place-Hampi courtesy Sarah Kenderdine, Jeffrey Shaw.
Sample mesh file: cylindrical.data
This last example is a large "standard" movie, a perspective projection. Here the
mesh simply implements a 8 times zoom, varying "Delta u" and "Delta v" allows one
to pan around within the larger movie.
Example: "6dF: Beyond the Crux" by Paul Bourke
Sample mesh file: planar.data
The patch has been tested so far with Leopard and QC3, as well as SnowLeopard
and QC4. Noting however that for many movie intensive compositions QC3 is often
much higher performing.
It should be installed in "/Library/Graphics/Quartz Composer Patches" directory.
The trick of course is how to generate the mesh file for a particular purpose. In
some cases (eg: cylindrical to perspective) there is a clear mathematical relationship.
In other cases, such as the spherical mirror based projections, one needs a calibration
program where one aligns a known test pattern while adjusting the mesh to give the
right transformation. This, meshmapper, is an example
of such an alignment program.
The mesh file format is quite straightforward. The first line of the file indicates the
type of image format the mesh is designed for. For example: 1=planar, 2=fisheye,
3=cylindrical panorama, 4=spherical panorama. The second line indicates the dimensions
of the mesh. The remaining lines describe each node of the mesh, they include the
(x,y) position in normalised screen coordinates, (u,v) texture coordinates, and
(i) a multiplicative intensity value.
The image quality seems to vary significantly with codec. Photo-JPEG seems to work
particularly well, Motion-JPEG seems to be particularly poor quality even though
this isn't evident with the QTPlayer or warpplayer.
There would also seem to be a bug in Apples QuickTime API for movies of width 4096
pixels, namely it mangles the 4096th column on nVidia cards but it works fine with ATI
cards. In both cases the cards tested have a maximum texture size of 4096.
Note that this happens in the QuickTime player from Apple, as well as warppatch.
Example: PufferSphere display balls
The Puffersphere displays employ a fisheye lens located at the radius of a
semitransparent sphere. The exact geometry varies depending on the radius of the
spheres and the projector being used. The challenge is to distort a spherical
projection into the fisheye format such that the result on the sphere appears
as a spherical projection should.
Sample input image from a video sequence of the LadyBug-3.
The resulting fisheye image, navigation is currently limited to rotations
about the north pole (center of the image).
The QC composition.
Note that this was additionally verification of pbmesh running with Snow Leopard and
the associated new version of Quartz Composer.
A simple example is given here:
pufferfishpackage.zip. Note that this is designed for the
0.8m sphere, the field of view for the fisheye is different for the other radii versions.