Creating Spherical Panoramas
with the Canon 5D and 15mm Fisheye Lens

Peter Gawthrop

Revised 27th September 2007

Introduction

This document describes how I create spherical panoramas with the Canon 5D and 15mm Fisheye Lens together with Hugin.

The original photos used in this tutorial are available here together with the pto file for stitching and this file.

Projections

**Figure 1:** Projections: Equirectangular, Sinusoidal, Mercator & Rectilinear

**Figure 2:** Projections: Transverse Mercator, Miller Cylindrical, Stereographic: looking up & Stereographic: looking down

Imagine standing at a viewpoint on a fine day and looking all around the horizon, up to the point above your head and down to your feet - you are viewing a spherical panorama. The word ``spherical'' comes from imagining that you are surrounded by a transparent sphere centred on you head; the scene then seems to be painted onto this sphere - the spherical panorama.

If this transparent sphere is imagined as the earth with lines of longitude (numbered from $0-360^\circ$ ) and latitude (numbered as $0^\circ$ at the equator, $90^\circ$ at the north pole and $-90^\circ$ at the south pole) then every point that we see can (like places on the earth) be assigned a unique latitude and longitude.

The problem of representing a spherical panorama on a flat surface is the same problem as that of creating maps to represent the surface of the earth and it is not surprising that many names and concepts have been taken from one problem to the other. In particular, the idea of a projection, for example Mercator's projection, is much used. It is clear that any projection from a sphere to a flat surface must lead to distortion.

As an example, Figures 1 and 2 show six different projections of the same spherical panorama. In each case, lines of latitude at $45^\circ$ south, $0^\circ$ (the equator) and $45^\circ$ north as well as lines of longitude at $0^\circ$ , $90^\circ$ west and $90^\circ$ east have been drawn on.

Equirectangular projection: . Lines of latitude and longitude are equally spaced; north corresponds to the top, south to the bottom. Because there are $360^\circ$ horizontally, and $180^\circ$ , the image is twice as wide as it is high. Because points representing the north and south poles are spread over the top and bottom respectively, there is considerable distortion towards the top and bottom - note that the tripod mount is spread out over the entire bottom of the picture.
This projection is an important basic projection as it is easy for a computer program to extract pixels for any latitude and longitude: thus it can be used as the basis of panoramic movies: see here for this example and others.
Sinusoidal projection: Like the equirectangular projection, this projects all points from the sphere onto a flat surface. The main difference is that the north and south poles become points ranter than lines, so there is less distortion.
Mercator's projection: This projection is the most common map of the world projection. It counteracts the sideways stretch near the poles by an upward stretch. Although is not possible to show points near the poles using this projection, it is a good general purpose way of making a flat panorama from a spherical panorama.
Transverse Mercator's projection: This is like Mercator's projection except that the sphere is put on one side before projection. It is good for extracting vertical objects when objects to the side are not important.
Stereographic projection: The effect of this projection is similar to using a fisheye lens. The two examples given correspond to pointing the lens straight up and straight down.

This article shows how to create and project spherical panoramas using an ordinary digital camera with a special mount together with some free software.

Equipment and Software

Canon EOS 5D

a full frame (35mm) digital camera with 12.8 Mpixel resolution.

Canon EF 15mm f/2.8 Fisheye

a full-frame fisheye lens.

Canon Remote Switch TC-80N3

to avoid vibration. The cheaper model RS-80N3 would do just as well.

Panoramic heads

There are two alternatives that I use:

Manfrotto 303SPH Virtual Reality Head: parallax-free adjustable tripod head with ``click'' to position horizontally and adjustable vertical angle. This is an excellent robust device but it is heavy - weighing in at about 2kg.
Nodal Ninja: a lighter and cheaper alternative - weighing about 0.5kg. This needs a Manfrotto 120 3/8" to 1/4" tripod adaptor to work with the next item.

Manfrotto 438 Ball Camera Leveller

to level the panoramic head with needing to adjust tripod legs - built-in level indicator.

Manfrotto 190MF3 Mag Fiber Tripod

to hold it all up. Rigid and lightweight.

Lowepro Compu Trekker AW 34030 backpack

Carries all the above and a laptop.

Hugin

to stitch the pictures into a panorama.

Gimp

for final processing.

Setting up

Camera

In the past, the critical thing here was to have nothing set automatically so that each picture in the panorama has the same settings. However, The latest release of hugin includes photometric adjustment; this means that the camera can be set on Av to automatically choose the shutter speed (variable aperture would cause problems with different lens characteristics at each aperture). It is still worthwhile to use auto exposure bracketing (AEB) give over and underexposed panoramas for increased dynamic range; see section 6. I use jpegs for simplicity and reduced space; this gives plenty of range due to three exposures. But use raw if you wish.

Set the camera mode dial to Av (fixed aperture).
Use the menu to set AEB (auto exposure bracketing). Set to the maximum of 2 stops.
Set ISO to 200 (good value for UK, use 100 for Australia).
Set white balance to ``daylight''.
Set aperture to 16 - this gives a good depth of field.
Use the menu to set the highest quality jpeg.
Use to menu to ``Save camera settings'' - you can now use the C mode on the dial whenever you take a panorama.
Set the lens to MF (manual focus).
The 5D has a useful ``select folder'' feature. Use the menu to create a new folder each time you take a panorama.

Panoramic Head

The critical thing here is to set up the head once and for all such that the camera rotates about the nodal point to give no parallax.

Fix the ``Ball Camera Leveller'' to the tripod - it can be left there permanently.
Fix the ``Virtual Reality Head'' to the ``Ball Camera Leveller''. Tighten at least one screw so that it can't move.
Use the ``Ball Camera Leveller'' to level the ``Virtual Reality Head''.
Follow the Manfrotto or Nodal Ninja instructions to set up the ``Virtual Reality Head'' . Accuracy is important.
Stick coloured insulation tape to each plate to mark positions; this saves later recalibration.
Select the ``n=8'' (45 degrees) rotation angle.
Remove the camera plate from the head, but leave attached to the camera.
Remove the head from the leveller - it will fit in the backpack without folding or disassembly when the vertical rotation is set to appropriately.

Taking the Pictures

**Figure 3:** Camera orientation. The firm lines show the four directions at 30deg up, the dashed lines at 30deg down. The numbers show the order in which the images are taken

**Figure 4:** The Eight Pictures: Nominal Exposure.

**Figure 5:** The Eight Pictures: Over Exposed

**Figure 6:** The Eight Pictures: Under Exposed

It is possible to cover the panoramic sphere with 8 pictures. I don't like using zenith and nadir shots as they are difficult to stitch; I find the 4+4 pattern described below works well. The top row of pictures are 30degrees up at 0,90,180,270; the bottom row are 30 degrees down at -45,45,135,225. A diagram is given in Figure 3; Figure 4 gives an example set of images.

Because spherical panoramas must include features in direct sunlight and shadow as well as the sky, it is not possible to get a satisfactory exposure to cover the sphere. I use the auto exposure bracketing (see section 4.1) feature to give me three exposures; this does give a satisfactory range when the three exposures are combined (see section 6). Figure 5 is an over exposed version of Figure 4; Figure 6 is an under exposed version of Figure 4.

Choose a location for the tripod. Consider:
1. Artistic merit
2. Firm footing for the tripod
3. Safety - you must be able to walk around the tripod whilst concentrating on the camera.
4. Hiding the sun - ie the camera is in shadow.
Fix the panoramic head and the camera. Make sure the camera plate is exactly located to the tape marks.
Attach the remote switch to the camera.
Level using the ``Ball Camera Leveller''.
Set the camera mode dial to C (see section 4.1).
Make sure the lens is set to MF (manual focus) and focus it to infinity.
Set the camera pointing up at 30 degrees to the horizontal. Set the horizontal position at the about zero and pointing towards the central feature of your panorama.
Use the remote to take the three bracketed exposures and rotate the camera though 90 degrees horizontally (2 clicks).
Repeat to give three further sets of pictures.
You now have the top row of each of Figures 4- 6 (0.jpg-03.jpg).
Set the camera pointing down at 30 degrees to the horizontal.
Set the horizontal position back by 45 degrees (1 click back). Repeat steps 8 and 9.
You now have the bottom row of each of Figures 4- 6(04.jpg-07.jpg).
The 24 pictures should now be in their own folder. Use item 9 of section 4.1 to create a new folder for the next panorama.

Processing

The basic idea is to stitch three panoramas from eight photographs - one for each exposure level. I prefer to choose my own control points. I use what I call ``cross-stitching'': each top row photo has control points to the two adjacent photos in the bottom row.

The three panoramas are then combined in GIMP to take advantage of the full exposure range.

Preprocessing

For those enlightened ones using Linux, the shell-script pano_init does some of the preprocessing. The following steps can, however be done by hand.

Rotate the 24 photos by 90 degrees:
```
mogrify -rotate -90 *jpg
```
Rearrange into 3 sets of 8 in separate directories; one set for each of the three exposures. Call the files (in order 0.jpg-7.jpg) in each directory; I call the directories ``Exposure_0'', ``Exposure_over'' and ``Exposure_under''.

Stitching

**Figure 7:** Stitched Panorama: nominal

**Figure 8:** Stitched Panorama: over exposed

**Figure 9:** Stitched Panorama: under exposed

**Figure 10:** The Final Panoramas: Using GIMP

**Figure 11:** The Final Panoramas: Using Qtpfsgui

In the directory corresponding to the nominal exposure, invoke Hugin to create mypano.pto¹:
```
hugin mypano.pto
```
Click on the second (``Images'') tab and load in the 8 files using the ``Load time series of Images'' button.
Choose image 0 (for example) as anchor and set the pitch to 30 degrees.
Click the ``Camera and lens'' tab. Select ``Full-frame fisheye''. Click on the first image and press the ``Load exif data'' button. This gives a field of view of about 91.4 degrees.
Under the same tab, click on the ``Photometric'' tab. Click on each image and then click on ``Load exif data'' - this loads the exposure value ``EV'' for each image computed from EXIF data.
Make sure that the ``Enable rotation'' box is ticked in the File -> Preferences -> Finetune tab - this is needed for fisheye images.
Click the ``Control points'' tab.
Choose picture 0 and picture 4 and select three well-spaced control points on non-moving objects. Repeat for:
1. 0 and 5
2. 1 and 5
3. 1 and 6
4. 2 and 6
5. 2 and 7
6. 3 and 7
7. 3 and 0
Click the ``Optimizer'' tab. Select ``incremental from anchor'' and optimize. Reoptimise using ``positions and barrel distortion''. This should give quite accurate stitching.
Click the ``Exposure'' tab. Click on ``Optimize now'', thyis matches up the EV for each image - it is this feature that allows the Av mode to be used when taking the images.
Click the ``Stitcher'' tab. The field of view should be 360x180. Choose the pixel dimensions of the panorama; up to 11470x5735 is possible but choose 3000x1500 to start with. Choose bicubic interpolation. Stitch a tif using enblend. An example, stitched from Figure 4, is given as Figure 7.
Copy (or symlink) the pto file to the two other directories and stitch the under and over exposed panoramas; the control points etc will be the same! An example, stitched from Figure 5, is given as Figure 8, and an example, stitched from Figure 6, is given as Figure 9.

Composing the image

There are at least three possible approaches:

Load the three panoramas into Gimp as separate layers and use layer masks to combine.
Load the three panoramas into Cinepaint using File->New from>HDR Bracketing. To create a 32bit floating point image.
Use qtpfsgui to create a High Dynamic Range (HDR) image from the three panoramas and tonemap the result to give a Low Dynamic Range image such as a jpeg.

I have had good results from the first and the third approaches; I prefer the third.

GIMP

Invoke Gimp with the correctly exposed panorama:
```
gimp Exposure_0/mypano.tif
```
Use the right mouse button to bring up the menu File->Open as Layer and load Exposure_under/mypano.tif.
Right click on the thumbnail for this new layer to bring up Add layer mask. Choose the ``Grayscale copy of layer'' option.
Repeat 2 and 3 with Exposure_over/mypano.tif except that the ``Invert Mask'' box is ticked in the mask menu.
You should now have a reasonable combined image with detail in the shadows and no overexposed sections. See Figure 10 as an example.
The top two layers can be adjusted by:
1. adjusting layer opacity,
2. adjusting the contrast of each layer mask and
3. painting on the layer mask.
Save the final version as a jpeg.

This process (with much more sophistication) is implemented in the
exposure-blend plugin - this is strongly recommended.

An alternative suitable for batch implementation is given in Appendix A.

Qtpfsgui

Qtpfsgui is a nice interface to algorithms for generating HDR images from multiple exposures and tonemapping the result. There are 2 preparatory steps:

Within each of the three exposure directories (``Exposure_0'', ``Exposure_over'' and ``Exposure_under'').
1. Convert the stitched panorama from tiff to jpeg. I use imagemagick
```
convert mypano.tif mypano.jpg
```
2. Load the camera EXIF data into the stitched panorama. I use exiftool
```
exiftool -TagsFromFile  0.jpg  --ThumbnailImage --PreviewImage  mypano.jpg
```
Copy the 3 jpegs into one directory - call them mypano_0.jpg, mypano_over.jpg and mypano_under.jpg.

Now run qtpfsgui and follow the instructions. Figure 11 uses the Drago tone mapping method with a post-processing gamma=0.7.

Creating Projections

**Figure 12:** Using Stereographic Projection

As discussed in the Introduction, there are many projections available to represent a spherical panorama on a flat sheet of paper. This article just discusses the technical aspects of projections; aesthetic considerations are left to the reader. One projection I quite like is given in Figure 12; the tripod could be easily removed using the clone tool if necessary.

Hugin has a nice interactive feature for generating views of projections which I used for generating all of the pictures here. It is worth noting that, to achieve the best quality, the final image should be generated using the original pictures rather than a previously generated equirectangular projection.

Final thoughts on Projections

Spherical panoramas provide new opportunities for digital photography. Apart from the obvious application of generating ``virtual tours'' for estate agents and on-line advertising, I believe that new artistic opportunities are opened up.

The development of perspective was one of the triumphs of the Renaissance art; and the renunciation of perspective has been the hallmark of 20th century art. However, perhaps because cameras make it so easy, perspective is often not a central concern of photographers. David Hockney, in his book ``That's the way I see it'', has interesting photo-collages which challenge conventional perspective. I believe that digital photography and perspective software such as hugin give us an opportunity to follow Hockney's seminal ``Pearblossom Hwy'' photo-collage in ``a panoramic assault on Renaissance one-point perspective''.

Acknowledgement

I am grateful to Dr Karl Harrison for his excellent web-page which got me off the ground. The developers of Hugin are to be thanked for there efforts in creating such excellent software.

Exposure blend using ImageMagick

The following shell script gives exposure blending suitable for batch use. It uses ImageMagick .

=src2latex.s2t =src2latex.s2t

#!/bin/sh

##Exposure-blend

##Mergesthreephotostakenusingexposurebracketing

if[-z"$4"];then

echoUsage:Mergenormal_imageunder_exposed_imageover_exposed_imagemerged_image

exit

fi

normal=$1

under=$2

over=$3

merged=$4

echo"Creating$mergedfrom$normal,$underand$over."

##Mergeunderexposure

convert-typeGrayscale$under_mask.tif#Createmask

convert$normal$under_mask.tif+matte-composite_normal_over.tif#Merge

##Mergeoverexposure

convert-negate-typeGrayscale$over_mask.tif#Createmask

convert_normal_over.tif$over_mask.tif+matte-composite$merged#Merge

##Clean

rm-f_mask.tif_under_masked.tif_normal_over.tif

;''

To do list

There are more things I would like to do but haven't yet.

Remove chromatic aberration from the original images there is a hugin tutorial hugin tutorial .
Align the each set of three images - there can be a pixel or two error. One approach is to use align_image_stack .
Use HDR throughout - see HDR workflow with hugin .

About this document ...

Creating Spherical Panoramas
with the Canon 5D and 15mm Fisheye Lens

This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.71)

The command line arguments were:
latex2html -no_navigation -local_icons -split 0 -address peter@gawthrop.net paper

The translation was initiated by Peter Gawthrop on 2007-09-27

Footnotes

... mypano.pto ¹: ``mypano'' can be replaced by any other name

peter@gawthrop.net

Creating Spherical Panoramas with the Canon 5D and 15mm Fisheye Lens

Contents

Introduction

Projections

Equipment and Software

Setting up

Camera

Panoramic Head

Taking the Pictures

Processing

Preprocessing

Stitching

Composing the image

GIMP

Qtpfsgui

Creating Projections

Final thoughts on Projections

Acknowledgement

Exposure blend using ImageMagick

To do list

About this document ...

Footnotes

Creating Spherical Panoramas
with the Canon 5D and 15mm Fisheye Lens