How Panoramic Stitching Software Works: The Technology Behind This Ultra-Wide-Angle Tool


Made from about a dozen vertical format shots with a Nikon D800 and a 24mm lens at f/4 with exposure times of 25 seconds, at ISO 3200.
© Seth Shostak

There are some problems that only brute force can solve.

Here’s an example—a problem that’s been tormenting photographers since Mathew Brady was knee-high to a field cannon: the limited view of most images. Just about everyone has occasionally felt cramped by the confines of the traditional three-to-two aspect ratio set by 35mm film and its digital descendants.

The straightforward response to this compositional claustrophobia is to shoot panoramas, pictures that can have an unlimited ratio either horizontally or vertically. You know: that iconic 180-degree view of the Manhattan skyline.

Well, panoramas are easy to wish for, but, until recently, were not easy to get. To shoot high-quality panos often required special cameras that could rotate their lens while simultaneously advancing an aperture slit across a curved film plane.

But in the digital age, when pictures are no more than numbers, the non-trivial problem of stitching together an array of individual photos into a pano can be solved by throwing a lot of compute power against it—either in the camera or in your laptop. Today’s very fast processors have turned a formerly intractable challenge into one that can be almost brainlessly solved by tyros. For many stitching programs, little more is required than to type in a list of photos and click the mouse.

So how do they work?

In fact, they imitate what you would do if you were shuffling a bunch of overlapping transparencies on a light table, trying to combine them into a bigger composite photo. You would look for features near edges that were common to two of the transparencies, and then line those features up by sliding and rotating the photos. Once two images were overlapped, you’d move on to the third.

Of course your eyes and brain are pretty good at recognizing the common features in overlapping imagery. Software, on the other hand, is dumb: It has no clue about pictorial content. But despite their low IQ scores, computers are fast. They can use brute-force computation to find distinctive features common to pairs of pictures you’re trying to combine. These are known as control points, and include such things as exceptionally bright pixels, edges, and corners.

Your photos typically have tens of millions of pixels each, so sorting through all of them to look for distinctive features requires hundreds of millions of individual computations. That would be a bore and a chore for humans, but not for today’s microprocessors, which can usually do the work in seconds.

Stitching software does more than simply overlap multiple images. It can also straighten out perspective effects and lens distortions by stretching and squeezing each image to make for a better fit to the control points. It can also correct for vignetting—the darkening of individual images in the corners caused by a variety of gremlins. Vignetting is seldom offensive for single photos, but in a pano will cause dark seams where the images are joined.

Finally, stitchers can adjust the final pano to whatever geometric projection you prefer. You might not know that you prefer any geometric projection, but in fact you do. The choice here is akin to that faced by map makers who have to transcribe the reality of a round Earth onto a flat piece of paper. You can choose a kind of spherical projection, but more often, you’re likely to prefer a cylindrical projection in which all vertical lines remain upright and parallel.

While you might be intimidated about mastering yet another interface and set of tools, stitching software often has a mode that will construct an impressive pano with very little expertise or effort. I wager that once you’ve seen the big picture, you’ll be tempted to fill up whatever free wall space remains with some stunning panos.

(Note: The author wishes to thank Joost Nieuwenhuijse of PTGui stitching software for helpful comments.)

Seth Shostak is an astronomer at the SETI Institute who thinks photography is one of humanity’s greatest inventions. His photos have been used in countless magazines and newspapers, and he occasionally tries to impress folks by noting that he built his first darkroom at age 11. You can find him on both Facebook and Twitter.

(Editor’s Note: Technically Speaking is a new column by astronomer Seth Shostak that will explore and explain the science of photography. In this inaugural piece for our Travel & Landscape issue, Shostak looks at the technology behind panoramic stitching software.)