Zoom and Enhance


Last week I watched dash cam and body cam footage of a fatal police shooting in Charlotte, NC. It was both awful and frustrating. Awful because this is just the latest in a long line of racially charged police violence. Frustrating because the video is too blurry to make out important details. “If only we could enhance that,” I caught myself saying. As a visual expert, I should know better.

Zoom and enhance” may work on TV shows, but in real life what you see in the video is what you get. Computers can adjust brightness and contrast but they can’t create new information. However, video footage is often more useful than it might seem on first viewing if you are patient enough to piece together clues that may be hidden in plain sight.

Fat Pencil Studio has worked on dozens of cases that included video evidence. There are many ways that we've been able to extract useful information from it. Some are relatively simple, such as adjusting brightness and contrast or syncing up multiple sources. Others involve paying careful attention to each frame and relating what is shown to other known facts. Here are two areas where this kind of analysis can be fruitful:

1. Timing. Video footage simulates motion by recording many still images at a (mostly) regular interval. This consistent timeline tells us a lot about how objects are moving. In one case, it was important to know when a train operator applied the brakes. Interior footage of hanging bicycles gave us the clue we needed. The bicycles started swinging when the brakes were applied. This is hard to see in real time, so the ability to control playback speed and move frame-by-frame is an important tool when hunting for tiny movements.

2. Location. When viewing footage from a moving camera, keep an eye out for reference objects that have a fixed location. In the previous example the train passed several utility poles that were visible through the window. This allowed us to determine speed and location when the brakes were first applied. When a camera location is fixed (mounted on a building or a parked car) it is often possible to accurately pinpoint the location of objects that move through the frame. This science is called photogrammetry, and we used it to figure out whether a pedestrian was in a bus driver’s blind spot in this downtown Philadelphia collision.

The increasing use of surveillance cameras presents legitimate privacy concerns, but also a tremendous opportunity to figure out things that might have perviously been considered unknowable. I recently listened to Eye in the Sky, a Radiolab episode about using aerial surveillance of entire cities to get real-time actionable information about criminal activity, from burglary to murder. If you’ve ever been curious about the capability of government intelligence operations, you’ll want to hear to this story. It certainly got me thinking about the ethics of placing so many surveillance cameras on our buildings, cars, and even police officers. Personally, I’ve come to accept this dramatic increase in surveillance, but remain nervous about how it will be used. Will our legal system require agencies to release footage for a proper and thorough vetting? Even if it tells a story they don’t like? I hope so. Feature image credit: 1982 Blade Runner, one of the first known uses of the Zoom and Enhance trope.


Joshua Cohen is a Principal at Fat Pencil Studio