YouTuber Colsto has made a name for himself by finding novel ways to track down hidden locations: those of album cover shots, Primitive Technology, and, um, Andrew Tate. (Happily, we can all do the last one now.) In his latest video, though, Colsto has outdone himself. The video describes a technique that can reveal where an otherwise unidentifiable recording was made by analyzing its audio for the faint sounds of passing aircraft. The technique is both fascinating and frightening.
[embedded content]
It works by looking at Doppler curves, which plot the way a plane’s sound changes as it passes overhead because of the Doppler effect. This is the same familiar Doppler effect that causes the frequency of a siren to rise as it approaches and then fall as it passes by and moves away. More generally, it’s the phenomenon whereby the frequency of a wave perceived by an observer changes when the observer and the source of the wave are moving in relation to one another. (It applies to all sorts of other waves, by the way, including electromagnetic ones, which we discussed recently in the context of the weird things that happen as you approach the speed of light.)
It’s sound that’s relevant here, though, and specifically the sound of aeroplanes. Early in his video, Colsto brings up a map of the flights that were in the air at the time the video he’s tracing was recorded. Much of this information is publicly available. Aircraft equipped with Automatic Dependent Surveillance–Broadcast (ADS-B) transmit data about their position, velocity, and altitude, which flight-tracking services collect and make available to the public. While not every plane is tracked, the system provides an enormous amount of information about aircraft movements. (We also wrote about this not so long ago in the context of drone geolocation.)
Here’s a screenshot from the OpenSky Network, which tracks ADS-B data, of planes above New York City at the time I was writing this piece:
And here’s one zoomed a little further out:
It’s clear, then, that there are a lot of planes in the air above the USA at any given moment. It’s not just the USA, either:
The point is that if you live in a country over which a large number of planes fly, then at any given moment, there’s likely to be a plane somewhere above you. The thing is, you don’t even need to hear the planes. Propeller aircraft produce distinctive low-frequency sounds that can travel long distances and register on microphones even when they’re too faint to be noticed by human listeners.
As you might have guessed, then, the basic idea is to use ADS-B data to match the planes heard in the recording to planes in the sky, and then use that information to figure out the video’s location. Of course, while this sounds simple in principle, in practice it’s difficult but startlingly effective.
To demonstrate the idea, Colsto starts with a video whose visual component is a simple, low-resolution still image of some trees. All we know is that the video was filmed on September 8 somewhere in the continental USA. As we watch, he pulls up a spectrogram of the accompanying audio and scans it to see if he can find what he’s after. As listeners, all we hear are ambient sounds and birdsong. But zooming into the spectrogram (which Aphex Twin fans may recognize as the same sort of plot in which our hero hid his face back in the day), Colsto locates two innocuous-looking streaks: Doppler curves from passing aircraft, the first 920 seconds before the second.
This is a start, but only a start, because it turns out that many, many places in the US heard two planes 920 seconds apart on the date in question. The key to zeroing in on the correct location is the Doppler curves themselves, which encode a remarkable amount of information about the planes that created them.
For a start, their basic shape provides decent estimates of both the plane’s velocity and distance from the microphone. Cross-referencing the curves with ADS-B data initially produces about 70,000 candidate locations. Colsto then narrows that number to approximately 15,000 by looking for places where two planes passed 920 seconds apart, while ruling out areas where additional aircraft should have been detected in the recording. To get from 15,000 to 1 requires looking at the subtle details of the two curves, and specifically at the small aspects of their shape that reveal what the plane was doing as it passed by: accelerating or decelerating, climbing or descending, changing heading, etc.
Rather than trying to extract this information from the curves’ minute details, Colsto works in reverse: he models a whole bunch of potential curves, then compares these to the actual data and scores them by how closely they fit. Once he has a strong candidate, he identifies the flights in question, narrows down the location, and bam: a quick jaunt on Google street view later and he has the exact security camera that took the footage.
This is both a really impressive technical feat and a kinda terrifying proof-of-concept. There are obvious disclaimers: the fact that there’s so much flight data about the USA means that you need to have some way of narrowing down the dataset to begin with, because just starting with a sound and no more information means looking for the proverbial needle in a haystack of flight paths. But with a recording containing the right aircraft sounds, and some information about when or where it was made, the technique offers a surprisingly powerful way to narrow down its location. It’s another reminder that even the background noise in a video can reveal information its creator never intended to share. Happy filming!