An ordinary photo is flat. A depth camera records how far away everything is.
AI-generated image, not a photograph of a real person or place.
Each pixel holds a distance instead of just a colour. There are three main ways to get it: bend a pattern of light over the scene, time the light's round trip, or compare two views like a pair of eyes.
A range image stores a distance in every pixel. Depth cameras differ in how they get it.
PATTERNS
Bend a pattern of light
Structured light projects stripes, dots or grids onto a scene. A camera to one side sees them deform over each surface, and the shape is worked out from that. Infrared light keeps the pattern invisible.
A time-of-flight camera sends out light and times its return. Light from something 2.5 m away is back in about 16.7 nanoseconds, so the electronics must be precise.
Stereo cameras work like two eyes, matching the same point in two images. They need no special lighting, but struggle on plain, even surfaces where nothing stands out to match.
Pick a situation to see how much trouble it causes a depth camera and why. The bar shows how likely the depth reading is to suffer.
General behaviour of each method; individual devices handle these cases in different ways.
TWO NUMBERS WORTH KNOWING
Very fast light, very bright sun.
One figure on how short the timing is, one on why outdoor depth sensing is hard.
≈ 16.7 ns
Delay for an object 2.5 m away
Light travels at 299,792,458 metres per second, about 30 cm in a nanosecond. A timing slip of just 10 picoseconds between light and sensor shifts the measured distance by 1.5 mm.
ROUND TRIP · TIME OF FLIGHT
≈ 50× stronger
Sun vs a 1 W camera light, after filtering
Sunlight delivers about 1050 watts per square metre, around 50 watts after the camera's band-pass filter. A typical time-of-flight light source gives about 1 watt.
SUNLIGHT · BACKGROUND
Time-of-flight camera accuracy is usually estimated at about 1% of the measured distance, and some cameras capture up to 160 depth frames a second.
FOUR HABITS FOR CLEANER DEPTH
Help the camera see the room.
Whether it is a gesture game, a scanner or a robot, these follow from how the methods work.
AI-generated image, not a photograph of a real person or place.
Step 1
Keep strong sun off the scene
Draw a blind if sunlight falls where the camera is looking: it can swamp the camera's own infrared light.
Step 2
Cover or move big mirrors
Reflections send light along extra paths and can make objects read further away than they are.
Step 3
Run one camera at a time
Some time-of-flight cameras interfere with each other. If two point at the same space, switch one off.
Step 4
Expect gaps on plain surfaces
Two-lens cameras need detail to match between their images. Areas of plain, even colour are hard to match, so depth there may be missing or unreliable.
HOW THE METHODS COMPARE
Compact, simple, or passive.
AI-generated image, not a photograph of a real person or place.
A time-of-flight camera is compact: the light sits right next to the lens, with no need for the wide spacing that stereo and triangulation set-ups require, and no moving parts. Working out distance from its signal takes little processing.
Stereo needs no special lighting, but matching points between two images is hard, so it often gives reliable depth for only some of the scene. Structured light gets depth from a single image of the pattern, but relies on its projector being seen clearly.
Questions
Questions on the way
What is a depth camera?
A camera whose image stores the distance to each point instead of, or as well as, its colour. It is also called a range camera, and the picture it makes is a range image or depth map.
What is the difference between structured light and time of flight?
Structured light projects a known pattern and works out shape from how it deforms on surfaces. Time of flight times how long light takes to reach the scene and come back.
Why do depth cameras use infrared light?
Infrared keeps the light unobtrusive: the pattern or pulses are invisible to people and do not get in the way of other camera tasks that use visible light.
Do depth cameras work outdoors?
It is harder. Sunlight can be around 50 times stronger than a typical camera's own light even after filtering, although some short-pulse laser systems avoid the problem and are used in space and considered for cars.
Why do phones have depth sensors?
Several phones include time-of-flight sensors, mainly to tell the camera software which parts of a scene are foreground and which are background.