What Are the Different Types of Stereo Camera Recording Devices?

TLDR

• Six families cover almost every build: integrated modules (ZED 2i, RealSense D455), active IR depth cameras, custom genlocked rigs, head-mounted egocentric recorders, stereo event cameras, and consumer 3D/VR capture.

• Baseline drives usable range: the 50 mm baseline on an Intel RealSense D435 suits tabletop manipulation, while 120 mm on a ZED 2i pushes reliable depth further out.

• Global shutter beats rolling shutter on any moving robot, because rolling-shutter skew corrupts rectification before your disparity map ever sees it.

• Hardware trigger or PTP sync, plus a stored ChArUco calibration and raw rectification parameters, decide whether a stereo dataset is trainable later.

• Match the device to the policy: LeRobot expects per-camera streams keyed to actions, so record left, right and timestamps separately.

What are the different types of stereo camera recording devices?

Stereo camera recording devices fall into six practical families: integrated fixed-baseline modules, active infrared depth cameras, custom multi-camera rigs with genlock, head-mounted egocentric recorders, stereo event cameras, and consumer 3D/VR capture devices. Each family trades baseline, shutter type, sync precision and portability differently, and that trade decides what your recordings can be used for.

• Integrated fixed-baseline modules: Stereolabs ZED 2i, ZED X, Luxonis OAK-D. Two sensors factory-calibrated on one board, depth computed on device or on host GPU.

• Active IR stereo: Intel RealSense D435/D455, Orbbec Gemini. Two IR sensors plus a projected dot pattern that adds texture to blank walls and plain tabletops.

• Custom rigs: two or more machine-vision cameras (FLIR Blackfly, Basler ace) on a rigid bar, hardware-triggered, with lenses and baseline you pick.

• Head-mounted egocentric recorders: Project Aria, Meta Quest 3 passthrough capture, and purpose-built wearables such as Robgence's Rebocam and the newer Rebocam 2 for first-person manipulation data.

• Stereo event cameras: paired DAVIS346 or Prophesee IMX636 sensors that report per-pixel brightness changes instead of frames.

• Consumer 3D and VR capture: Canon RF 5.2mm Dual Fisheye, Insta360 and similar, built for 180-degree stereoscopic video rather than metric depth.

How does a fixed-baseline stereo module differ from a custom stereo rig?

A fixed-baseline module ships calibrated, synchronized and packaged, so you get rectified pairs minutes after unboxing. A custom rig gives you the baseline, lens, shutter and sensor you want, at the cost of building sync, calibration and thermal stability yourself. Pick modules for speed, rigs for control.

• Calibration ownership: ZED and RealSense store intrinsics and extrinsics in firmware. On a custom FLIR pair, you own the ChArUco capture, the stereoCalibrate run and the drift checks.

• Baseline freedom: a rig lets you set 200 mm for outdoor mobile robots or 40 mm for close-range gripper cameras. A module's baseline is fixed in aluminium.

• Mechanical rigidity: factory modules hold alignment on one PCB. A two-camera bar needs a stiff mount, because a fraction of a degree of flex re-breaks rectification.

• Cost and spares: replacing a RealSense D455 is one part number. Replacing one half of a bespoke rig means recalibrating the pair.

• Compute: OAK-D runs depth and neural inference on device. A raw rig hands you two image streams and nothing else.

Which stereo devices work for egocentric, head-mounted recording?

Head-mounted stereo needs a short baseline near human interpupillary distance (around 63 mm), low weight, global shutter and on-board IMU for motion compensation. Project Aria, Quest 3 passthrough capture and dedicated wearable rigs like Rebocam fit that envelope. Bench modules bolted to a helmet usually fail on weight and cable strain.

• Project Aria pairs mono global-shutter SLAM cameras with an RGB camera, IMUs and microphones, built for research egocentric capture.

• Meta Quest 3 records stereo colour passthrough, useful for VR teleoperation sessions where the operator's view is the training signal.

• Robgence's [Rebocam](https://robgence.com/products/rebocam) targets first-person manipulation capture in homes, warehouses and labs, where the hands and the objects both stay in frame.

• [Rebocam 2](https://robgence.com/products/rebocam-2) is the next revision of that wearable, aimed at the same egocentric manipulation workload at larger collection volumes. Check the product page for current specifications before you spec a fleet.

• Helmet-mounted machine-vision pairs still make sense when you need raw Bayer data and exact lens control, though you carry the sync box.

Egocentric stereo only pays off if the dataset keeps hand poses, gaze or head trajectory alongside the frames. Cross-embodiment work such as the EgoCross benchmark shows how much the surrounding annotation shapes what transfers. Robgence's roundup of 25 egocentric datasets for robotics compares the capture setups behind the best-known sets.

What does baseline length actually change?

Baseline sets depth range and minimum distance. Disparity scales with baseline, so a wider pair resolves depth further out and loses close objects to non-overlapping fields of view. A 50 mm pair like the RealSense D435 excels at 0.3 to 3 m; a 120 mm pair like the ZED 2i holds up much further down a corridor.

• Short baseline, 40 to 60 mm: wrist cameras, tabletop manipulation, bin picking. Minimum range drops to tens of centimetres.

• Human-scale baseline, 60 to 80 mm: head-mounted and humanoid-head capture, including OAK-D's 75 mm spacing.

• Wide baseline, 100 to 250 mm: mobile bases, AMRs, outdoor platforms where 5 to 20 m matters more than the near field.

• Very wide, vehicle-scale: automotive stereo front cameras, where accuracy at 50 m outranks anything under 2 m.

Depth error grows with the square of distance and shrinks with baseline, so doubling the baseline roughly halves range error at a given distance. Lens focal length pulls the same lever, with a narrower field of view as the price.

Active IR or passive stereo: which should you record with?

Record with active IR indoors on textureless surfaces, and passive stereo outdoors or wherever multiple cameras share a scene. The projected pattern on a RealSense D455 rescues blank walls and white tabletops. Sunlight washes that pattern out, and two projectors in one room interfere with each other.

• Active IR strengths: dense depth on plain surfaces, reliable under dim warehouse lighting, consistent output for grasp planning.

• Active IR limits: outdoor performance drops hard, reflective and transparent objects still fail, multi-camera cells need projector scheduling.

• Passive stereo strengths: no interference between units, works in daylight, preserves true scene appearance for vision policies.

• Passive stereo limits: needs texture. A matte white table gives a correlation engine nothing to match.

• Hybrid approach: many teams record passive RGB pairs for policy training and a separate active depth stream for ground truth.

Where do stereo event cameras and shutter choice fit?

Use global shutter on anything that moves, and consider stereo event cameras when motion blur or dynamic range breaks conventional frames. Rolling shutter skews a moving scene line by line, which breaks epipolar geometry before matching starts. Event sensors such as the Prophesee IMX636 report microsecond-scale brightness changes instead of exposures.

• Global shutter: every pixel exposes at once. Standard on RealSense D4xx IR pairs, OAK-D mono sensors and most machine-vision rigs.

• Rolling shutter: fine for a static tripod scan, poor on a moving arm or a walking operator.

• Stereo event pairs: high dynamic range, no blur, sparse output. Matching across two event streams needs different algorithms than block matching.

• IMU coupling: inertial data closes the gap when vision degrades, which is why inertial motion capture benchmarks for dual-arm teleoperation treat IMU and camera as one system.

How do you keep stereo recordings synchronized and calibrated?

Sync stereo pairs in hardware, not software. A shared trigger line or IEEE 1588 PTP holds both sensors to sub-millisecond agreement; USB timestamps drift and the drift lands in your disparity map. Calibrate with a ChArUco board before and after every session and store both results with the recording.

• Hardware trigger: one signal fires both sensors. Standard on FLIR and Basler, exposed on ZED X and OAK-D variants.

• PTP over Ethernet: keeps distributed cameras and robot controllers on one clock across a cell.

• Calibration cadence: capture 40 to 60 board poses covering corners, edges and multiple depths, then check reprojection error.

• Drift checks: a post-session calibration that disagrees with the pre-session one flags recordings for review instead of silently poisoning training.

• Storage discipline: keep raw left and right frames plus the rectification parameters, never rectified frames alone.

Robgence's guide to building a data annotation pipeline that scales covers the validation side of that workflow.

What should a stereo dataset ship with to be usable for training?

A usable stereo dataset ships raw left and right streams, per-frame timestamps, camera intrinsics and extrinsics, IMU where available, and synchronized action or pose data. Depth maps alone are a dead end, because you can't re-derive them with a better algorithm two years later.

• Raw rectified-and-unrectified pairs, lossless or high-bitrate, with the codec documented.

• Timestamps per camera, in a single monotonic clock, not per-file creation times.

• Intrinsics, distortion coefficients and baseline, plus the calibration date.

• IMU, joint states, gripper state and teleoperation commands aligned to the same clock.

• A dataset card describing the scenes, lighting, operators and failure cases.

Policy frameworks expect that shape. LeRobot keys each camera stream to actions per timestep, so a recording missing its clock alignment can't be loaded without repair work.

How do you choose a stereo device for a robot learning program?

Start from the task distance, then fix shutter, then sync, then portability. Manipulation inside 1 m favours a short-baseline global-shutter module; navigation favours a wide baseline; human demonstration data favours a head-mounted rig. Buy the cheapest device that satisfies all four constraints, and spend the savings on recording hours.

• Tabletop manipulation: RealSense D435 or an OAK-D, wrist and overhead, active IR on.

• Mobile robots and AMRs: ZED 2i or ZED X, passive, 120 mm baseline, PTP synced to the base controller.

• Human demonstration at scale: Project Aria or a wearable such as Rebocam or Rebocam 2, with hand and head tracking recorded alongside.

• High-speed or high-dynamic-range work: stereo event cameras, accepting the algorithmic overhead.

• Buy versus collect: labelling platforms such as Encord and Labellerr handle annotation and curation once footage exists; collection vendors including Robgence handle the capture side, from rigs and operators through to delivery. Compare both on sync guarantees, calibration records and scene diversity before committing.

For a broader view of what's already available, see Robgence's 2026 guide to robotics training datasets and its teleoperation data collection tooling overview. Teams planning a custom capture can reach the Robgence team through its contact page.

Frequently asked questions

What are the main types of stereo camera recording devices?

Six families: integrated fixed-baseline modules such as the ZED 2i and OAK-D, active IR depth cameras such as the RealSense D455, custom genlocked machine-vision rigs, head-mounted egocentric recorders such as Robgence's Rebocam and Rebocam 2, stereo event cameras, and consumer 3D/VR capture devices like the Canon RF 5.2mm Dual Fisheye.

Is a stereo camera the same as a depth camera?

No. Stereo cameras infer depth by matching two views, while depth cameras include time-of-flight and structured-light designs that measure distance directly. Many devices blur the line: the RealSense D455 is a stereo pair assisted by an IR projector, so it behaves as both.

What baseline should I use for robot manipulation?

Roughly 40 to 75 mm for work inside 1 m, which matches the RealSense D435 at 50 mm and the OAK-D at 75 mm. Wider baselines push the minimum range too far out, so near-field objects fall outside the overlapping field of view.

Do I need global shutter for stereo recording?

Yes, for anything moving. Rolling shutter exposes a frame line by line, which skews moving objects and breaks the epipolar alignment that stereo matching depends on. Static tripod scanning is the one case where rolling shutter is acceptable.

How do I synchronize two cameras in a stereo rig?

Use a hardware trigger line or IEEE 1588 PTP so both sensors expose together within sub-millisecond tolerance. Software timestamps from USB drivers drift under load, and that drift shows up as depth error you can't correct afterwards.

Can I use VR headsets as stereo recording devices?

Yes, within limits. A Meta Quest 3 records stereo colour passthrough that captures an operator's view during teleoperation, which suits imitation learning. Headsets expose less raw sensor control than research devices such as Project Aria, so metric depth work usually needs dedicated hardware.

What's the difference between Rebocam and Rebocam 2?

Both are Robgence head-mounted stereo recorders for first-person manipulation data. The original Rebocam covers wearable egocentric capture in homes, warehouses and labs, and Rebocam 2 is the newer revision of that capture hardware. Check the product pages for current specifications before you spec a collection programme.

What does an egocentric stereo recorder add over a fixed camera?

First-person viewpoint, hand-object occlusion as the human sees it, and natural head motion. Devices like Project Aria and Robgence's Rebocam capture demonstrations in real environments, which fixed overhead cameras can't reproduce because the viewpoint never matches the robot's.

What metadata must ship with a stereo dataset?

Raw left and right frames, per-camera timestamps on one monotonic clock, intrinsics, distortion coefficients, baseline, calibration date, IMU if present, and aligned action or pose data. Delivering only pre-computed depth maps makes the recordings unusable with better algorithms later.