Skip to content

Recording & Playback

XRTracker can record camera feeds for offline playback, and also render synthetic sequences from the Scene View for development without physical objects.

Platform Availability

Recording works on all platforms (iOS, Android, Windows). Playback is available on Windows only — transfer recordings from mobile devices to your desktop for replay and analysis.

Live Camera Recording

The primary recording method captures the live camera feed (color + depth) during a tracking session. Recording is handled entirely in native code — the tracker automatically captures every frame that passes through the pipeline.

Native API

// Start recording — frames are saved automatically during tracking
FTBridge.FT_StartRecording(outputDirectory);

// Check state
bool isRecording = FTBridge.FT_IsRecording() != 0;
int frameCount = FTBridge.FT_GetRecordedFrameCount();

// Stop and finalize
int totalFrames = FTBridge.FT_StopRecording();

Recordings are saved to the specified directory. A recording also carries the camera pose of every frame (the extrinsics), so a recording made on a phone with AR can replay AR pose fusion later. On mobile, a common location is:

string dir = Path.Combine(Application.persistentDataPath, "Recordings",
    DateTime.Now.ToString("yyyy-MM-dd_HH-mm-ss"));
FTBridge.FT_StartRecording(dir);

GUI Controls Helper

For quick recording during development, add the CameraFeedRecorderGuiControls component to any GameObject. It provides an on-screen Start/Stop button with a frame counter and blinking REC indicator — useful for recording on device without writing any code.

Setting Description
Anchor Screen corner for the recording panel
Offset Margin from the screen edge
Panel Width Width of the recording UI

Scene View Recorder (Editor Only)

The Scene View Recorder is an Editor tool for creating synthetic sequences from the Unity Scene View. This lets you set up and test tracking without access to the physical object or a camera — you render your 3D model from the Scene View and use the resulting sequence as input.

Open it via Tools > XRTracker > Scene View Recorder in the menu bar.

How It Works

  1. Arrange your 3D model in the scene
  2. Open the Scene View Recorder window
  3. Set the output directory and capture settings
  4. Navigate around the model in the Scene View
  5. Click Start Recording — the tool captures color (and optionally depth) from the Scene View camera at the configured frame rate
  6. Click Stop Recording when done

The recorder works outside of Play Mode — no need to run the game. It captures directly from the Scene View camera.

Setting Description Default
Directory Output folder for the recording —
Width / Height Capture resolution 640 x 480
Target FPS Capture frame rate 30
Capture Depth Also render depth from the scene Off
Depth Scale (meters per unit) Scale of the depth values, shown while Capture Depth is on 0.001
Truth Body (Optional) When set, the recorder writes the true pose of this object for every frame to a truth.jsonl file next to the recording. The file lets you check tracking accuracy later. Empty

Playback (Windows Only)

To play back a recorded sequence:

  1. On the XRTrackerManager, set the Image Source to Sequence
  2. Set the Sequence Directory to the path containing the captured data
  3. Enter Play Mode — the tracker receives frames as if they came from a live camera

Sequence Player Feeder

The Sequence Player Feeder component controls the playback of the sequence.

Setting Description
Auto Play The sequence starts playing when the tracker initializes.
Loop The player starts the range again when it reaches the end.
Hold Start Frame The player repeats the start frame of the range on every step and loops on it, so the tracker refines the pose on that one image again and again until it settles. The end of the range and the Loop setting are not used while this is on, and the inspector greys them.
Custom Start / Custom End The first and the last frame of the range that the player plays.

Playback with AR Pose Fusion

When a recording carries camera poses and Use AR Pose Fusion is on, the tracker replays AR pose fusion as it ran live. A replay has no AR session, so XRTracker moves the camera along the recorded poses.

  • When a recording with camera poses plays under pose fusion and the scene has no RecordedCameraPoseDriver, XRTracker adds one to its own object. The driver puts the camera on the pose of every finished frame.
  • Before the bodies are enabled, the driver seats the scene on the first recorded pose. A body that you placed before the recording's world frame existed then starts where the recording had it.
  • A scene that already carries a driver is left as it is, so the camera is never driven twice.
  • A desktop recording has no usable camera pose. With pose fusion on, XRTracker ignores pose fusion for it, logs one line and plays it without fusion.

A recording that carries an anchor history from an earlier version still plays back.

Recording Format

Recordings are stored as a directory of files:

2026-03-13_14-30-00/
├── sequence.json        # Camera intrinsics, resolution, frame count
├── color/
│   ├── 000000.png       # Color frames
│   ├── 000001.png
│   └── ...
├── extrinsics.jsonl     # Camera pose of every frame (recordings with AR data)
├── depth/               # (if depth was captured)
│   ├── 000000.exr
│   ├── 000001.exr
│   └── ...
└── poses/
    └── poses.json       # Per-frame body poses

Use Cases

Debugging on Desktop

Record a session on device where tracking fails, transfer the recording to your desktop, and replay it while adjusting parameters — no device needed.

Regression Testing

Build a library of recorded sequences covering key scenarios. After changing tracking settings, replay all sequences and verify quality stays within range.

Development Without Hardware

Use the Scene View Recorder to create synthetic sequences from your 3D model. This is especially useful early in a project when physical prototypes aren't available yet, or when the target device isn't at hand.

Tips

  • Record on the target platform — camera characteristics differ between devices
  • Include challenging scenarios — partial occlusion, fast motion, poor lighting
  • Live recordings capture exactly what the tracker sees — they reproduce issues faithfully
  • Scene View recordings are synthetic — they're great for initial setup but don't replace testing on real camera feeds