AR Foundation Integration¶

XRTracker integrates with Unity's AR Foundation for object tracking on iOS and Android. AR Foundation supplies the camera feed, depth data, and SLAM-based world tracking that XRTracker builds on.

Requirements¶

Requirement	Version
Unity	6.0+
AR Foundation	6.x+
ARKit XR Plugin	6.x+ (iOS)
ARCore XR Plugin	6.x+ (Android)

Required Packages¶

Install these via the Unity Package Manager:

com.unity.xr.arfoundation
com.unity.xr.arkit (iOS) or com.unity.xr.arcore (Android)
com.formulaxr.tracker

Scene Setup¶

The fastest way to get started is GameObject > XRTracker > AR Tracker. This creates a pre-configured hierarchy with all required components:

AR_Tracker
├── AR Session          (ARSession, ARInputManager)
└── XR Origin           (XROrigin)
    └── Camera Offset
        └── Main Camera (Camera, ARCameraManager, ARCameraBackground,
                         TrackedPoseDriver, ARFoundationCameraFeeder)

The root AR_Tracker has XRTrackerManager (Image Source = Injected) and TrackedBodyManager. Add TrackedBody components for each object you want to track.

Manual setup

If you prefer to set up from scratch:

AR Session — Create a GameObject with ARSession and ARInputManager
XR Origin — Create an XR Origin with a Camera Offset child containing the Main Camera
On the Main Camera, add: ARCameraManager, ARCameraBackground, TrackedPoseDriver, and ARFoundationCameraFeeder
Create a separate GameObject with XRTrackerManager — set Image Source to Injected and assign the Main Camera
Add a TrackedBodyManager component to the same GameObject

Warning

The ARSession must be active before XRTracker starts. If the AR session fails to initialize, tracking will not receive camera frames.

Camera Feed¶

The ARFoundationCameraFeeder component captures frames from ARCameraManager and feeds them to the tracker automatically each frame. In the AR Tracker prefab, this is already on the Main Camera alongside the ARCameraManager.

On iOS, ARKit provides frames from the wide-angle camera
On Android, ARCore provides frames from the main camera
Camera intrinsics (focal length, principal point) are supplied automatically by the subsystem per frame

Camera Background¶

AR Foundation handles the camera background rendering on its own — add the AR Camera Background component to the AR camera (this is standard AR Foundation setup). No additional XRTracker background components are needed on mobile.

URP setup

When using URP, the AR Camera Background requires the AR Background Renderer Feature added to your URP Renderer. Without it, the camera feed won't render. Check Edit > Project Settings > Graphics > URP Renderer > Add Renderer Feature > AR Background Renderer Feature.

Depth from LiDAR and ToF¶

When a depth sensor is available, XRTracker can use it to enable depth-based tracking:

Device	Sensor	Depth Source
iPhone Pro / iPad Pro	LiDAR	`AROcclusionManager`
Android devices with Depth API support (experimental)	ToF sensor or estimated depth	`AROcclusionManager` (see Android)

To enable depth:

Add an AROcclusionManager component to the AR Camera
Enable Depth Tracking on the TrackedBody inspector
XRTracker will automatically use the depth frames when available

Android depth is experimental

Depth tracking on Android is experimental in XRTracker 2.0. Android gives depth only once the camera has moved a little, so when a body in the scene uses depth the feeder waits at start for the first depth image, for up to Depth Wait Seconds on the ARFoundationCameraFeeder (3 seconds by default). Tracking starts without depth when none arrives. See Android.

Note

Depth tracking is optional. Silhouette and edge modalities work with color-only cameras. Depth improves robustness when available but is not required.

AR Pose Fusion (Stationary Objects)¶

AR Pose Fusion combines AR Foundation's SLAM world tracking with XRTracker's object tracking to produce stable AR overlays. Without it, tracked objects may drift or jitter relative to the real world.

How It Works¶

SLAM (AR Foundation) provides stable world-space camera tracking
Object tracking (XRTracker) provides precise object-relative pose
Pose fusion blends both signals: the object's pose is held in the AR session's world frame and composed with the AR camera pose every frame, while object tracking provides local accuracy

Requirements¶

AR Pose Fusion needs the AR session and the XR Origin of the standard AR Foundation setup, and nothing else. It creates no AR anchor, so a scene needs no ARAnchorManager. A scene that still has one loads and runs, and XRTracker does not use it.

If the AR system moves its world frame (for example after a relocalization), XRTracker carries the tracked bodies into the new world. See When the AR Session Moves Its World.

Enabling Stationary Mode¶

On the TrackedBody component (the Tracking section):

Stationary — This checkbox marks the object as fixed in the real world. A body that has a parent follows its parent and does not show the checkbox.
Smooth Time — This value controls how quickly pose corrections are blended in (seconds, default 0.1).
Accept Steady Fits — On by default, and drawn in the Edge Tracking section of the component (below Tracking Method) for a body that has no parent. It lets a stationary object that tracks by the edge method read Tracked on a soft or cluttered camera picture. It has no effect on a body that tracks by the silhouette method. See Accept Steady Fits.

On the XRTrackerManager:

Use AR Pose Fusion must be enabled (default: on when AR Foundation is available)

Tip

AR Pose Fusion is most useful for stationary objects, for example a product on a table or machinery on a floor. For objects that move in the real world, leave Stationary off.

Before the First Start¶

Until a body first starts, XRTracker keeps it drawn at the guide you placed, whether the body is stationary or not. The guide does not walk onto whatever fits, so a wrong guide no longer ends in a start on the background.

The guide of a body that starts from a viewpoint follows the camera until the first start, and XRTracker draws it on every rendered frame. After the first start, the guide stays where the body started. See Starting & Recovery for the guide, the start search and the start rules.

An edge body reads two scores under AR pose fusion. The Start Score decides the start, and the Hold Confidence decides whether the body keeps its lock. The Runtime Status of the Tracked Body inspector shows both, and TrackedBody.StartScore returns the first one. See Quality Metrics.

How a Stationary Object Stays on Target¶

A stationary object holds its pose in the AR session's world frame, and the tracker corrects that pose only with solves that the image confirms.

A steady display — The displayed pose settles on the average of recent verified solves, so the outline does not follow the frame-to-frame scatter of each solve.
A fast catch-up when the object is moved — The settled pose takes in each verified solve gently while the solve stands within about 1.5 pixels of it. From about 4 pixels on, the settled pose is seated on the solve, and between the two values the pull grows in proportion. A person who moves a stationary object sees the outline follow at once.
A safeguard against wandering fits — On a soft picture with clutter behind the object, a solve that the image does not verify can drift away from the object. When such a solve stands farther from the held pose than the AR pose can explain (30 millimeters or 8 degrees, plus twice the distance and the angle the AR camera moved in the step), XRTracker puts the tracker back at the held pose on that step. A verified solve is never touched by this. The safeguard works from the first accepted fit on, because the held pose before that is only the guide you placed.
A solve that has left the object is let go — The body is held at its last verified pose and takes the lock back only after the image confirms it again, so the augmentation does not follow the tracker onto the background.
A coast when the object leaves the view — A stationary object that leaves the camera's view is held in the world frame and no longer counts as a tracking failure. The Enable World Frame Coast setting on the Tracked Body Manager controls this, and it is on by default.

Accept Steady Fits¶

On a soft camera picture, the strict registration check seldom passes, so a body that had started could stay at its placed pose, get given up on and read Lost while the tracker was on the object. With Accept Steady Fits on, XRTracker also accepts a fit that the strict check does not pass when the coverage score clears the start bar and the fit lies inside the safeguard above. These are the same score and the same bars that decide Tracked without AR, so the outline keeps correcting on soft or low-contrast pictures instead of staying at the last fit that the strict check passed. The body keeps its lock and reads Tracked for as long as the coverage score stays at or above the stop bar.

A body that is not stationary reads Tracked and stops on the coverage score as it does without AR, so it keeps tracking under AR instead of stopping. Turning Accept Steady Fits off returns a stationary body to the strict check.

With AR, a stationary body also keeps its shown pose still while part of the object is hidden or the view is clearly weaker than usual, for example while a hand covers the object, and it takes new fits again once the view recovers. With AR, a stationary body also keeps its shown pose while the object moves fast on screen. Turning Accept Steady Fits off turns these holds off as well.

With AR, the first fit of a stationary object is accepted only after it has settled, which takes a few frames, so a fit that is still moving is not locked in. You can switch Accept Steady Fits off for a body that must accept a fit only through the strict registration check, and that also turns this wait off. The row sits in the Edge Tracking section of the Tracked Body inspector, and it is greyed while Stationary is off, because the setting applies only to a stationary body under AR pose fusion.

Hold Toggles¶

Each of the three holds that Accept Steady Fits switches on has a toggle of its own. The inspector shows them indented under Accept Steady Fits, and it greys them while that setting is off.

Toggle	What it does
Hold During View Dips	With AR, the shown pose stays still while the view health is below 0.8 of the usual level for this view, until the view has been healthy again for 5 steps in a row. A dip can only begin after 30 healthy steps in a row. With this off, new fits keep moving the pose while part of the object is hidden.
Hold During Fast Motion	With AR, no new fit is stored while AR moves the object more than 6 pixels per frame on screen, counted for a camera with a focal length of 450 pixels. The shown pose is carried by AR until the camera slows down. The first fit and the first 30 steps after the start are never held back. With this off, fits taken during fast motion keep moving the pose.
First Accept Guard	With AR, before the first fit is shown, 3 solves in a row must agree within 8 millimeters and 2 degrees. The guard lets go after 30 steps, so a fit that never settles is still taken in the end. With this off, the first fit that passes the other checks is taken at once.

All three are on by default, and each one acts only while Accept Steady Fits is on. In code they are TrackedBody.HoldDuringViewDip, HoldDuringFastMotion and FirstAcceptGuard. The tracker reads them on every step, so a change applies at once. A scene that was saved before the toggles existed reads all three on, so it behaves as it did.

When the AR Session Moves Its World¶

After a relocalization, the camera pose of the AR session can jump by far more than a hand can cover in one frame, while the session still reports tracking. A body under pose fusion holds its pose in the world of the session, so without a correction the held pose stays in the old world, and the body can lose the object.

XRTracker compares the camera pose of every step with the step before it. A step counts as a jump when the camera moved more than 25 centimeters and faster than 5 meters per second, or turned more than 45 degrees and faster than 900 degrees per second. On such a step, XRTracker carries every tracked body into the new world, so each body keeps its place relative to the camera. The console shows one line for each jump.

The Follow AR World Jumps setting on the XRTrackerManager controls this. It is on by default, and the inspector shows it under Use AR Pose Fusion, only while pose fusion is on. With the setting off, the bodies stay at the poses they held in the old world, so a body can lose the object after a jump. The tracker reads the setting on every step, so a change applies at once, and a scene that was saved before the setting existed reads it on. In code the setting is XRTrackerManager.FollowArWorldJumps.

State Machine¶

When a stationary body is tracking, it follows this state machine:

None → Stabilizing → Accepting ↔ Holding

State	Behavior
Stabilizing	Applies native tracking pose for ~30 frames to settle
Accepting	Object stays in place in the AR world frame. Tracker corrections are blended in smoothly when quality is good
Holding	Tracking quality dropped — holds the last good pose, feeds it back to the tracker until quality recovers

Tips¶

Always include an ARSession in the scene — without it, AR Foundation will not initialize
Test on device — AR Foundation features (especially depth) behave differently than in the Editor
On iOS, ensure Camera Usage Description is set in Player Settings
On Android, enable ARCore Required in XR Plug-in Management if your app requires AR
Depth on Android is experimental in 2.0. If depth is noisy or unavailable on a device, rely on silhouette or edge modalities instead