OCELLUS Supports stable marker-free tracking that integrates cameras with multiple sensors for virtual production, AR and broadcast applications including sports, news and weather.
OCELLUS camera tracking system - top, Sensor Unit; left, Processing Box, right, Lens Encoder.
Sony’s new OCELLUS (ASR-CT1) is a markerless camera tracking system purpose-built for augmented reality and virtual production applications in broadcast and cinema. Tracking the camera through multiple sensors instead of by recording markers, OCELLUS is the first tracking system Sony has produced so far, and can be used with both cinema and broadcast cameras.
Sony already supports Virtual Production and AR projects with tools ranging from acquisition to display screens such as the Crystal LED VERONA to software-based systems such as the Virtual Production Toolset.
"The growing market for virtual production and AR demands simpler, more efficient workflows. Metadata such as camera position and lens values are crucial for pre- and post-production," said Sebastian Leske, Head of Business Development, Cinema Line. "Our new camera tracking system enhances efficiency through its ability to deliver data from productions in real time."
Mapping and Metadata
OCELLUS, suited for virtual production of content such as in-camera VFX and AR material, sends the camera position and orientation data while the camera is shooting. The system comprises a sensor unit containing five sensors, a processing box and three lens encoders, and can be used with Sony Cinema Line cameras, system cameras and non-Sony cameras.
The Sensor Unit has 5 sensors.
With the sensor unit and Sony's Visual SLAM (Simultaneous Localization and Mapping) system, the system creates a limited-range reference map, enabling stable marker-free tracking both indoors and outdoors. SLAM systems use algorithms to construct and update an approximate map of an unknown environment, while keeping track of the camera’s location inside it at the same time.
When using Sony cameras, metadata about focus, iris and zoom values from the camera and lens can be acquired via the camera's SDI output and transmitted in real-time to external devices via Ethernet cable. Cooke /i lenses, B4 lenses and E-mount lenses are supported, and the camera in use must support metadata embedding on SDI output.
If the lens doesn’t support metadata acquisition through the camera, lens encoders can be fixed to the camera to capture it - see the componenets below. The acquired metadata can then be used for virtual production and AR. The system also records tracking data, other camera/lens metadata, timecode and file name, which then becomes available to use in post-production workflows.
OCELLUS Components
The system’s sensor unit is small, lightweight [86 mm × 60 mm × 43 mm, 259g] and has five image sensors. For stable marker-free tracking able to resist occlusion, at least one of the image sensors has to capture valid feature points in order to extract tracking data.
infra-red LEDs on both sides of each image sensor help when tracking in low-light environments, and a Visible Light Cut Unit is included for stable tracking in environments with frequent lighting changes.
In real time, the Processing Box transmits tracking data and camera/lens metadata to CG rendering software like Unreal Engine via Ethernet cable in the free-d format through Genlock input, Timecode input, SDI I/O and lens encoder connection ports. This data can also be recorded as FBX files on SDXC memory cards (UHS-II/UHS-I) that are synchronized with the video files of main camera. The Box has an OLED display for checking IP address, tracking information, lens data and so on.
The Lens Encoder detects precise rotation angles and positions of lens focus, zoom and iris values, and transmits detected data to the Processing Box via LEMO 7-pin cable. It is built with five different types of gears for various lenses. These features make it possible to acquire metadata for lenses and cameras that don’t support lens data embedding on SDI output.
The camera tracking system OCELLUS ASR-CT1 will be exhibited at the NAB Show 2025 in Las Vegas in April. Expected release date is November 2025. pro.sony
