Vizrt’s Viz Engine 5.1 now has a number of interesting upgrades that help graphics teams and crews working on live XR videos to combine real spaces, sets and people with virtual elements in real time.
Vizrt’s Viz Engine 5.1 upgrade contributes to live graphics workflows with a new AI-driven function called Reality Connect, designed with AI algorithms that support precise realistic talent immersion, including shadow casting and reflections within virtual studios and XR sets.
“Reality Connect solves the problem of people appearing to float in a virtual studio by enabling realistic shadow casting and reflections,” said Gerhard Lang, CTO at Vizrt. “Reality Connect anchors the talent within the virtual studio by using virtual 3D models of the talent that are continuously updated using Vizrt’s AI pose estimation. This feature operates on the same GPU and uses the live video feed that is used for keying.”
Reality Connect does not require extra hardware. Presently, Reality Connect is compatible only with the Viz Engine 5 native render pipeline and works in conjunction with the Precision Keyer. Users also have the option to directly integrate integrate data from motion capture systems for more flexibility.
New Options inside Virtual Windows
Vizrt's development of the Virtual Window gave new options to video wall design when it was first introduced in 2011. The virtual window acts as an extension to the real studio space by creating a representation of a virtual background in the same perspective, based on camera position. This capability adds depth to the physical studio for displaying 3D virtual graphics, and it still serves as a fundamental component of Vizrt’s virtual studios.
Viz Engine 5.1 evolves the way Virtual Windows are created when using Viz Engine’s native render pipeline or Unreal Engine’s new render blade. The resulting approach takes advantage of the sensitivity of both renderers to screen resolutions, and helps to make working with more complex video wall configurations easier, such as curved surfaces and floors.
The combined output resolution of the updated Virtual Window effect means that video walls, ceilings and floors can be up to four times UHD, a change that allows large setups to be managed by a single Viz Engine.
GhostFrame Previews for XR Sets
Previously, a common challenge users faced when using an XR sets was the lack of a proper preview. Vizrt works with ROE Visual to combine Viz Engine 5 and ROE’s GhostFrame to create a performance experience that is more comfortable and natural for talent.
GhostFrame works by displaying more than just the video feed on the LED panel, adding different inputs, some of which are invisible to the human eye but visible to the camera. The cameras are able to capture one or more sets of images while the talent only sees one. Creating multiple and hidden sources in one video feed gives production teams the freedom to operate without restrictions, and allow real-time post with free positioning and camera movement. All of this occurs in a near-zero latency environment.
For example, the person on-stage can be working with a photorealistic background, while at the same time camera tracking or green screen can be captured in-camea that is invisible for the talent.
GhostFrame permits up to four content feeds into an LED screen, which can be individually captured by a camera. The user can select which feeds are visible for the presenter and production crew and which will be the live broadcast feed. The talent can actively work with an AR object, for instance, without the risk of virtual collisions. They can look directly at the video wall for markers and teleprompting cues, or control eye-lines when conducting a remote interview. None of these will be visible in the live broadcast feed.
GhostFrame can also incorporate four different brandings or implement multiple video wall layouts using Viz Multiplay. Vizrt and GhostFrame partner ROE Visual will continue collaborating on potential creative workflows and possibilities for GhostFrame and Viz Engine 5.
More Adaptive Graphics
Another Viz Engine 5.1 advancement is an extension of Adaptive Graphics, which now includes user-defined aspect-ratio support for video walls. Originally designed for serving graphics across multiple platforms that each have different aspect-ratios, Adaptive Graphics can now be intelligently adjusted to fit video walls of any size, shape and resolution.
By extending Adaptive Graphics’ more efficient graphics production line to studio video walls, users will be able to achieve a more consistent look and unified brand identity across different platforms and displays. www.vizrt.com
Vizrt_Reality Connect_AI Pose Estimation_Shadow Casting
