Adobe Project Aero Set to Link Physical and Digital Worlds for Artists
On 5 June at the 2018 Apple Worldwide Developers Conference, Adobe previewed Project Aero, a new augmented reality (AR) authoring tool and multi-platform system for designing immersive content in a way that combines physical and digital elements and environments.
Apple has been building a platform, ARKit 2, for augmented reality, and Adobe is working with them on software designers and developers can use to create AR experiences. Meanwhile, Adobe continues to partner with other technologist and developer companies aiming to standardise interaction models and file formats in this area. They are also working with other platform vendors, open standards efforts like usdz and glTF, as well as media companies, artists and designers to deliver a comprehensive AR set of tools.
Consequently, in collaboration with Apple and Pixar, Adobe will begin to add USDZ file format support to Adobe Creative Cloud apps and services. Designers will be able to create AR content from applications they are already using such as Photoshop CC and Dimension CC and convert assets into USDZ that can be used and viewed natively by users with the Apple devices.
The new USDZ file format, co-created with Pixar, is used to create augmented reality content on iPhones or iPads. Pixar describes USDZ as a zero-compression, unencrypted zip archive of the USD (Universal Scene Description) format it developed and has used for many years to work with 3D content - now optimised for mobile devices. The intention is to give iOS 12 users the ability to create 3D objects that seem to appear in real, 3D space, then share them with colleagues, friends and so on. USDZ act as a container that holds several files and the tools needed to render them.
The format is accompanied by Apple's ARKit 2, a platform where developers can integrate shared experiences, persistent AR experiences tied to a specific location, object detection and image tracking to make AR apps more dynamic.
For example, shared experiences with ARKit 2 allow multiple users to play a game or collaborate on projects. Developers can also add a spectator mode, which will display the best views of AR material from a different iOS device.
Persistent AR will also form a different way for consumers to interact with AR apps by creating options to leave virtual objects in the real world, to which users can return. Typical examples would include leaving a puzzle on a table and coming back to it later in the same state, or creating an art project over the course of a few weeks without starting over each time.
ARKit 2 also extends support for image detection and tracking, making it possible to detect 3D objects like toys or sculptures. Also interesting is the ability to automatically apply reflections of the real world onto AR objects, making the AR experience a more realistic combination of the virtual and real world.
Returning to Adobe's plans for Project Aero, because AR content development blends creative and technical skills, the application is intended to be a system for developers and for creatives to build simple AR scenes and experiences through the use of Apple’s ARKit. Designers can create immersive content that can then be brought into Xcode, Apple's app development code, for further refinement and development.
As well as creating AR content in Photoshop CC and Dimension CC, Project Aero designers will be able to lay out and manipulate designs in physical spaces. In this case, Adobe will lean on their Sensei machine learning get past the expected technical complexity arising from working in more than one environment. The engine should work reasonably well because, althugh it is a new medium, AR relies on the same assets that creatives produce now, such as images, video, and 3D content, and the same skills. That fact has also made it possible for Adobe to build, from the ground up, a cloud service just for Project Aero.
As a Project Aero preview, Adobe partnered with 15 artists for the Festival of the Impossible, large-scale immersive art exhibition displaying artwork that explores the impossible during a 3-day festival in San Francisco 8-10 June. www.adobe.com
