Discover USDKit and what’s new in OpenUSD

Welcome to Discover USDKit, and what's new in OpenUSD. I'm Lee, an engineer on Apple's Spatial Standards team.

USD is the backbone of the spatial experiences Apple creates and how we represent spatial scenes.

From the apps you create, to the content you see across our platforms, all these spatial experiences are built on top of USD.

USD was created by Pixar, and Apple was early to recognize its potential. We've been collaborating with them ever since.

Today, we're taking that even further. I'll be covering the latest advancements in USD on our platforms and a powerful new framework called USDKit and how it can bridge between Mac and Vision Pro for seamless 3D experiences.

Lets start off by looking at the foundational updates we've made to USD on our platforms.

USD has become the common 3D language across many industries. Award-winning films, AAA games, factory floors, surgical suites, autonomous vehicles, AI-driven simulations; and the reach continues to expand.

Apple is leading the charge in making USD the foundation for the next generation of spatial experiences.

The open source project behind USD is called OpenUSD, the industry-standard library for describing 3D scenes originally pioneered by Pixar. But USD doesn't work alone.

It integrates with MaterialX, originally from Lucasfilm for rich material descriptions, and new this year, OpenVDB, originally from DreamWorks, bringing volumetric data into the mix. Each of these technologies was built by world-class visual effects and animation studios, and together, they form a powerful, composable foundation for 3D.

Across all our platforms this year, we've updated all three, so you can take advantage of the latest improvements in each.

Apple is a member of the Academy Software Foundation, the home of open source projects like MaterialX and OpenVDB.

Through the Foundation, we actively contribute to these projects, ensuring they evolve in ways that benefit developers and creators across not just our platforms, but the 3D ecosystem as a whole.

Beyond the code, Apple is a founding member of the Alliance for OpenUSD, working to make USD a true industry standard, not just in practice, but on paper.

This year, we helped release the first formal specification for the core of USD, with domain specifications for geometry, materials, and physics already underway.

This is an open effort, and these working groups are where the future of USD gets decided. If you want a voice in shaping how the industry builds and exchanges 3D content, we encourage you to get involved.

Gaussian Splats are one of the most exciting recent breakthroughs in 3D representation.

Rather than using traditional geometry, splats capture a scene as millions of fuzzy, overlapping particles, each encoding position, color, and opacity to faithfully reconstruct complex real-world environments.

As you can see in this visualization, splats are able to capture incredibly subtle lighting responses and are capable of bringing real-world scenes to life.

Working with our Alliance for OpenUSD partners, including NVIDIA, Adobe, and Pixar, we are introducing a new USD primitive type, Particle Fields, which is capable of describing Gaussian Splats as well as other representations from this rapidly evolving area of research.

This brings Gaussian Splats into the same scene as your meshes, materials, and other traditional 3D data for the very first time.

Let's take a look at powerful new experiences across our platforms and the new USDKit framework built to power them.

USDKit takes care of the heavy lifting, so you can stay focused on your content and applications. All of this is made possible by the standardization work we have been building towards.

Preview has long been the go-to place on Mac for viewing images, PDFs, and 3D content, and for images and PDFs, it has always offered a powerful set of editing tools right out of the box - no extra software needed.

This year, we are bringing that same philosophy to 3D.

Preview now brings essential 3D editing to your Mac, covering the operations you reach for most.

You can manipulate objects directly in the scene, edit properties and lighting, work with full scene hierarchies, and convert and compress assets, all without needing to learn a dedicated 3D application.

While it's simple on the surface, it's all backed by a production quality rendering and processing pipeline.

Preview and Quick Look on Mac now give you a choice of renderer. RealityKit brings consistency across Mac, iPhone, iPad, and Vision Pro, and Storm remains available for those with existing production pipelines needs.

For your most complex scenes, Preview adds a brand new Raytracer for stunning, high fidelity results.

All three support OpenPBR, a significant upgrade over USDPreviewSurface that brings richer, more physically accurate materials to your workflow.

The new Raytracer in Preview is built for scenes that demand more. Whether you are visualizing architectural spaces or preparing product imagery, it delivers accurate reflections, precise shadows, and physically correct lighting.

A production quality ground-truth renderer on every Mac.

Preview integrates with the new Spatial Preview framework on macOS 27, creating a direct connection between your Mac and Quick Look on Vision Pro.

As you work on your USD scene in Preview, changes are visible live in Quick Look on the Vision Pro, right in your own space, and through SharePlay, you can bring an entire team in at once. Creative directors and artists can walk around the same scene together, reviewing lighting, composition, and spatial scale in real time.

And now, with the Spatial Preview framework, this kind of collaborative spatial workflow is easier than ever to bring to your own Mac apps. Check out the Spatial Preview session to learn how.

USD is now part of the web too.

Safari introduces the Model tag, bringing 3D content to web pages as naturally as images or video.

Embed a USD model in your page, and on macOS and iOS, your users get a fully interactive 3D experience right in the browser.

And on visionOS, it goes even further. That same model breaks out of the page and is presented, spatially, right in the user's space.

Everything we have shown you today is powered by a brand new system framework: USDKit. Let's take a look.

USDKit brings first class USD support to your Swift apps, with deep integration for RealityKit and Spatial Preview built right in. We designed USDKit to work for everyone. Developers who already know USD will find the concepts immediately feel familiar, and for Swift developers coming to USD for the first time, USDKit meets you where you are, with patterns and paradigms you already know. Before I dive in to an example, let's cover a few key USD concepts. In USD, a Layer is a single data file. Layers can be combined together through a powerful feature called Composition. And a Stage is the composed result of one or more layers. This is your window into the full scene. Everything in a scene is represented as a USD Prim. Each prim has a Schema, which defines its type.

Prims also carry Attributes, which hold the actual data and Metadata, which describes information about the prim itself. Now I've covered some key concepts; let's walk through an example of how to use USDKit in practice. I will load a stage, make some modifications, and export the result. Getting started with USDKit begins with a stage. I can create a fresh one in memory with a simple USDStage initializer. For this demo though, I have an existing scene I want to work with. I can open it by passing in a file URL to USDStage.open. Since this involves file access, it can throw, so I use try. And there it is. My scene loads up and it's already looking great. I thought there was supposed to be an oscilloscope on the bench. Let's see if it's in the scene somewhere, and if not, I can add it in.

First, let me traverse the stage hierarchy to see if the oscilloscope is already in there. No luck! So I define a new transform prim at the path I want it to live at. Now here is where things get really useful. Rather than copying all the data for the asset directly into my stage, I can add a light-weight reference to it. The asset lives in its own file, or layer, authored by someone else entirely, and I am simply pulling it in. This is the power of composition. Everyone works on their own piece of the scene, and USD brings it all together. And the best part? Any updates they make automatically show up in my stage too, because I am just referencing in their file. Great! The asset is in the scene, but as you can see, it's not quite where I want it to be. Let me show you how I can move it into the right place.

To move the prim, I first call addTransformOperation, which takes care of creating the correct attributes on the prim and updating the transform order automatically.

After that all I need to do is to set the translation value, and the asset moves up on to the workbench, right where I want it to be.

Perfect! This is looking much better already, but before I share this with the world, I want to make sure it's ready for as wide an audience as possible.

Great 3D experiences should be accessible to everyone. At Apple, accessibility is a core value, and that includes spatial content too.

That is why we have driven the standardization of accessibility metadata directly in USD, establishing how assistive labels and descriptions are defined on 3D objects across the industry. We have designed it with flexibility built in, so it can evolve in the future.

Since it is native to USD, you can author it through any USD API, and to make it as easy as possible to adopt, we have added direct support right in Blender and Maya.

To add the accessibility data to my new asset, I first apply the AccessibilityAPI schema to the prim. This adds the necessary metadata to signal that the schema is present.

Since USDKit does not provide all the schema-specific APIs, I create the label and description attributes directly, making sure to use the correct attribute names as defined in the specification.

With those in place, I can set their values: a concise label and a rich description that gives assistive technologies everything they need to understand the object in context. The asset is now ready, but high quality production USD scenes can grow to many gigabytes in size. The ALab scene we have been working with today is a great example of that. Sharing something this large is not always practical, so let me show you how I can shrink it down.

In collaboration with the Alliance for Open Media, we have added support for a state of the art mesh compression codec capable of reducing mesh sizes by up to 90%.

Combined with our existing texture compression using AVIF, the numbers speak for themselves. The average asset is now seven times smaller, without compromising visual quality.

Smaller assets mean faster delivery, lower storage costs, and a better experience for users across every platform.

USDKit support for compression is built right into the exportPackage API. I pass in the output URL to the exportPackage method on my stage, and then I enable the texture and mesh compression through the export options. A couple of lines and I am done. And if you're not writing code, you can get the same results directly in Preview or through the usdcrush command line tool. We are working with Pixar to bring this compression support to the OpenUSD project, so the whole ecosystem can benefit. Because integrating USD can be complex, there are a number of different ways to access the technology. For app developers on our platforms, USDKit is the way to go. It is system provided, deeply integrated, and everything we have shown today is built on top of it.

For those with more advanced needs or cross-platform workflows that go beyond what USDKit covers, we have you taken care of too. SwiftUSD brings open source Swift bindings available right through the Swift Package Manager, and for cross-platform C++ codebases, we have made it easier than ever to embed OpenUSD directly as a framework.

Whatever path fits your workflow, they are all built on top of the same foundation. That means your USD files move freely between all of them.

Let's recap what we've covered. USD on Apple platforms has taken a big step forward this year. Preview now brings essential 3D editing and powerful rendering options to your Mac. Spatial Preview allows you to easily immerse yourself in your content and share it with others, and the new Model tag in Safari brings USD natively to the web. At the center of it all is USDKit, a new system framework that makes working with USD in Swift a first class experience. If you are looking to author and work with 3D content in your app, USDKit is the best place to start.

There's a lot more to explore this year. We have sessions diving deeper into Spatial Preview, bringing USD to the web, and building rich spatial experiences with RealityKit and Reality Composer Pro. Whatever your workflow, there is something here for you. Check out the linked resources for more information.

USD has never been more powerful, and we cannot wait to see what you create with it. Thank you for watching!