Iterate your spatial scenes faster with Reality Composer Pro 3

Hi everyone! I'm JP, an engineer on Reality Composer Pro here at Apple. In this session I want to talk about how you can iterate on your spatial content faster than ever using Reality Composer Pro 3. Since the launch of Apple Vision Pro, we have seen remarkable spatial experiences brought to the platform. Reality Composer Pro has been at the center of bringing that amazing content to visionOS, and it's inspiring to see what this community has been building. Our goal is to continue to give you more and better tools to bring your ideas to life with as little friction as possible. With that in mind, we are introducing some brand new capabilities that will help you realize even your most ambitious spatial projects! Reality Composer Pro 3 is built from the ground up for fast, iterative and collaborative workflows. It helps you get further along in your development without the need to touch Xcode. In this session, I'll take you through some of these brand-new capabilities, like Live preview, Lightmaps, and the Reality Composer Pro Assistant, which can generate 3D content using AI. I'll show you the process of adding a few small features to the interactive game Chaparral Village. I'll start with a quick overview of the editor core functionalities. I'll cover how to use entities and components to build up a scene. I'll introduce the new prototype and instancing system, designed to help you efficiently organize, reuse and manage your assets. I'll demonstrate how you can preview and even author your content directly on the Vision Pro. I'll use Lightmaps to bake the indirect lighting of my scene. And finally, I'll introduce you to the Reality Composer Pro Assistant, and how you can leverage it to add content to your world. First things first. Reality Composer Pro 3 is no longer available as an Xcode developer tool. You can now download it from developer.apple.com and simply launch it straight from your Applications folder. If you would like to familiarize yourself with the basics of using the Reality Composer Pro Editor, I would recommend checking out the session "Meet Reality Composer Pro" from WWDC23. Let's start with a quick overview of the project I'll work on today. This is the Alchemy Area from the Chaparral Village. All objects in this scene were modeled in Blender, imported as USD files, and then laid out in Reality Composer Pro. To get a better look, I'll use the Focus Mode from the View menu and explore the scene a bit.

This is a gorgeous scene. I'll start by adding an asset to my project. To do so, I'll use the import asset icon available in the Project Browser and select the Cauldron USD file I have on my desktop.

When a USD file is imported, its content gets organized and optimized into an import bundle. Once imported, I can expand the bundle to inspect it.

Within the bundle, I see the imported geometry, materials, textures, and more. The cauldron asset looks good. Now, to bring it into the lab, I'll just drag and drop the bundle into the viewport.

This added asset is shown in the hierarchy view and it's called an entity. It has a Transform Component that's shown in the inspector panel on the right. Entities and components are the core building blocks of everything you create in Reality Composer Pro 3. Let's go through them in more detail. In the hierarchy panel I can see a list of all the entities that make up my scene, and they can be re-ordered and nested as needed. I'll expand the fireplace and drag the cauldron underneath it.

I'll select the cauldron entity and position it at the right spot and angle by adjusting the values within the Transform Component.

I can choose to add more components to an entity using the Add Component button.

You can add lights, physics, audio, and a lot more to your entity using this panel.

I'll use a couple of these components to create an interesting visual effect for my scene. First, I'll create a child under the Table entity. I will use the context menu and choose 'Add Child Entity'. I will call it 'Magic Effect'.

To frame an entity, you can always press 'f'.

To the Magic Effect entity, I'll add another child called 'Glow'.

and I'll add a simple Point Light component.

I'll adjust the position of the light and tweak its attenuation, adjust the color, and adjust the intensity.

I'll also add the new Compute Simulation component to the Magic Effect.

In the inspector panel, I'll choose the Magic Graph that I created for my project. I can do that using the Compute Graph picker. Notice that it shows all available compute graphs in this project. I have a Magic Graph and a Brewing Graph available. I'll use the Magic Graph for now, and keep the Brewing Graph for later. Compute Graph makes GPU programming accessible to anyone. Its node-based graphs lets you build anything, from simple particle systems to complex fluid simulations. For a deeper dive on this topic, watch the session "Supercharge your spatial workflows with Reality Composer Pro 3". You'll notice that the Compute Graph isn't visible right now. That's because this graph only runs during the simulation stage. To test it, I'll use the launch control and run my game by pressing the Play button. This is where Reality Composer Pro 3 really starts to shine.

The Alchemy Area is now running and I can see the Compute Graph being simulated. Let me get a closer look.

I'll dock the simulation tab next to the scene tab.

This allows me to keep authoring my content even as the game is running. I'll place this Magic Effect into the bowl on the table and tweak the graph's twist amount to my liking.

The simulation tab allows me to make tweaks rapidly without any deployment process getting in the way. From physics simulation to script graphs, to animations, everything you author in Reality Composer Pro can be previewed in real time in the simulation tab, greatly reducing the friction between you and the final experience. Next, I'll cover a new addition to Reality Composer Pro called 'prototypes', which can be used to create powerful and reusable objects. To turn an entity into a prototype, you can drag it from the hierarchy tab directly into the Project Browser.

This creates a new prototype asset. I can then instantiate this new prototype by dragging it into the viewport.

I'll rename this new instance 'Brewing Effect'.

Notice that now I have two instances using the Magic Effect prototype. I can provide overrides to these instances, allowing me to customize them. First, I'll change the effect of the brewing instance to the Brewing Graph I mentioned earlier...

and adjust the color, attenuation, and falloff of the Glow entity.

Uh oh... that looks bad. I can always reset an override back to its source value by choosing Reset in the context menu. I'll right click on the Attenuation Falloff property and reset it.

Ok, that's much better. I'll keep the original falloff value. With prototypes, you edit your content in one place and the system handles the rest. You can instantiate a prototype multiple times and override any of those instances individually. If you don't like those overrides, you can reset them to their original values or you can even propagate the overrides back to the source. Nothing is ever permanently changed unless you want it to be. Next, I will show you another cool new feature in Reality Composer Pro called Live Preview. Since we're building this experience for the Vision Pro, I can target a simulation to be played on any Vision Pro device currently connected to my Mac. I'll use the launch control panel to start a Live Preview session which will ship later this year. This will open the companion app on visionOS. Now I can continue to author in Reality Composer Pro and see the updates reflected instantly. Notice how the blue fill light has the new physical space lighting feature enabled. Authoring an effect like this on device allows me to get an instant feel of its impact in a spatial experience. This way of live previewing dramatically cuts down on iteration times. It removes any guesswork from the process of authoring. What you see is truly what you get. This is coming along nicely but after these lighting edits, I'm noticing that the scene no longer feels quite right. This is because the indirect lighting previously generated for this scene no longer matches the new lighting. To fix this, I'll use the new Lightmaps in Reality Composer Pro. Indirect lighting captures the way light bounces around a scene and contributes to areas not directly visible by any single light. The space underneath this table for instance has no direct lighting to it but by simulating indirect lighting, we can capture the subtle light that still reaches that area, however faint it may be. For example, in my scene, most of the Alchemy Area isn't directly lit by the fireplace. But Lightmaps helps fill in those darker areas with soft bounced light, greatly improving the overall look of the scene. Simulating indirect lighting can be costly but as the lights in the Alchemy Area don't move, I can use the new lightmapping component to pre-calculate the indirect lighting term and save the results to a texture called a Lightmap. The Alchemy Area entity has a Lightmap component attached to it.

Here I can control what lighting term gets baked and fine-tune quality settings. I'll change the quality of the Lightmaps under Bake Settings from low to high.

To preview the output of the baked lights, I'll open the Lightmap Preview tab. I can do this from the Tab menu.

This allows me to see in real time how much the indirect lighting impacts the scene. I'll tweak the Lightmap settings.

The preview tab lets me get a clear picture of the final result before committing to a full bake. I'm pretty happy with those settings, and I'm ready to regenerate the scene's Lightmap. While my lights are getting baked, let me go through the different lighting terms that the Lightmap component supports. In addition to the Indirect Lighting, Reality Composer Pro can also generate Ambient Occlusion and Beauty Lightmaps. Ambient Occlusion represents the visibility of each point in a scene to its surroundings. And Beauty represents the final color of each point in a scene taking into account both indirect and direct lighting. Let me go check on my Lightmaps. And it's done! My Lightmaps are completely baked and my scene looks beautiful. Before I wrap up, I'd like to add one more thing to this scene. I was thinking it would look great if we had a few more items on the work bench. For this I'll use the all new Reality Composer Pro Assistant. The new AI Assistant is always available from the right panel.

From there I can simply prompt the assistant for help.

That looks great. Let's also add a few candles.

And my scene is looking complete. The Reality Composer Pro Assistant uses powerful generative models to craft 3D objects and materials on demand, letting you iterate faster, experiment freely, and turn ideas into reality with ease. It is also ready to answer any Reality Composer Pro questions you might have! I've covered a lot of ground today, and it only scratches the surface of what Reality Composer Pro 3 has to offer. To continue your journey with Reality Composer Pro, first, download it from developer.apple.com. While you're there, be sure to explore the available sample projects. There is a lot more in Reality Composer Pro that I wasn't able to cover in this session. To learn more, I recommend that you check out the Reality Composer Pro sessions. I speak on behalf of the whole team when I say we cannot wait to see what you will create with it. Thanks for watching!