Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

Is HDR image content recommended inside volumes or ImmersiveSpace on visionOS?
I have a question about using HDR visual content in visionOS, especially inside a volumetric window or an ImmersiveSpace. For example, if I display an HDR photo or use very bright lighting on reflective materials, I can sometimes trigger a noticeably higher peak brightness on Vision Pro. However, in my testing, this higher brightness only lasts for a short time. After that, the overall scene brightness appears to be reduced automatically. I would like to understand the intended behavior and recommended usage pattern. Specifically: Is HDR image content recommended for use inside volumetric windows or immersive spaces? Are HDR photos or high-brightness reflective highlights intended mainly for short-duration local highlights rather than sustained full-scene brightness? Does visionOS apply automatic brightness, tone mapping, thermal, or power-related management when immersive content contains very bright HDR areas? Are developers expected to avoid large or sustained high-brightness HDR content in immersive scenes? What is the recommended way to author HDR images, bright highlights, emissive materials, or reflective materials so they look realistic without causing the whole scene to dim? Are there any visionOS-specific guidelines for HDR content, peak brightness, average scene brightness, or comfort when presenting HDR photos or bright 3D materials? My goal is to use HDR content responsibly for realistic highlights, reflections, glass, metal, or photo display, while avoiding visual discomfort or unexpected global dimming behavior. Any guidance would be appreciated.
1
0
370
Jun ’26
ARKit Face Tracking works in total darkness?
I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true? I'm using an iPhone 16 to test, and the Face ID works well in darkness.
1
1
547
Jun ’26
Is a volumetric window recommended for menus in 3D-focused visionOS apps?
For a visionOS app whose main content is 3D, such as a game or immersive experience, is it recommended to use a volumetric window as the main menu or in-game menu? I’d like to understand the intended use cases for menus built as volumetric windows. Specifically: What are the advantages of using a volumetric window for menu UI compared with a regular 2D window or an in-scene SwiftUI attachment? What limitations should developers be aware of, such as fixed size, placement behavior, lighting separation, interaction comfort, or window management? For a 3D game-like app, is a volumetric menu generally considered a good visionOS design pattern, or should volumetric windows be reserved for 3D content rather than menu-heavy UI? Any guidance on the recommended design approach would be appreciated.
4
1
431
Jun ’26
Tracking with a single rear camera
M4 iPad Pros only have a single camera rear camera, are the older iPad Pro models assisting the tracking with the ultra-wide camera? We noticed that a AR Session recording records all rear and front cameras, but covering the ultra-wide doesn't impact the tracking in our testing. Would a partial cover (at the corners not the center) of the ultra-wide lens POV impact the tracking as well?
1
0
345
Jun ’26
Lack of documentation on RealityKit
SceneKit was perfect to import 3D assets directly into our app scene. Since it's soft deprecation and the coming out of RealityKit, tutorials or documentation on how to bridge old possibilities are missing, everything seems to be about AR. More specifically, I am not able to find any resources on how to create a scene, load 3D assets, add a light and a camera, and everything within the app itself. No augmented reality. Is it intended to add some support about this? Any guidance would be appreciated.
2
1
543
Jun ’26
What is the latest recommendations and best practices for using 3-D content creation, tools, and importing 3-D objects into reality composer pro 3
Currently, I have a very basic entity in RCP that I call in my app's immersive view based on an included Package. I'm trying to get a much better 3d asset to use in my app instead. I've been afraid to put much effort into learning Blender to create and export an USDx asset for import into RCP3 (it's a whole world of learning itself...). I may be able to start using agents like Codex tied into Blender to see if I can more quickly create a 3d asset. I tried using the new AI assistant in RCP3 and it seems like I could create a 3d entity, but I'm not sure it will be the level of detail I want. Any high-level advice for the workflow (nowadays with AI agents) to create a desirable 3d asset, get it into RCP3, and then into Xcode?
3
0
410
Jun ’26
Why are polygon count limits still important in ImmersiveSpace with foveated rendering?
I have a question about rendering performance guidelines for visionOS ImmersiveSpace. Since visionOS uses foveated rendering, why are polygon count and scene complexity still treated as strict performance constraints for immersive content? My understanding is that foveated rendering reduces rendering cost outside the user’s central field of view. If so, should polygon count mainly matter near the gaze/foveal region, while objects in the peripheral area are much cheaper to render? Specifically: Does foveated rendering reduce only pixel shading cost, or does it also significantly reduce geometry processing cost? Are polygons outside the foveal region still submitted, culled, transformed, and rasterized in a way that affects CPU/GPU performance? For large immersive environments, should developers still optimize total scene polygon count, or focus mainly on what appears near the user’s gaze? Are there recommended guidelines for LOD, culling, and polygon budgets in visionOS immersive spaces, even when foveated rendering is enabled? I’d like to better understand how foveated rendering affects geometry budgets, and how developers should think about scene complexity for immersive visionOS apps. Any guidance would be appreciated.
2
0
400
Jun ’26
Motion Blur during ARKit Sessions
When not sunny in outdoor environments during an ARKit session, captured frames often suffer from motion blur due to automatically selected longer shutter speeds. What are the best practices for prioritizing faster shutter speeds to reduce motion blur? Are there specific APIs or recommended configurations to safely balance the low-light/shutter speed trade-off without disrupting tracking?
1
0
394
Jun ’26
Reality Composer 3 iOS versions support
I'm building an iOS game with RealityKit + Reality Composer Pro and was hoping to migrate to Reality Composer Pro 3. I created a new project in RCP and used the "link a new xcode project" flow. It built and rendered correctly on iOS 27. However, I encountered two issues when trying to build for iOS 26. The new RealityKitScripting package threw an error indicating <iOS 27 is not supported. I removed the related code, as I do not intend on using any Script Graphs. The iOS 27 scene still loaded correctly after removing this package. Are there any other consequences to removing this package other than not being able to use Script Graphs? Once the scene loading was working on iOS 26, I noticed that no materials built with the Shader Graph would render. Unlit and Physically Based materials render correctly, but Shader Graph materials do not. Even a simple Shader Graph with one Unlit material node input would not work. Is this expected behavior? My main question is, should I not expect to be able to use Reality Composer Pro 3 if I'm going to target an OS version <27?
0
0
410
Jun ’26
Automated testing via session replay
Is there a supported route to automated regression testing of ARKit apps? Reality Composer's "Record AR Session" plus Xcode's "ARKit Replay data" scheme option work well for manual debugging, but replay isn't wired into XCUITest, doesn't run in CI, and ARKit doesn't exist in the simulator — so every AR regression today needs to be run by a human holding a device. Even replay-driven ARSession in the simulator, or an XCTest hook for selecting replay files, would unlock some automated coverage.
2
0
396
Jun ’26
website environment and visionOS 27 beta
Is the website environment feature available on this first visionOS 27 beta? I am having troubles converting a web page that used the visionOS 26 website environment approach to the new visionOS 27 approach. Before I keep working on it, I would like confirmation that the feature is working. Also, does Apple have any web pages featuring a website environment that I can inspect?
3
0
403
Jun ’26
SharePlay connection issues between gen 1 and gen 2 vision pro's
SharePlay Connectivity Issue: Asymmetric Connection Failures Between Vision Pro Generations Environment: Device 1: 2nd generation Vision Pro Device 2: 1st generation Vision Pro OS Version: visionOS 6 build 23O471 Custom App Version: Both devices running identical version Both devices fully updated to the same OS and app versions Issue Description: SharePlay connections exhibit asymmetric behavior based on invitation direction, despite successful FaceTime connectivity between devices: 2nd gen → 1st gen: SharePlay invitation fails consistently 1st gen → 2nd gen: SharePlay invitation succeeds consistently. FaceTime calls: Successful in both directions Impact: Users cannot reliably initiate SharePlay sessions from newer generation devices to older generation devices. The issue is isolated to SharePlay functionality, as standard FaceTime connectivity works bidirectionally and both devices run identical versions of the custom app.
1
2
825
Jun ’26
LowLevelRenderer: visionOS custom shaders, composition with RealityView, custom surface shader ABI
A few questions about the intended usage of LowLevelRenderer / LowLevelRenderContextStandalone for custom rendering (custom Metal materials, dynamic meshes, custom splat compositing): visionOS custom shaders in immersive space LowLevelRenderer is listed as available on visionOS 27. Does that include user-authored Metal surface shaders via makeSurfaceShader when rendering in an immersive space, or are custom fragment shaders restricted there the way CustomMaterial was? Any foveation / Compositor Services constraints we should know about? Composition with RealityView Is LowLevelRenderer (via LowLevelRenderContextStandalone) intended only for fully standalone/offscreen rendering where the caller composites the output, or can it inject draws into an existing RealityView's managed render pass? On visionOS specifically, what's the supported way to get its output onto the drawable alongside RealityKit-rendered content with correct depth interaction (mutual occlusion between custom draws and normal entities)? Custom surface shader ABI For makeSurfaceShader with a user-authored Metal function: what's the function signature contract? Specifically — which [[buffer(n)]] / [[texture(n)]] slots are reserved by the renderer vs. available to the caller, how do camera and per-instance uniforms arrive in the shader, and how do LowLevelArgumentTable bindings map to argument indices? A pointer to sample code or header documentation for the shader-authoring contract would be ideal. Hot reload / resource lifetime For an authoring tool that recompiles shaders frequently: can LowLevelRenderContextStandalone.Resources be rebuilt incrementally, and what's the cost of re-init via the synchronous init(resources:) path? Any guidance on reusing resources across context instances? Relationship to GaussianSplatComponent Is GaussianSplatComponent built on LowLevelRenderer internally? For 4D / animated splats and counts above the component's internal limit, is implementing custom splat rendering directly on LowLevelRenderer a path you'd recommend, or discourage? Thank you!
3
3
392
Jun ’26
Cloth on skinned characters
Can the new cloth sim be driven on a skinned character mesh — a cape pinned to the shoulders following skeletal animation — and does it run at interactive rates on visionOS, or is it aimed at non-real-time/hero content? Can a sim be cached/baked for deterministic playback?
2
0
304
Jun ’26
Is HDR image content recommended inside volumes or ImmersiveSpace on visionOS?
I have a question about using HDR visual content in visionOS, especially inside a volumetric window or an ImmersiveSpace. For example, if I display an HDR photo or use very bright lighting on reflective materials, I can sometimes trigger a noticeably higher peak brightness on Vision Pro. However, in my testing, this higher brightness only lasts for a short time. After that, the overall scene brightness appears to be reduced automatically. I would like to understand the intended behavior and recommended usage pattern. Specifically: Is HDR image content recommended for use inside volumetric windows or immersive spaces? Are HDR photos or high-brightness reflective highlights intended mainly for short-duration local highlights rather than sustained full-scene brightness? Does visionOS apply automatic brightness, tone mapping, thermal, or power-related management when immersive content contains very bright HDR areas? Are developers expected to avoid large or sustained high-brightness HDR content in immersive scenes? What is the recommended way to author HDR images, bright highlights, emissive materials, or reflective materials so they look realistic without causing the whole scene to dim? Are there any visionOS-specific guidelines for HDR content, peak brightness, average scene brightness, or comfort when presenting HDR photos or bright 3D materials? My goal is to use HDR content responsibly for realistic highlights, reflections, glass, metal, or photo display, while avoiding visual discomfort or unexpected global dimming behavior. Any guidance would be appreciated.
Replies
1
Boosts
0
Views
370
Activity
Jun ’26
Can I use the Camera API to shoot pictures with the wide camera, while AR is running on the main camera
I want to: Run ARKit on the main rear camera, and while it's running shoot high resolution pictures on the wide camera, without disturbing the AR tracking. Is this possible?
Replies
1
Boosts
0
Views
1.3k
Activity
Jun ’26
ARKit Face Tracking works in total darkness?
I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true? I'm using an iPhone 16 to test, and the Face ID works well in darkness.
Replies
1
Boosts
1
Views
547
Activity
Jun ’26
Is a volumetric window recommended for menus in 3D-focused visionOS apps?
For a visionOS app whose main content is 3D, such as a game or immersive experience, is it recommended to use a volumetric window as the main menu or in-game menu? I’d like to understand the intended use cases for menus built as volumetric windows. Specifically: What are the advantages of using a volumetric window for menu UI compared with a regular 2D window or an in-scene SwiftUI attachment? What limitations should developers be aware of, such as fixed size, placement behavior, lighting separation, interaction comfort, or window management? For a 3D game-like app, is a volumetric menu generally considered a good visionOS design pattern, or should volumetric windows be reserved for 3D content rather than menu-heavy UI? Any guidance on the recommended design approach would be appreciated.
Replies
4
Boosts
1
Views
431
Activity
Jun ’26
Tracking with a single rear camera
M4 iPad Pros only have a single camera rear camera, are the older iPad Pro models assisting the tracking with the ultra-wide camera? We noticed that a AR Session recording records all rear and front cameras, but covering the ultra-wide doesn't impact the tracking in our testing. Would a partial cover (at the corners not the center) of the ultra-wide lens POV impact the tracking as well?
Replies
1
Boosts
0
Views
345
Activity
Jun ’26
Import Blender material to Reality Composer Pro
What are the best practices for import from Blender? I have a material that looks beautiful in Blender, but my attempts at migrating it to Reality Composer Pro had very poor results. An image of the nodes making up the material is attached, along with a rendering from Blender and RCP.
Replies
1
Boosts
2
Views
310
Activity
Jun ’26
Lack of documentation on RealityKit
SceneKit was perfect to import 3D assets directly into our app scene. Since it's soft deprecation and the coming out of RealityKit, tutorials or documentation on how to bridge old possibilities are missing, everything seems to be about AR. More specifically, I am not able to find any resources on how to create a scene, load 3D assets, add a light and a camera, and everything within the app itself. No augmented reality. Is it intended to add some support about this? Any guidance would be appreciated.
Replies
2
Boosts
1
Views
543
Activity
Jun ’26
What is the latest recommendations and best practices for using 3-D content creation, tools, and importing 3-D objects into reality composer pro 3
Currently, I have a very basic entity in RCP that I call in my app's immersive view based on an included Package. I'm trying to get a much better 3d asset to use in my app instead. I've been afraid to put much effort into learning Blender to create and export an USDx asset for import into RCP3 (it's a whole world of learning itself...). I may be able to start using agents like Codex tied into Blender to see if I can more quickly create a 3d asset. I tried using the new AI assistant in RCP3 and it seems like I could create a 3d entity, but I'm not sure it will be the level of detail I want. Any high-level advice for the workflow (nowadays with AI agents) to create a desirable 3d asset, get it into RCP3, and then into Xcode?
Replies
3
Boosts
0
Views
410
Activity
Jun ’26
Questions about VD B and some best implementation suggestions for simulations
Hello, I would like to inquire about recommendations regarding the performance overhead of VDB simulation on the M5 Vision Pro. Additionally, I would like suggestions for the best implementation for simulating rivers, waterfalls, snowflakes, and flames.
Replies
1
Boosts
0
Views
355
Activity
Jun ’26
Why are polygon count limits still important in ImmersiveSpace with foveated rendering?
I have a question about rendering performance guidelines for visionOS ImmersiveSpace. Since visionOS uses foveated rendering, why are polygon count and scene complexity still treated as strict performance constraints for immersive content? My understanding is that foveated rendering reduces rendering cost outside the user’s central field of view. If so, should polygon count mainly matter near the gaze/foveal region, while objects in the peripheral area are much cheaper to render? Specifically: Does foveated rendering reduce only pixel shading cost, or does it also significantly reduce geometry processing cost? Are polygons outside the foveal region still submitted, culled, transformed, and rasterized in a way that affects CPU/GPU performance? For large immersive environments, should developers still optimize total scene polygon count, or focus mainly on what appears near the user’s gaze? Are there recommended guidelines for LOD, culling, and polygon budgets in visionOS immersive spaces, even when foveated rendering is enabled? I’d like to better understand how foveated rendering affects geometry budgets, and how developers should think about scene complexity for immersive visionOS apps. Any guidance would be appreciated.
Replies
2
Boosts
0
Views
400
Activity
Jun ’26
Motion Blur during ARKit Sessions
When not sunny in outdoor environments during an ARKit session, captured frames often suffer from motion blur due to automatically selected longer shutter speeds. What are the best practices for prioritizing faster shutter speeds to reduce motion blur? Are there specific APIs or recommended configurations to safely balance the low-light/shutter speed trade-off without disrupting tracking?
Replies
1
Boosts
0
Views
394
Activity
Jun ’26
Reality Composer 3 iOS versions support
I'm building an iOS game with RealityKit + Reality Composer Pro and was hoping to migrate to Reality Composer Pro 3. I created a new project in RCP and used the "link a new xcode project" flow. It built and rendered correctly on iOS 27. However, I encountered two issues when trying to build for iOS 26. The new RealityKitScripting package threw an error indicating <iOS 27 is not supported. I removed the related code, as I do not intend on using any Script Graphs. The iOS 27 scene still loaded correctly after removing this package. Are there any other consequences to removing this package other than not being able to use Script Graphs? Once the scene loading was working on iOS 26, I noticed that no materials built with the Shader Graph would render. Unlit and Physically Based materials render correctly, but Shader Graph materials do not. Even a simple Shader Graph with one Unlit material node input would not work. Is this expected behavior? My main question is, should I not expect to be able to use Reality Composer Pro 3 if I'm going to target an OS version <27?
Replies
0
Boosts
0
Views
410
Activity
Jun ’26
Automated testing via session replay
Is there a supported route to automated regression testing of ARKit apps? Reality Composer's "Record AR Session" plus Xcode's "ARKit Replay data" scheme option work well for manual debugging, but replay isn't wired into XCUITest, doesn't run in CI, and ARKit doesn't exist in the simulator — so every AR regression today needs to be run by a human holding a device. Even replay-driven ARSession in the simulator, or an XCTest hook for selecting replay files, would unlock some automated coverage.
Replies
2
Boosts
0
Views
396
Activity
Jun ’26
website environment and visionOS 27 beta
Is the website environment feature available on this first visionOS 27 beta? I am having troubles converting a web page that used the visionOS 26 website environment approach to the new visionOS 27 approach. Before I keep working on it, I would like confirmation that the feature is working. Also, does Apple have any web pages featuring a website environment that I can inspect?
Replies
3
Boosts
0
Views
403
Activity
Jun ’26
SharePlay connection issues between gen 1 and gen 2 vision pro's
SharePlay Connectivity Issue: Asymmetric Connection Failures Between Vision Pro Generations Environment: Device 1: 2nd generation Vision Pro Device 2: 1st generation Vision Pro OS Version: visionOS 6 build 23O471 Custom App Version: Both devices running identical version Both devices fully updated to the same OS and app versions Issue Description: SharePlay connections exhibit asymmetric behavior based on invitation direction, despite successful FaceTime connectivity between devices: 2nd gen → 1st gen: SharePlay invitation fails consistently 1st gen → 2nd gen: SharePlay invitation succeeds consistently. FaceTime calls: Successful in both directions Impact: Users cannot reliably initiate SharePlay sessions from newer generation devices to older generation devices. The issue is isolated to SharePlay functionality, as standard FaceTime connectivity works bidirectionally and both devices run identical versions of the custom app.
Replies
1
Boosts
2
Views
825
Activity
Jun ’26
LowLevelRenderer: visionOS custom shaders, composition with RealityView, custom surface shader ABI
A few questions about the intended usage of LowLevelRenderer / LowLevelRenderContextStandalone for custom rendering (custom Metal materials, dynamic meshes, custom splat compositing): visionOS custom shaders in immersive space LowLevelRenderer is listed as available on visionOS 27. Does that include user-authored Metal surface shaders via makeSurfaceShader when rendering in an immersive space, or are custom fragment shaders restricted there the way CustomMaterial was? Any foveation / Compositor Services constraints we should know about? Composition with RealityView Is LowLevelRenderer (via LowLevelRenderContextStandalone) intended only for fully standalone/offscreen rendering where the caller composites the output, or can it inject draws into an existing RealityView's managed render pass? On visionOS specifically, what's the supported way to get its output onto the drawable alongside RealityKit-rendered content with correct depth interaction (mutual occlusion between custom draws and normal entities)? Custom surface shader ABI For makeSurfaceShader with a user-authored Metal function: what's the function signature contract? Specifically — which [[buffer(n)]] / [[texture(n)]] slots are reserved by the renderer vs. available to the caller, how do camera and per-instance uniforms arrive in the shader, and how do LowLevelArgumentTable bindings map to argument indices? A pointer to sample code or header documentation for the shader-authoring contract would be ideal. Hot reload / resource lifetime For an authoring tool that recompiles shaders frequently: can LowLevelRenderContextStandalone.Resources be rebuilt incrementally, and what's the cost of re-init via the synchronous init(resources:) path? Any guidance on reusing resources across context instances? Relationship to GaussianSplatComponent Is GaussianSplatComponent built on LowLevelRenderer internally? For 4D / animated splats and counts above the component's internal limit, is implementing custom splat rendering directly on LowLevelRenderer a path you'd recommend, or discourage? Thank you!
Replies
3
Boosts
3
Views
392
Activity
Jun ’26
Curved windows in visionOS 27
Safari, Freeform and some other apps now support curved windows. It this now available to developers as well? I couldn't find any information about it. Thanks!
Replies
6
Boosts
1
Views
533
Activity
Jun ’26
I can’t complete assistant setup up in Reality Composer Pro 3 beta
I can’t complete assistant setup up in Reality Composer Pro 3 beta. I filled in the set up fields for OpenAI and clicked done. However, when I click connect nothing happens.
Replies
12
Boosts
2
Views
506
Activity
Jun ’26
Cloth on skinned characters
Can the new cloth sim be driven on a skinned character mesh — a cape pinned to the shoulders following skeletal animation — and does it run at interactive rates on visionOS, or is it aimed at non-real-time/hero content? Can a sim be cached/baked for deterministic playback?
Replies
2
Boosts
0
Views
304
Activity
Jun ’26
Gaussian Splats, Ingestion + Mesh Interop
The new GaussianSplatComponent: what's the supported ingestion path — standard .ply splats, .spz, or a conversion step inside RCP? And can a splat coexist in a scene with mesh geometry and depth-sort/occlude correctly against it, or are splats a separate pass with no depth interaction with meshes?
Replies
2
Boosts
1
Views
278
Activity
Jun ’26