With the recent announcement that VisionOS 27 would support custom actively tracked tools, via arrays of embedded LEDs, I would just like to confirm if the demo given in this video here: https://developer.apple.com/videos/play/wwdc2026/283/ at minute 1:00 is indeed an actively tracked tool, or if it is passive IR. if it's passive IR (reflectance), are there any examples of how this could be achieved, i.e. can we access the onboard IR cameras directly?

The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)

When switching from RealityView’s .spatialTracking camera mode to .virtual camera mode, the camera’s orientation relative to the scene is preserved permanently with no way to reset to default World-Up orientation. Since .spatialTracking’s camera mode will always have a non-default orientation, switching to .virtual camera mode ensures that the cameras’s ‘UP’ direction will never match the device display’s ‘UP’ direction as is default. This is especially noticeable when using .orbit camera controls, as the orbit’s UP direction matches the scene, not camera, and all rotation directions give unexpected results. Expected: When setting virtual camera mode after using spatialTracking camera mode, either 1. The Virtual Camera orientation returns to default (world up). Or 2. A 'content.camera.resetOrientation()' call is made available which resets the RealityView camera to default orientation. Reality: Switching from .spatialTracking -> .virtual camera mode permanently locks the .virtual camera’s orientation the final frame of the .spatialTracking camera’s rotation (relative to the RealityView content scene). One imperfect workaround is to reset / rebuild the entire RealityView after changing modes (by resetting .id() or otherwise. This is not ideal as it causes everything inside the make closure to rerun, which not only is a performance & time cost, visually incurs a flicker and can also be problematic with managing increasingly complicated views. Another imperfect alternative is to use more than one RealityView - which is not ideal as it incurs double the base ram usage, significantly increases code, and seemingly goes against the intent of being able to change the camera .virtual/.spatatialTracking mode at will. Code Sample: import SwiftUI import RealityKit struct RKSpatialVirtualToggle: View { @State var showAR: Bool = false var body: some View { RealityView { content in let cube = ModelEntity(mesh: .generateBox(size: 0.25), materials: [SimpleMaterial()]) cube.position.z = -1 content.add(cube) content.camera = showAR ? .spatialTracking : .virtual content.cameraTarget = cube } update: { content in content.camera = showAR ? .spatialTracking : .virtual } .realityViewCameraControls(.orbit) VStack{ Spacer() Button("Toggle AR"){ showAR.toggle() } .buttonStyle(.borderedProminent) } } } Xcode Version: Version 26.0 (17A324) iOS Version: iOS 26.5 (23F75) Tested on devices, iPhone 12 Pro, iPhone 15 Pro

Hello, we are evaluating Apple Vision Pro for an AR calibration experiment. Can BarcodeDetectionProvider on visionOS 2.0+ detect a QR code/barcode and provide its 3D position and orientation, such as a transform, so that we can align virtual content with a real-world marker? Does the BarcodeAnchor update continuously when the headset or the barcode moves? Thank you.

When I click on the download link for reality converter, it just sends me here: https://developer.apple.com/augmented-reality/tools/ Is there something I am missing?

attachments: { Attachment(id: "UI_Book_01") { Button { print("Tap...") } label: { Text("Tap me") .padding() .background(.ultraThinMaterial) }.hoverEffect { effect, isActive, _ in if isActive { print("hover...") //Unable to trigger } return effect } } }

Hello, I am doing a personal research project around ARKit camera frames and camera calibration data. For each accepted ARFrame, I export: ARFrame.timestamp ARFrame.camera.imageResolution ARFrame.camera.intrinsics ARFrame.capturedImage The target video mode is 4K / 30 FPS. In my test captures, I noticed two things: ARFrame.camera.intrinsics is not constant across the recording. Some neighboring frames from ARFrame.capturedImage have noticeably different sharpness: one frame can look sharp, while the next frame can look blurred. Regarding intrinsics, I observe small per-frame changes in fx/fy/cx/cy. For example, in one session: fx/fy changed by approximately 36 px cx changed by approximately 1.5 px cy changed by approximately 2.3 px I understand the standard meaning of the intrinsic matrix: fx / fy = focal length in pixels cx / cy = principal point in pixels relative to the image/reference frame My main question is not about the definition of these values, but about how to correctly interpret their changes over time. In particular, I am interested in cx/cy, because the principal point is important for measurements. I would like to understand whether a change in cx/cy corresponds to a meaningful update of the camera model for the current frame, and what physical or camera-pipeline process may cause this change. Questions about ARFrame.camera.intrinsics: Are ARFrame.camera.intrinsics values expected to change between frames during a single ARWorldTrackingConfiguration session? If fx/fy changes over time, can this reflect autofocus / focus breathing, internal camera calibration updates, digital crop/scaling, stabilization-related mapping, or other camera pipeline changes? If cx/cy changes over time, should I interpret this as an updated principal point for the current ARFrame image/reference frame? Does ARKit update cx/cy to account for any internal crop, scaling, stabilization, virtual camera behavior, lens movement, or calibration changes? Is there a known physical interpretation for small cx/cy changes reported by ARFrame.camera.intrinsics, or should these values simply be treated as ARKit’s best current camera model for that frame? In an ARKit-first pipeline where focus/exposure/stabilization are managed by ARKit/camera pipeline, is the recommended approach to store ARFrame.camera.intrinsics per frame together with ARFrame.timestamp and ARFrame.camera.imageResolution? Is there any public API that exposes the reason for intrinsics changes, such as focus position, stabilization transform, crop transform, active camera constituent, lens movement, or per-frame calibration update reason? I also have a related question about ARFrame.capturedImage frame quality. In some recordings, neighboring frames can have noticeably different sharpness. For example, one frame can look sharp, while the next frame can be visibly blurred, even though the scene and camera movement are continuous. Questions about frame sharpness / camera pipeline behavior: Is frame-to-frame sharpness variation expected when using ARFrame.capturedImage as the video source? Can ARKit change focus, exposure duration, ISO, white balance, crop/scaling, or other camera pipeline parameters between frames during an ARWorldTrackingConfiguration session? Is there any public API to inspect per-frame exposure duration, ISO, focus position, lens position, stabilization state, crop/scaling transform, or other capture parameters for ARFrame.capturedImage? If some frames are sharp and neighboring frames are blurred, should this usually be interpreted as motion blur / exposure behavior / autofocus behavior, or can ARKit internal processing also affect this? Is there any recommended way in an ARKit-first pipeline to reduce frame-to-frame sharpness inconsistency, or is the only reliable approach to use an AVFoundation-first capture pipeline with locked focus/exposure/ISO/white balance? The main goal of my research is to understand how to correctly interpret ARKit-provided intrinsics over time and how much of the camera pipeline behavior is observable through public ARKit APIs. Since fx/fy/cx/cy are important for measurements, I want to know whether treating ARFrame.camera.intrinsics as the Apple-delivered per-frame camera model is the safest approach. Thank you.

FB22720952 (ObjectCaptureSession fails with ARError.Code.sensorFailed on iOS 26.4.2 while RoomPlan and Measure continue to work). 17 pro Max

I have a visionOS app that plays audio using AVAudioEngine and presents both a window and an immersive space. If I close the window, the audio session gets interrupted and attempting to restart the session and audio engine has no effect. I need to dismiss the app, then reopen it, which reopens the main window, in order for audio to start playing again. This is in all visionOS 2 betas. Note that I have background audio enabled for my app.

Hi everyone, I’m currently using the RoomPlan API, which has been working reliably until recently. However, I’ve started encountering an intermittent error and I’m trying to understand what might be causing it. The error is triggered in the ARSession observer method: session(_ session: ARSession, didFailWithError error: Error) It has occurred on at least two devices: iPhone 14 Pro iPhone 17 Pro Here’s the full error message: ARSession failed domain=com.apple.arkit.error code=102 desc=Required sensor failed. userInfo=["NSLocalizedFailureReason": A sensor failed to deliver the required input., "NSUnderlyingError": Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedDescription=Cannot Complete Action, NSLocalizedRecoverySuggestion=Try again later.}, "NSLocalizedDescription": Required sensor failed.] This seems to indicate that a required sensor (likely LiDAR or camera) failed to provide input, but I’m not sure what’s causing it or why it happens only occasionally. Has anyone experienced something similar or has insight into possible causes or fixes? Thanks in advance!

I'm trying to find where to buy the Vision Pro Developer Strap Gen 2. I've looked all around this site and cannot find it. Help

Hi. I want to know what's the difference between VirtualEnvironmentProbeComponent and ImageBasedLightComponent? It seems they both can achieve the same light and reflection effect of environment.

Hello, does anyone have experience using Apple Vision Pro in countries where it has not yet been officially released? I work for a company in Austria, and we are interested in developing internal XR applications for Vision Pro. Since the device is not officially available in Austria, we are considering purchasing it in Germany. My main question is whether it is possible to develop and test Vision Pro apps using an Austrian Apple ID / developer account, or if there are any regional restrictions we should be aware of (e.g., related to App Store access, provisioning, or device functionality). Apple Support was unfortunately unable to provide a definitive answer and recommended asking here. Any insights or experiences would be greatly appreciated. Best regards, Don Appelonie

When interacting with RealityView’s realityViewCameraControls .orbit and setting a new RealityViewCameraContent .cameraTarget, the resulting camera target and camera orbit is incorrect. This can be demonstrated where one finger is orbiting the RealityView, and another pushes a button which changes the camera target. Instead of the camera facing the new target, some point in the scene is the new effective camera target and orbit point. This only occurs when an orbit interaction is currently taking place. If you stop interacting with the orbit, change target, then start orbit interacting again, everything works as expected. Though this example uses two-touches, any change of the camera target has this conflict with orbit interaction. This means interacting with orbit will result in the wrong camera view which is unexpected for users and difficult to reconcile or detect, for developers. Expected: Interacting (orbiting) the scene while setting a new camera target with the buttons on screen (at the same time), the camera’s new target shows centred in view the orbit revolves the new target and continues to match my gestures. Reality: Interacting (orbiting) the scene while setting a new camera target with the buttons on screen (at the same time), the camera’s new target is not centred in view, and camera is now orbiting an unexpected point in the scene, that is not my expected target. One imperfect workaround is to force a rebuild of the view after setting a new cameraTarget. This sets all targets correctly but results in a flicker, loss of orbit controls until re-touch and ultimately is a poor user experience, but is better than the wrong target being shown unexpectedly. Code Sample: import SwiftUI import RealityKit struct RKOribtTarget: View { @State private var target: Int = 0 @State private var rcContent: RealityViewCameraContent? @State private var rkID: UUID = UUID() let root = Entity() let center = ModelEntity(mesh: .generateSphere(radius: 0.05), materials: [UnlitMaterial(color: UIColor(.gray.opacity(0.5)))]) let red = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .red, isMetallic: false)]) let blue = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .blue, isMetallic: false)]) let green = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .green, isMetallic: false)]) var body: some View { VStack{ RealityView { content in red.position.x = 0.5 blue.position.z = 0.5 green.position.y = 0.5 center.position = .init(repeating: 0.25) content.cameraTarget = target == 0 ? root : blue root.addChild(red) root.addChild(blue) root.addChild(green) root.addChild(center) content.add(root) } update: { content in switch target{ case 0: content.cameraTarget = root case 1: content.cameraTarget = blue case 2: content.cameraTarget = red case 3: content.cameraTarget = green default: content.cameraTarget = root } } .id(rkID) .realityViewCameraControls(.orbit) VStack{ Text("Target") Button("Default") { target = 0 // Force rebuilding view resets orbit target and rotation // But shows a flicker, interaction requires touch reset // Not an ideal workaround // rkID = UUID() } .buttonStyle(.bordered) Button("Blue") { target = 1 // rkID = UUID() } .buttonStyle(.bordered) .tint(.blue) Button("Red") { target = 2 // rkID = UUID() } .buttonStyle(.bordered) .tint(.red) Button("Green") { target = 3 // rkID = UUID() } .buttonStyle(.bordered) .tint(.green) } } } } Xcode Version: Version 26.0 (17A324) iOS Version: iOS 26.5 (23F75) Tested on devices, iPhone 12 Pro, iPhone 15 Pro

ARCamera.TrackingState questions: Did the threshold or sensitivity for transitioning ARCamera.TrackingState from .normal to .limited(.excessiveMotion) or .limited(.insufficientFeatures) change between iOS 26.3 and iOS 26.4? What does "ARWorldTrackingTechnique: resource constraints [33]" mean, and is it new in iOS 26.4? Does it correspond to a tracking state degradation? Is there a way for the client to detect or respond to ARKit entering a resource-constrained mode short of the full tracking state transition — for example, a lower-level notification or a flag on ARFrame — so that apps can take protective action without interpreting it as a full tracking failure? ARCamera.unprojectPoint questions: Did the behavior of ARCamera.unprojectPoint(_:ontoPlane:orientation:viewportSize:) change between iOS 26.3 and iOS 26.4 for near-parallel geometry? Specifically, on iOS 26.3 this method returns nil when the camera ray is nearly parallel to the target plane (denominator of the ray-plane intersection → 0 at ~90° of camera rotation). On iOS 26.4, with identical code and environment, it returns a large finite value instead — we observed z = −12.27m. Since the method's optional return type implies nil is the documented signal for no valid intersection, this reads as a behavioral regression rather than an intentional change. If returning the computed value for near-parallel geometry is now the intended behavior, is there a recommended way for the caller to guard against it? For example, should we check abs(dot(rayDirection, planeNormal)) against a threshold before calling, and if so, is there a documented epsilon Apple uses internally? Alternatively, is there a newer API we should prefer over unprojectPoint(:ontoPlane:) for this use case that handles degenerate geometry more gracefully — such as ARSession.raycast(:)? Are there any other ARKit API adjustments between OS 26.3 and 26.4? We are using the same codebase, but it behaves differently in between these 2 OS versions now. Thanks!

The object capture feature in Reality Composer App is only available in iOS and iPadOS at the moment, would this feature be available for visionOS in near future? Reality Composer App Store https://apps.apple.com/us/app/reality-composer/id1462358802

The texture baking section of the WWDC25 session "Optimize your custom environments for visionOS" (https://youtu.be/RELnRZmb02c?t=1485) moves very quickly and leaves a lot unexplained. Has anyone worked through this part of the toolkit in practice and can speak to what's actually going on, particularly around projection baking and how it addresses the reprojection artifacts the presenter briefly mentions? Thankyou

I'm experiencing an issue where all RealityView content disappears when the user selects "Lock In Place" from the window management menu (long press on close button). "Follow Me" works correctly and this happens in Testflight builds only not reproducible when I run locally I have reproduced this with a minimal project containing nothing but a simple red cube — no custom anchors, no app state, no dependencies. Steps to Reproduce: Open an ImmersiveSpace A red cube is placed 1m in front of the user via RealityView Long press the X button on any floating window Select "Lock In Place" The cube disappears immediately Expected: Cube remains visible after window is locked Actual: Cube disappears. Note: "Follow Me" does NOT reproduce this issue. Minimal reproducible code: struct ImmersiveView: View { var body: some View { RealityView { content in let cube = ModelEntity( mesh: .generateBox(size: 0.3), materials: [SimpleMaterial(color: .red, isMetallic: false)] ) cube.setPosition(SIMD3(0, 1.5, -1), relativeTo: nil) content.add(cube) } } } Device: Apple Vision Pro visionOS version: Vision OS 26.2 (23N301) Xcode version: Version 26.3 (17C529) Is this a known issue? Is there a recommended workaround to preserve RealityView content during Lock In Place transitions? Thank you!

I’m trying to build a persistent world map of my college campus using ARKit, but it’s not very reliable. Anchors don’t consistently appear in the same place across sessions. I’ve tried using image anchors, but they didn’t improve accuracy much. How can I create a stable world map for a larger area and reliably relocalize anchors? Are there better approaches or recommended resources for this?

Area: visionOS 26.4 · AVFoundation · AVCapture · External/UVC video Classification: Suggestion / API Enhancement Request (also: Incorrect/Missing Documentation) Device / OS: Apple Vision Pro, visionOS 26.x. Xcode 26.4.1, XROS26.4.sdk. Summary On visionOS, a third-party app cannot display two UVC USB cameras (connected through a powered USB-C hub) at the same time. Every AVFoundation path that would enable this on iPadOS is either unavailable or fails at runtime on visionOS, and there is no public non-AVFoundation fallback (no IOUSBHost, no DriverKit, no usable CoreMediaIO, no MFi path for generic UVC devices). This is a real capability gap relative to iPadOS and macOS, and Camo Studio on iPadOS (App Store ID 6450313385) demonstrates the two-camera USB-hub use case is legitimate and valuable for spatial-video/hybrid-capture workflows on Vision Pro. Steps to reproduce Connect a powered USB-C hub to Apple Vision Pro with two UVC webcams attached. Build a visionOS app that uses AVCaptureDevice.DiscoverySession(deviceTypes: [.external], …). Observe: both cameras are discovered and enumerate as distinct AVCaptureDevices. Attempt A — two independent sessions: Create two independent AVCaptureSessions, each with one AVCaptureDeviceInput and one AVCaptureVideoDataOutput, start both. Result: only one session delivers sample buffers. The other stalls silently with no error and no interruption notification. Attempt B — AVCaptureMultiCamSession with manual connections (the pattern that works on iPadOS 18+): Result: code does not compile. In XROS26.4.sdk: AVCaptureInputPort is API_UNAVAILABLE(visionos) (AVCaptureInput.h) AVCaptureInput.ports is API_UNAVAILABLE(visionos) AVCaptureDeviceInput.portsWithMediaType:sourceDeviceType:sourceDevicePosition: is API_UNAVAILABLE(macos, visionos) Therefore AVCaptureConnection(inputPorts:output:) cannot be constructed. AVCaptureMultiCamSession itself is declared API_AVAILABLE(… visionos(2.1)), which is misleading because without input-port access the manual-connection path the class requires is unreachable. Expected behavior Either of the following would resolve this, in order of preference: Expose the missing API surface on visionOS. Make AVCaptureInputPort, AVCaptureInput.ports, and AVCaptureDeviceInput.portsWithMediaType:sourceDeviceType:sourceDevicePosition: available on visionOS so the documented iPadOS multi-cam pattern compiles and runs. AVCaptureMultiCamSession is already declared available — the supporting API surface should match. Allow two concurrent plain AVCaptureSessions to each own a distinct external AVCaptureDevice. Each session binds a different hardware device, and the current serialization appears to be a software policy rather than a hardware constraint (a powered hub has bandwidth for both). Document the limit explicitly and surface a clear error or interruption reason on the stalled session so apps can fail loudly instead of appearing to work. Actual behavior AVCaptureMultiCamSession advertises visionos(2.1) availability but the APIs required to wire its connections are marked unavailable on visionOS. Two concurrent AVCaptureSessions silently deliver frames to only one session; no error is reported on the other. There is no public alternative framework on visionOS for raw UVC access to work around this: IOUSBHost.framework — not present in XROS26.4.sdk DriverKit — not present in XROS26.4.sdk IOKit — ships a stub (IOKit.tbd); no public USB device interfaces CoreMediaIO — headers are an apinotes stub on visionOS ExternalAccessory — MFi-only; generic UVC devices don't enumerate This means there is no public path, AVFoundation or otherwise, for a third-party visionOS app to display two UVC cameras at once. Impact / use cases Apple Vision Pro is uniquely suited to multi-camera monitoring and capture workflows — spatial creators, broadcast/AV producers, multi-angle reference during immersive authoring, clinical and field-recording use cases, and apps that combine a primary UVC cinema camera with a secondary UVC reference/overview angle. iPadOS already supports this via AVCaptureMultiCamSession (demonstrated shipping by Camo Studio). The current visionOS limitation pushes these workflows back to iPad or macOS and undermines Vision Pro's positioning as a pro capture/monitor environment. References iPadOS reference implementation: Apple sample Displaying Video From Connected Devices + AVCaptureMultiCamSession with manual AVCaptureConnection wiring — works on iPadOS 18+ with two UVC cameras via a powered hub. Shipping precedent: Camo Studio — two simultaneous UVC cameras via USB hub on iPad — https://apps.apple.com/us/app/camo-studio-stream-record/id6450313385 visionOS 26.4 SDK headers cited above (AVCaptureInput.h, AVCaptureSession.h).