Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

iOS 17 camera capture assertions and issues
Hello, Starting in iOS 17, our application started having some issue publishing to our video session. More specifically the video capture seems to be broken in some, but not all sessions. What's troubling is that we're seeing that it fails consistently every 4 sessions. It also fails silently, without reporting any problems to the app. We only notice that there are no frames being rendered or sent to the remote devices. Here's what shows-up in the console: <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:253) - (err=-16453) Anyone seeing this? Any idea what could be the cause? Our sessions work perfectly on iOS16 and below. Thanks
5
1
1.9k
2w
PHObject.localIdentifier reliability
For a PHAsset in the same Photos library on the same device/Mac, what are the documented stability guarantees of PHObject.localIdentifier? Is it safe to persist and use for future PhotoKit operations in that same local library? Are there known cases where it can change or stop resolving? If a persisted localIdentifier no longer resolves but a persisted PHCloudIdentifier.archivalStringValue does resolve in the same library, is updating the stored local identifier from that cloud mapping the recommended recovery path? Thanks!
3
1
132
2w
Broadcast Extension deprecation in iOS 28
I saw that the Broadcast extension is deprecated in iOS 28, as well as the ReplayKit with nothing mentioned on what is going to be used going forward. I found out that the Screen Capture Kit is mentioned as replacement but is available for macOS only, even though some methods now are marked with iOS 28+ (https://developer.apple.com/documentation/screencapturekit). Will the ScreenCaptureKit be used for screen sharing going forward and replace the Broadcast Extension?
1
0
226
2w
Local PHAssetResource availability
Is there a way to determine whether a specific PHAssetResource original is locally available/downloaded without requesting it? If not, is the recommended approach to request the resource with PHAssetResourceRequestOptions.isNetworkAccessAllowed = false and handle networkAccessRequired error? Is that error the definitive signal that the requested resource is not locally available, or are there other errors that should be treated the same way? Thanks!
1
1
71
2w
Solving AVFoundation FigCaptureSourceRemote err=-17281 on iOS 26 — reliable workaround for repeated camera initialization
While working on a heart rate measurement app (photoplethysmography via camera), we faced systematic err=-17281 (FigCaptureSourceRemote) issues on real devices starting from iOS 17+, and the problem became more noticeable after iOS 26. The error often appeared during AVCaptureSession initialization or when restarting capture, especially under high frame rates (30-60 FPS) and frequent foreground/background transitions. Root cause (our understanding): Camera hardware/session not fully released after previous use Race conditions between session teardown and new setup Changes in AVFoundation capture pipeline in recent iOS versions Our solution: Instead of blocking delays, we implemented asynchronous retry logic with explicit hardware readiness check via AVCaptureDevice.lockForConfiguration().
3
0
525
2w
Smooth transition between cameras
In Apple's Camera app (set to "Photo" mode), when the system switches between different cameras, the transition is incredibly smooth. In particular, when getting too close to an object for the wide angle camera (due to the minimal focus distance), it automatically switches to the ultra-wide angle. When switching, the app transitions between the cameras very smoothly, by which I mean the position of the object in frame doesn't shift despite the physical location of the cameras being different. Is it possible to replicate this behaviour with the AVFoundation API? When creating a AVCaptureSession using a AVCaptureDevice with the builtInTripleCamera device type, the session automatically switches between the constituent cameras as expected. However, unlike Apple's Camera app, there is a very noticeable shift in perspective when switching. This is especially noticeable when switching between the WA and UWA cameras while zoomed in, sometimes resulting in the center of the frame being shifted all the way to the edge of the screen. I was able to somewhat replicate the smooth switching by setting activeVideoStabilizationMode of my AVCaptureConnection (with AVCaptureVideoDataOutput) to .cinematic, but this would introduce too much delay for a live preview, and none of the other available stabilization modes seemed to support this smooth transitioning. Is it possible to replicate the behaviour of the Camera app using the AVFoundation API?
2
0
139
2w
Feature request: Linear scene-referred preview for Bayer RAW capture
I would like to request a linear scene-referred preview stream for Bayer RAW workflows via AVCaptureVideoDataOutput or AVCaptureVideoPreviewLayer, without tone mapping, computational processing, or a baked-in rendered look. This is particularly important for third-party camera apps built around Bayer RAW workflows, including my app, Lampa Camera. In other words, it would be great to have a preview mode that closely matches the CIRAWFilter output for a Bayer DNG with the following configuration: rawFilter.baselineExposure = 0.0 rawFilter.shadowBias = 0.0 rawFilter.boostAmount = 0.0 rawFilter.localToneMapAmount = 0.0 Currently, the live preview comes from the processed camera feed. As a result, the viewfinder can differ noticeably from the final RAW-converted image in exposure, dynamic range, contrast, and sharpness. A Bayer RAW-compatible linear preview mode would make it possible to provide a reliable what-you-see-is-what-you-get shooting experience for RAW-based camera apps. Would Apple consider exposing such a preview mode in AVFoundation? Related Feedback Assistant report: FB22660803
3
1
156
2w
Portrait flag detection
PHAsset.mediaSubtypes.contains(.photoDepthEffect) exposes the portrait/depth-effect flag through PhotoKit. Is there a way to detect the same Photos “Portrait” classification when reading the image file directly using ImageIO? I've found that key 25 in kCGImagePropertyMakerAppleDictionary appears to correlate with the Portrait flag. Is the existence of this key a stable signal for that classification, or should I use another ImageIO signal? Thanks!
1
0
60
2w
photokit AI metadata
Is there a way to get AI metadata from photokit without having to run the algorithms again? I'd like to be able to get captions, featureprint, and optimal crops. I can use Vision and CoreML to generate these, but it'd be easier/better to read the info directly from Photokit.
1
0
38
2w
Setting up video and image capture pipeline creates internal errors in AVFoundation.
I have created code for iOS that allows me to start and stop video acquisition from a proprietary USB camera using AVFoundation's AVCaptureSession and AVCaptureDevice APIs. There is a start and stop method. The start method takes an argument to specify one of two formats that I use for my custom camera application. I can start the session and switch between formats all day without any errors. However, if I start and then stop the camera three times in a row, on the third invocation of start, I get errors in the console output and the CMSampleBuffers stop flowing to my callback. Additionally, once I get AVFoundation into this state, stoping the camera doesn't help. I have to kill the app and start over. Here are the errors. And below these, the code. I'm hoping someone who has experience with these errors or an engineer from Apple who knows the AVFoundation image capture pipeline code, can respond and tell me what I'm doing wrong. Thanks. <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:558) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:253) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:269) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:511) - (err=-16453) Capture session error: The operation could not be completed Capture session error: The operation could not be completed func start(for deviceFormat: String) async throws -> AnyPublisher<CMSampleBuffer, Swift.Error> { func configureCaptureDevice(with deviceFormat: String) throws { guard let format = formatDict[deviceFormat] else { throw Error.captureFormatNotFound } captureSession.beginConfiguration() defer { captureSession.commitConfiguration() } try captureDevice.lockForConfiguration() captureDeviceFormat = deviceFormat captureDevice.activeFormat = format captureDevice.unlockForConfiguration() } return try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Start capture session for \(deviceFormat): \(String(describing: captureSession))") // If we were already steaming camera images from a different mode, terminate that stream. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" do { // Re-configure with the new format; should be harmless if called with the currently configured format. try configureCaptureDevice(with: deviceFormat) // Return a new stream publisher for this invocation. bufferPublisher = PassthroughSubject<CMSampleBuffer, Swift.Error>() // If we are not currently running, start the image capture pipeline. if captureSession.isRunning == false { captureSession.startRunning() } continuation.resume(returning: bufferPublisher!.eraseToAnyPublisher()) } catch { logger.fault("Failed to start camera: \(error.localizedDescription)") continuation.resume(throwing: error) } } } } func stop() async throws { try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Stop capture session: \(String(describing: captureSession))") // The following invocation is synchronous and takes time to execute; // looks like a stall but you can ignore it as the MainActor is not blocked. captureSession.stopRunning() // Terminate the stream and reset our state. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" // Signal the caller that we are done here. continuation.resume() } } }
1
0
433
2w
Feedback on Apple Spatial Audio re-render behavior for Dolby Atmos music delivery — perspective from a working mix engineer
Hey everyone, quick disclaimer before jumping in - I used my LLM to structure this around notes/observations I've been taking the last several months. I apologize for the length but felt this was the best distillation of an important challenge my peers and I are facing in mixing music for the largest device/service segment of the listening community - Airpods Pro/Max via Apple Music. Thanks in advance for reading and any feedback you can offer! -Kyle I'm a professional mix engineer working primarily in contemporary pop, indie, and country. After 20+ years of working in stereo, I've started delivering Dolby Atmos ADM masters for Apple Music distribution. I want to share some specific observations about the Apple Spatial Audio re-render in the hope that it's useful to the team that owns this rendering pipeline — and to ask a few questions I haven't been able to find answered in public documentation. I recognize this sits at an unusual intersection of the developer platform and the Apple Music delivery side of the house, but since the rendering behavior is ultimately a platform-level decision, this felt like the right place to start. Background: the three-format problem When delivering an Atmos ADM master, a mixer effectively has to satisfy three distinct listening contexts simultaneously: Speaker playback (7.1.4 or similar) via the Dolby renderer Dolby binaural re-render (AC-4), as heard on TIDAL and Amazon — which respects the OFF/NEAR/MID/FAR binaural mode settings on beds and objects Apple Spatial Audio headphone re-render on Apple Music The first two have reasonably predictable translation. The third is where I'm running into consistent issues — and where I'd value any guidance Apple is able to share. The core issue: Apple's re-render discards binaural mode metadata As best I can tell from testing and from community documentation, Apple's pipeline ingests the ADM, creates an internal 7.1.4 render, and then applies its own proprietary binaural spatialization — one that does not reference the OFF/NEAR/MID/FAR binaural mode parameters embedded by the mixer. This is distinct from the Dolby AC-4 path, which does honor those settings. In practice, this means: Apple's re-render applies a consistent room character regardless of what the mixer has specified for individual elements Elements like lead vocals and kick/snare — which I'm routing through beds or objects with OFF or NEAR binaural settings specifically to preserve intimacy and punch — receive the same ambient room treatment as wider, more spacious elements The result on Apple Music has noticeably more perceived distance and "room" on transient-heavy and close-mic'd elements than either the speaker mix or the Dolby binaural render To be specific about the perceptual effect: the Apple re-render's virtual room introduces early reflections and a sense of speaker-to-listener distance that significantly undercuts the intimacy and impact of close elements. On a pop or country vocal, this is the difference between a performance that feels present and direct versus one that feels recessed in a listening space. On drums, transient attack is softened in a way that doesn't happen in any other delivery context for the same master. Questions for the team I'd be grateful for any clarity on the following: Is the behavior of ignoring OFF/NEAR/MID/FAR metadata intentional and permanent, or is it something that may change as the rendering pipeline evolves? Is there any mechanism — existing or planned — by which a mixer can influence the room character or "closeness" of elements in Apple's re-render, outside of object positioning metadata? Is there any documentation of how Apple's binaural spatialization layer translates object distance metadata (as opposed to binaural mode) — i.e., does Z-axis positioning in the Atmos object space affect perceived distance in the re-render? Is there a recommended workflow or set of delivery parameters that Apple's audio team considers optimal for music content specifically, as opposed to film/TV? Notes on the Audiomovers Binaural Renderer for Apple Music I'm aware of and have used the Audiomovers plugin, which I understand was developed in collaboration with Apple and accurately reflects the Apple Spatial re-render during session monitoring. It's a genuinely useful tool and has improved my ability to anticipate Apple's output. My questions above are about the underlying rendering behavior — not the monitoring workflow, which is solved. Why this matters for music specifically Film and TV post content has different expectations around spatialization — a consistent room or "cinema" quality to the binaural render is arguably appropriate for that material. For music, particularly in contemporary genres where the stereo mix is already highly produced and intimate, an added room layer competes with the mix's own space design and consistently pushes elements further from the listener than intended. I'd argue music content would benefit from a rendering mode with a more "dry" or near-field room character — and I suspect I'm not alone in this among working Atmos music mixers. I'm happy to provide specific A/B examples or additional technical detail if that's useful to anyone on the platform team. Thanks for reading.
0
1
229
2w
MediaDeviceExtension is not launched; audiomxd logs “missing required entitlement” (iPadOS 27)
I'm trying to implement a MediaDeviceExtension, but it is not being launched. I followed the documentation, but I see "missing required entitlement" in log, even though I've added the entitlement. My Setup com.apple.developer.media-device-extension on both the app and the extension: <key>com.apple.developer.media-device-extension</key> <array> <string>media-device-protocol.myradio</string> </array> The same id is in the extension’s UTExportedTypeDeclarations → UTTypeIdentifier (conforming to public.media-sharing-protocol) and in the Swift protocolType property. The app’s Info.plist has MDESupportedProtocols with the same id. EXExtensionPointIdentifier = com.apple.media-device-extension in EXAppExtensionAttributes. The capability is enabled on both App IDs in Certificates, Identifiers & Profiles. Problem Opening an AVRoutePickerView does not call startDeviceDiscovery(), and no device appears in the picker. Device log: -FigCustomEndpointManager- manager_shouldAllowDiscoveryForProtocol: Protocol media-device-protocol.myradio has never been activated - allowing discovery for first-time use -FigCustomEndpointManager- manager_SetDiscoveryMode: Failed to launch system casting instance for protocol media-device-protocol.myradio -MXAppExtensionMonitor- -[MXAppExtensionInstance launchExtensionRequiring:]: Extension with identifier: <_EXExtensionIdentity: 0x…> is missing required entitlement, dropping! -MXSystemMediaCasting- -[MXSystemCastingExtensionInstance launch]: Failed to launch <SystemMediaCastingExtension<MyRadio:media-device-protocol.myradio>: …, <EXExtensionProcess: (null) PID: 0>, 0> Question What entitlement is MXAppExtensionMonitor checking for when launching a SystemMediaCastingExtension? It is not named in the log. If an entitlement beyond com.apple.developer.media-device-extension is required, how is it obtained? FB23043277
1
1
197
2w
iOS 17 camera capture assertions and issues
Hello, Starting in iOS 17, our application started having some issue publishing to our video session. More specifically the video capture seems to be broken in some, but not all sessions. What's troubling is that we're seeing that it fails consistently every 4 sessions. It also fails silently, without reporting any problems to the app. We only notice that there are no frames being rendered or sent to the remote devices. Here's what shows-up in the console: <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:253) - (err=-16453) Anyone seeing this? Any idea what could be the cause? Our sessions work perfectly on iOS16 and below. Thanks
Replies
5
Boosts
1
Views
1.9k
Activity
2w
15 minutes to go
Thanks for your great questions. We'll be wrapping up our live feedback session in 15 minutes. Please get your questions in soon.
Replies
0
Boosts
0
Views
54
Activity
2w
PHPhotoLibraryPersistentChangesObserver description
What is the intended purpose of the new PHPhotoLibraryPersistentChangesObserver? I currently use PHPhotoLibraryChangeObserver as a trigger, then call fetchPersistentChanges(since:). What behavior or guarantees differ when using PHPhotoLibraryPersistentChangesObserver instead? Thanks!
Replies
3
Boosts
0
Views
102
Activity
2w
PHObject.localIdentifier reliability
For a PHAsset in the same Photos library on the same device/Mac, what are the documented stability guarantees of PHObject.localIdentifier? Is it safe to persist and use for future PhotoKit operations in that same local library? Are there known cases where it can change or stop resolving? If a persisted localIdentifier no longer resolves but a persisted PHCloudIdentifier.archivalStringValue does resolve in the same library, is updating the stored local identifier from that cloud mapping the recommended recovery path? Thanks!
Replies
3
Boosts
1
Views
132
Activity
2w
Deep link to PHAsset in Photos app?
My app displays photos from user's photo library. I want to provide a way to deep link to that photo in the Photos app (open the photo in the Photos app). Is there any way to do this?
Replies
4
Boosts
2
Views
138
Activity
2w
Broadcast Extension deprecation in iOS 28
I saw that the Broadcast extension is deprecated in iOS 28, as well as the ReplayKit with nothing mentioned on what is going to be used going forward. I found out that the Screen Capture Kit is mentioned as replacement but is available for macOS only, even though some methods now are marked with iOS 28+ (https://developer.apple.com/documentation/screencapturekit). Will the ScreenCaptureKit be used for screen sharing going forward and replace the Broadcast Extension?
Replies
1
Boosts
0
Views
226
Activity
2w
Testing and purging optimized-storage assets
Is there a supported way for developers to purge locally downloaded Photos originals/resources in order to force an asset/resource into a non-downloaded iCloud-only state? Basically simulate "Optimize Mac Storage" behavior on demand for testing. Thanks!
Replies
1
Boosts
0
Views
64
Activity
2w
Local PHAssetResource availability
Is there a way to determine whether a specific PHAssetResource original is locally available/downloaded without requesting it? If not, is the recommended approach to request the resource with PHAssetResourceRequestOptions.isNetworkAccessAllowed = false and handle networkAccessRequired error? Is that error the definitive signal that the requested resource is not locally available, or are there other errors that should be treated the same way? Thanks!
Replies
1
Boosts
1
Views
71
Activity
2w
Identifying Personal vs. Shared Library Assets in Photos.framework
Can Photos.framework distinguish between assets that are stored in the user’s Personal Library and assets that are stored in the Shared Library? If not, is there a planned API that would allow developers to identify the library membership of a PHAsset?
Replies
3
Boosts
0
Views
93
Activity
2w
Solving AVFoundation FigCaptureSourceRemote err=-17281 on iOS 26 — reliable workaround for repeated camera initialization
While working on a heart rate measurement app (photoplethysmography via camera), we faced systematic err=-17281 (FigCaptureSourceRemote) issues on real devices starting from iOS 17+, and the problem became more noticeable after iOS 26. The error often appeared during AVCaptureSession initialization or when restarting capture, especially under high frame rates (30-60 FPS) and frequent foreground/background transitions. Root cause (our understanding): Camera hardware/session not fully released after previous use Race conditions between session teardown and new setup Changes in AVFoundation capture pipeline in recent iOS versions Our solution: Instead of blocking delays, we implemented asynchronous retry logic with explicit hardware readiness check via AVCaptureDevice.lockForConfiguration().
Replies
3
Boosts
0
Views
525
Activity
2w
Smooth transition between cameras
In Apple's Camera app (set to "Photo" mode), when the system switches between different cameras, the transition is incredibly smooth. In particular, when getting too close to an object for the wide angle camera (due to the minimal focus distance), it automatically switches to the ultra-wide angle. When switching, the app transitions between the cameras very smoothly, by which I mean the position of the object in frame doesn't shift despite the physical location of the cameras being different. Is it possible to replicate this behaviour with the AVFoundation API? When creating a AVCaptureSession using a AVCaptureDevice with the builtInTripleCamera device type, the session automatically switches between the constituent cameras as expected. However, unlike Apple's Camera app, there is a very noticeable shift in perspective when switching. This is especially noticeable when switching between the WA and UWA cameras while zoomed in, sometimes resulting in the center of the frame being shifted all the way to the edge of the screen. I was able to somewhat replicate the smooth switching by setting activeVideoStabilizationMode of my AVCaptureConnection (with AVCaptureVideoDataOutput) to .cinematic, but this would introduce too much delay for a live preview, and none of the other available stabilization modes seemed to support this smooth transitioning. Is it possible to replicate the behaviour of the Camera app using the AVFoundation API?
Replies
2
Boosts
0
Views
139
Activity
2w
Feature request: Linear scene-referred preview for Bayer RAW capture
I would like to request a linear scene-referred preview stream for Bayer RAW workflows via AVCaptureVideoDataOutput or AVCaptureVideoPreviewLayer, without tone mapping, computational processing, or a baked-in rendered look. This is particularly important for third-party camera apps built around Bayer RAW workflows, including my app, Lampa Camera. In other words, it would be great to have a preview mode that closely matches the CIRAWFilter output for a Bayer DNG with the following configuration: rawFilter.baselineExposure = 0.0 rawFilter.shadowBias = 0.0 rawFilter.boostAmount = 0.0 rawFilter.localToneMapAmount = 0.0 Currently, the live preview comes from the processed camera feed. As a result, the viewfinder can differ noticeably from the final RAW-converted image in exposure, dynamic range, contrast, and sharpness. A Bayer RAW-compatible linear preview mode would make it possible to provide a reliable what-you-see-is-what-you-get shooting experience for RAW-based camera apps. Would Apple consider exposing such a preview mode in AVFoundation? Related Feedback Assistant report: FB22660803
Replies
3
Boosts
1
Views
156
Activity
2w
ARKit multi-frame capture question
In a multi-frame 360° stitching pipeline with ARKit, what's the recommended way to lock exposure and white balance consistently across frames, and is AVFoundation-first better than ARKit-first for this?
Replies
1
Boosts
0
Views
33
Activity
2w
IOS27beta screenshot error
After taking a screenshot, click the picture to crop and save it. Only the uncropped content will be displayed, not the cropped content.
Replies
1
Boosts
0
Views
91
Activity
2w
Portrait flag detection
PHAsset.mediaSubtypes.contains(.photoDepthEffect) exposes the portrait/depth-effect flag through PhotoKit. Is there a way to detect the same Photos “Portrait” classification when reading the image file directly using ImageIO? I've found that key 25 in kCGImagePropertyMakerAppleDictionary appears to correlate with the Portrait flag. Is the existence of this key a stable signal for that classification, or should I use another ImageIO signal? Thanks!
Replies
1
Boosts
0
Views
60
Activity
2w
photokit AI metadata
Is there a way to get AI metadata from photokit without having to run the algorithms again? I'd like to be able to get captions, featureprint, and optimal crops. I can use Vision and CoreML to generate these, but it'd be easier/better to read the info directly from Photokit.
Replies
1
Boosts
0
Views
38
Activity
2w
Setting up video and image capture pipeline creates internal errors in AVFoundation.
I have created code for iOS that allows me to start and stop video acquisition from a proprietary USB camera using AVFoundation's AVCaptureSession and AVCaptureDevice APIs. There is a start and stop method. The start method takes an argument to specify one of two formats that I use for my custom camera application. I can start the session and switch between formats all day without any errors. However, if I start and then stop the camera three times in a row, on the third invocation of start, I get errors in the console output and the CMSampleBuffers stop flowing to my callback. Additionally, once I get AVFoundation into this state, stoping the camera doesn't help. I have to kill the app and start over. Here are the errors. And below these, the code. I'm hoping someone who has experience with these errors or an engineer from Apple who knows the AVFoundation image capture pipeline code, can respond and tell me what I'm doing wrong. Thanks. <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:558) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:253) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:269) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:511) - (err=-16453) Capture session error: The operation could not be completed Capture session error: The operation could not be completed func start(for deviceFormat: String) async throws -> AnyPublisher<CMSampleBuffer, Swift.Error> { func configureCaptureDevice(with deviceFormat: String) throws { guard let format = formatDict[deviceFormat] else { throw Error.captureFormatNotFound } captureSession.beginConfiguration() defer { captureSession.commitConfiguration() } try captureDevice.lockForConfiguration() captureDeviceFormat = deviceFormat captureDevice.activeFormat = format captureDevice.unlockForConfiguration() } return try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Start capture session for \(deviceFormat): \(String(describing: captureSession))") // If we were already steaming camera images from a different mode, terminate that stream. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" do { // Re-configure with the new format; should be harmless if called with the currently configured format. try configureCaptureDevice(with: deviceFormat) // Return a new stream publisher for this invocation. bufferPublisher = PassthroughSubject<CMSampleBuffer, Swift.Error>() // If we are not currently running, start the image capture pipeline. if captureSession.isRunning == false { captureSession.startRunning() } continuation.resume(returning: bufferPublisher!.eraseToAnyPublisher()) } catch { logger.fault("Failed to start camera: \(error.localizedDescription)") continuation.resume(throwing: error) } } } } func stop() async throws { try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Stop capture session: \(String(describing: captureSession))") // The following invocation is synchronous and takes time to execute; // looks like a stall but you can ignore it as the MainActor is not blocked. captureSession.stopRunning() // Terminate the stream and reset our state. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" // Signal the caller that we are done here. continuation.resume() } } }
Replies
1
Boosts
0
Views
433
Activity
2w
Feedback on Apple Spatial Audio re-render behavior for Dolby Atmos music delivery — perspective from a working mix engineer
Hey everyone, quick disclaimer before jumping in - I used my LLM to structure this around notes/observations I've been taking the last several months. I apologize for the length but felt this was the best distillation of an important challenge my peers and I are facing in mixing music for the largest device/service segment of the listening community - Airpods Pro/Max via Apple Music. Thanks in advance for reading and any feedback you can offer! -Kyle I'm a professional mix engineer working primarily in contemporary pop, indie, and country. After 20+ years of working in stereo, I've started delivering Dolby Atmos ADM masters for Apple Music distribution. I want to share some specific observations about the Apple Spatial Audio re-render in the hope that it's useful to the team that owns this rendering pipeline — and to ask a few questions I haven't been able to find answered in public documentation. I recognize this sits at an unusual intersection of the developer platform and the Apple Music delivery side of the house, but since the rendering behavior is ultimately a platform-level decision, this felt like the right place to start. Background: the three-format problem When delivering an Atmos ADM master, a mixer effectively has to satisfy three distinct listening contexts simultaneously: Speaker playback (7.1.4 or similar) via the Dolby renderer Dolby binaural re-render (AC-4), as heard on TIDAL and Amazon — which respects the OFF/NEAR/MID/FAR binaural mode settings on beds and objects Apple Spatial Audio headphone re-render on Apple Music The first two have reasonably predictable translation. The third is where I'm running into consistent issues — and where I'd value any guidance Apple is able to share. The core issue: Apple's re-render discards binaural mode metadata As best I can tell from testing and from community documentation, Apple's pipeline ingests the ADM, creates an internal 7.1.4 render, and then applies its own proprietary binaural spatialization — one that does not reference the OFF/NEAR/MID/FAR binaural mode parameters embedded by the mixer. This is distinct from the Dolby AC-4 path, which does honor those settings. In practice, this means: Apple's re-render applies a consistent room character regardless of what the mixer has specified for individual elements Elements like lead vocals and kick/snare — which I'm routing through beds or objects with OFF or NEAR binaural settings specifically to preserve intimacy and punch — receive the same ambient room treatment as wider, more spacious elements The result on Apple Music has noticeably more perceived distance and "room" on transient-heavy and close-mic'd elements than either the speaker mix or the Dolby binaural render To be specific about the perceptual effect: the Apple re-render's virtual room introduces early reflections and a sense of speaker-to-listener distance that significantly undercuts the intimacy and impact of close elements. On a pop or country vocal, this is the difference between a performance that feels present and direct versus one that feels recessed in a listening space. On drums, transient attack is softened in a way that doesn't happen in any other delivery context for the same master. Questions for the team I'd be grateful for any clarity on the following: Is the behavior of ignoring OFF/NEAR/MID/FAR metadata intentional and permanent, or is it something that may change as the rendering pipeline evolves? Is there any mechanism — existing or planned — by which a mixer can influence the room character or "closeness" of elements in Apple's re-render, outside of object positioning metadata? Is there any documentation of how Apple's binaural spatialization layer translates object distance metadata (as opposed to binaural mode) — i.e., does Z-axis positioning in the Atmos object space affect perceived distance in the re-render? Is there a recommended workflow or set of delivery parameters that Apple's audio team considers optimal for music content specifically, as opposed to film/TV? Notes on the Audiomovers Binaural Renderer for Apple Music I'm aware of and have used the Audiomovers plugin, which I understand was developed in collaboration with Apple and accurately reflects the Apple Spatial re-render during session monitoring. It's a genuinely useful tool and has improved my ability to anticipate Apple's output. My questions above are about the underlying rendering behavior — not the monitoring workflow, which is solved. Why this matters for music specifically Film and TV post content has different expectations around spatialization — a consistent room or "cinema" quality to the binaural render is arguably appropriate for that material. For music, particularly in contemporary genres where the stereo mix is already highly produced and intimate, an added room layer competes with the mix's own space design and consistently pushes elements further from the listener than intended. I'd argue music content would benefit from a rendering mode with a more "dry" or near-field room character — and I suspect I'm not alone in this among working Atmos music mixers. I'm happy to provide specific A/B examples or additional technical detail if that's useful to anyone on the platform team. Thanks for reading.
Replies
0
Boosts
1
Views
229
Activity
2w
Microphone Gain on Safari/MacOS
The microphone gain for our app running on Safari/MacOS is substantially lower than with other browsers. What might account for that and what is the preferred method for a web app to control microphone gain.
Replies
4
Boosts
1
Views
159
Activity
2w
MediaDeviceExtension is not launched; audiomxd logs “missing required entitlement” (iPadOS 27)
I'm trying to implement a MediaDeviceExtension, but it is not being launched. I followed the documentation, but I see "missing required entitlement" in log, even though I've added the entitlement. My Setup com.apple.developer.media-device-extension on both the app and the extension: <key>com.apple.developer.media-device-extension</key> <array> <string>media-device-protocol.myradio</string> </array> The same id is in the extension’s UTExportedTypeDeclarations → UTTypeIdentifier (conforming to public.media-sharing-protocol) and in the Swift protocolType property. The app’s Info.plist has MDESupportedProtocols with the same id. EXExtensionPointIdentifier = com.apple.media-device-extension in EXAppExtensionAttributes. The capability is enabled on both App IDs in Certificates, Identifiers & Profiles. Problem Opening an AVRoutePickerView does not call startDeviceDiscovery(), and no device appears in the picker. Device log: -FigCustomEndpointManager- manager_shouldAllowDiscoveryForProtocol: Protocol media-device-protocol.myradio has never been activated - allowing discovery for first-time use -FigCustomEndpointManager- manager_SetDiscoveryMode: Failed to launch system casting instance for protocol media-device-protocol.myradio -MXAppExtensionMonitor- -[MXAppExtensionInstance launchExtensionRequiring:]: Extension with identifier: <_EXExtensionIdentity: 0x…> is missing required entitlement, dropping! -MXSystemMediaCasting- -[MXSystemCastingExtensionInstance launch]: Failed to launch <SystemMediaCastingExtension<MyRadio:media-device-protocol.myradio>: …, <EXExtensionProcess: (null) PID: 0>, 0> Question What entitlement is MXAppExtensionMonitor checking for when launching a SystemMediaCastingExtension? It is not named in the log. If an entitlement beyond com.apple.developer.media-device-extension is required, how is it obtained? FB23043277
Replies
1
Boosts
1
Views
197
Activity
2w