Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

MediaDeviceExtension is not launched; audiomxd logs “missing required entitlement” (iPadOS 27)
I'm trying to implement a MediaDeviceExtension, but it is not being launched. I followed the documentation, but I see "missing required entitlement" in log, even though I've added the entitlement. My Setup com.apple.developer.media-device-extension on both the app and the extension: <key>com.apple.developer.media-device-extension</key> <array> <string>media-device-protocol.myradio</string> </array> The same id is in the extension’s UTExportedTypeDeclarations → UTTypeIdentifier (conforming to public.media-sharing-protocol) and in the Swift protocolType property. The app’s Info.plist has MDESupportedProtocols with the same id. EXExtensionPointIdentifier = com.apple.media-device-extension in EXAppExtensionAttributes. The capability is enabled on both App IDs in Certificates, Identifiers & Profiles. Problem Opening an AVRoutePickerView does not call startDeviceDiscovery(), and no device appears in the picker. Device log: -FigCustomEndpointManager- manager_shouldAllowDiscoveryForProtocol: Protocol media-device-protocol.myradio has never been activated - allowing discovery for first-time use -FigCustomEndpointManager- manager_SetDiscoveryMode: Failed to launch system casting instance for protocol media-device-protocol.myradio -MXAppExtensionMonitor- -[MXAppExtensionInstance launchExtensionRequiring:]: Extension with identifier: <_EXExtensionIdentity: 0x…> is missing required entitlement, dropping! -MXSystemMediaCasting- -[MXSystemCastingExtensionInstance launch]: Failed to launch <SystemMediaCastingExtension<MyRadio:media-device-protocol.myradio>: …, <EXExtensionProcess: (null) PID: 0>, 0> Question What entitlement is MXAppExtensionMonitor checking for when launching a SystemMediaCastingExtension? It is not named in the log. If an entitlement beyond com.apple.developer.media-device-extension is required, how is it obtained? FB23043277
1
1
198
2w
Support for webm playback and processing
Hi, Safari supports playing back webm files (such as those recorded by Chrome/Firefox browsers with the MediaRecorder API), but our native app is unable to play these back with AVPlayer. Is there a supported way to do this? Right now we have to use a WebView for this! Would also love to ExtAudioFileURLOpen or process these files with AVAudioEngine. Have I missed something obvious here?
4
0
140
2w
Auv3 Synth Zippering
I am running into zippering and zipper storms in making my Auv3 synth. Do you have advice? It is a big synthesizer, big cinematic sounds, big presets, dual mode, I am mostly hitting zipper storms when trying to play fast on big preset recipes. Is there something I need to know to avoid zippering?
1
0
112
2w
RemoteMediaSession device add/remove
Are there plans to extend the session or MediaDevice capabilities to be able to add and remove speakers via the Lock Screen and Control Center? For example, user may have multiple speakers, they start playback on a subset of them and would like to add other speakers or remove some from the session through Control Center.
1
0
144
2w
My biggest priorities after 9 months of shipping Albums for macOS
Hi everyone! Adam here, the developer of Albums. Bummed not to get to have our yearly WebEx reunion where I beg y’all to add an endpoint to the Apple Music API to allow deleting items from a user’s library. But that’s not what this thread is about. Back in October, thanks to your team’s multi-year efforts, I was finally able to ship Albums for macOS. It’s been a genuine dream come true to use it on the platform I always envisioned it on, and I hear from users all the time who feel the same way. I know it was a significant engineering effort, and I’m genuinely very grateful. I’ll be judicious with my time here (I’d love to chat in more detail about my adventures in MusicKit on the Mac somehow, sometime). These are the main things on my wishlist for MusicKit on the Mac. There are three main things I hear from users that my app “can’t do” that can be dealbreakers for them. The first two are AirPlay support and a volume slider. Users are only able to AirPlay using the control center utility, because AVRoutePickerView does not work with ApplicationMusicPlayer (FB13934910). Relatedly, the MPVolumeView does not work with ApplicationMusicPlayer (FB21042385), so I can’t allow users app-specific volume control for my app. The lack of those two things is a real detriment to my app being able to be taken seriously alongside all of the other music player apps on the platform for which those features are table stakes. I know there’s a challenge here given the playback actually happens in the subprocess, but hoping some progress can be made here. The other issue is that library tracks played in ApplicationMusicPlayer do not update the last played date or play count in Music.app or in the user’s iCloud Music Library (FB17675148). Some people refuse to use the app for that reason, and I can’t say I blame them. I’ve only been able to test this briefly in Golden Gate, but it seems like this is still the case. Are you able to share anything about your work on the music library in macOS this year? Thank you again for all your work on MusicKit! I’m planning to get the last of the load-bearing MediaPlayer code out of my codebase later this year. Hooray!
1
1
156
2w
Audio Prioritization
Is there anyway that we as users can set apps to not be able to play audio? There have been times that I want to listen to music and then scroll specific social media apps, and the social media apps keep interrupt the audio. I made a feedback related to this last year: FB18494861
0
0
109
2w
microphone level monitoring
My macOS app provides real-time microphone level monitoring, and I also have a macOS widget that shows recent audio level status. Since WidgetKit widgets are snapshot/timeline based rather than continuously live SwiftUI views, is there any new capability in macOS 27 that allows a widget to show more responsive or animated audio-level changes? If true real-time animation is still not supported in widgets, what is Apple’s recommended approach for displaying frequently changing signal levels, such as audio meters, in a macOS widget?
2
0
180
2w
CATaps
I’ve noticed that CATap appears to scale the stereo/mono mixdown based on the number of channels in the output audio device. For example, if the selected output device has 4 channels, the tapped audio is reduced by half. This behavior does not seem to be documented anywhere, but I’m assuming it may have been done to reduce the chance of clipping when downmixing. The issue is that users are complaining that the audio level is too low. Is there any workaround for this that does not involve tapping every audio device separately and doing my own mixdown? Ideally, I’d like to keep using the system tap/mix behavior but avoid the automatic level reduction, or at least compensate for it in a reliable way.
1
0
107
2w
What's the recommended way for a Mac app to receive audio from other apps in 2026?
My app allows users to transcribe and translate audio from other apps running on your Mac. I currently use ScreenCaptureKit to do this, and just ignore the video stream and only use the audio stream. This works, but it's awkward because it requires I ask users for screen recording permission, even though all I want is the audio, not the screen. Is there a better way to do this in modern macOS releases? Thanks!
1
0
101
2w
AudioServerPlugin AddDeviceClient/RemoveDeviceClient calls.
I added logging in an AudioServerPlugin to trace the arrival and removal of a particular client in the AddDeviceClient() and RemoveDeviceClient() methods. To my surprise the app was removed and added as a client often, even when it was in a steady state, for example when using the Audio MIDI Setup to change sample rates. Is this expected behavior, and if so, are there any explanations about this.
0
0
102
2w
Optimization of multiple audio track synchronization.
I have an app where multiple audio files of the same length are played and they need to be in sync to produce the full song. My current setup syncs each track to the same position every time the user pauses or plays the song. However it is super laggy when the Audio is played either for the first time or when unpausing/pausing. What's the best way to do this?
2
0
124
2w
How to Seamlessly Handle FIDO QR Codes in Your iOS App
When scanning a FIDO QR code within an iOS app—whether using a custom AVFoundation (AVCaptureSession) implementation or DataScannerViewController—the system displays a native OS confirmation prompt. However, scanning the same QR code using the native system Code Scanner bypasses this prompt entirely. As a developer: Is there a way to suppress or avoid this native prompt when using custom in-app scanners? Alternatively, can I programmatically invoke the system Code Scanner directly from my app and have it deep-link back to the app once the scan is complete?
0
0
116
2w
Displaying a processed image from AVCaptureVideoDataOutput in a swiftUI view?
What is the recommended way of showing a processed image from AVCaptureVideoDataOutput in a swiftUI view? Currently my chain is AVCaptureVideoDataOutput(SampleBuffer) -> CIImage -> CIFilters -> createCGImage from processed CIImage -> Create swiftUI 'Image' from CGIImage, that is in a view Is there a better way to go from AVCaptureVideoDataOutput to a swiftUI view, with image processing?
2
0
148
2w
Offscreen drawing to generate video content
We develop software for video broadcasting, including for sporting events, where animated scoreboards play a key role. macOS offers excellent features for such animations—such as CAAnimations—but this holds true only as long as the animations run on visible screens. Distributing the video signal (e.g., via NDI) requires access to the CVPixelBuffers of the individual frames. Currently, we generate the animations on an external screen and create the pixel buffers using ScreenCaptureKit. We are unaware of any way to obtain these pixel buffers without relying on such—strictly speaking, unnecessary—external screens (or multiple screens). If the window is created offscreen, it updates only once per second, which is unusable. Are there alternatives to external screens? If not, why can’t we create a virtual offscreen device for such use cases—one where we specify the required frame rate and rendering frequency—to generate the necessary video frames? This would also be helpful for HTML-based overlays, which are becoming increasingly popular; currently, these are also rendered only once per second when offscreen.
1
0
202
2w
QuickTime Player will not playback with MediaExtention
Hi Everyone! I am writing a media extension to playback old "Amiga" ANIM files, as a test project. I have build the following objects: MEFormatReader METrackReader MESampleCursor The subclassed objects seem fairly straightforward. Rather than create a Video Decoder, I followed the instructions in the SampleCursor object header, and use the function: loadSampleBufferContainingSamplesToEndCursor:completionHandler: to deliver an BGRA 8-bit image. This works great AVPlayer object in my Swift based test app, but QuickTime Player will not actually play the movie. If I scrub on QuickTime Player's timeline, I can watch the movie just fine. But if I click on the "Play" button. Nothing happens. For fun, I created a custom pixel type, and implemented a MEVideoDecoder object. This also works with my AVPlayer test app, but again, same problem with QuickTime Player. I have even generated a JPEG image in the SampleCursor, and that fails too. I am stumped on this. I do not see any way to have QuickTime Player work properly with my MediaExtension, nor is there any documentation as to what to do. Suggestions? bob
1
0
302
2w
Automatic sign-in on tvOS
I have a streaming video app in the App Store. I would like to use the automatic sign in feature for tvOS that was introduced in WWDC '25. However, it requires the com.apple.developer.video-subscriber-single-sign-on entitlement. I'm told that I have to join the video partnership program to access that entitlement. I filled out the forms to join the partnership program, but was ghosted. Would you consider making this feature available to everyone, not just those in the video partnership program? It is frustrating that a new and useful feature isn't actually available.
0
0
102
2w
Apple AUGraphicEQ dead-lock
Hi, I encounter a dead-lock with Apple AUGraphicEQ. I have attached backtrace: thread #44, name = 'com.apple.audio.toolbox.AUScheduledParameterRefresher' frame #0: 0x0000000182059bb0 libsystem_kernel.dylib`semaphore_wait_trap + 8 frame #1: 0x000000018e9e7e00 caulk`caulk::semaphore::timed_wait(double) + 224 frame #2: 0x000000018e9e7cac caulk`caulk::concurrent::details::worker_thread::run() + 32 frame #3: 0x000000018e9e794c caulk`void* caulk::thread_proxy<std::__1::tuple<caulk::thread::attributes, void (caulk::concurrent::details::worker_thread::*)(), std::__1::tuple<caulk::concurrent::details::worker_thread*>>>(void*) + 96 frame #4: 0x000000018209dc58 libsystem_pthread.dylib`_pthread_start + 136 regards, Joël
2
0
528
2w
MediaDeviceExtension is not launched; audiomxd logs “missing required entitlement” (iPadOS 27)
I'm trying to implement a MediaDeviceExtension, but it is not being launched. I followed the documentation, but I see "missing required entitlement" in log, even though I've added the entitlement. My Setup com.apple.developer.media-device-extension on both the app and the extension: <key>com.apple.developer.media-device-extension</key> <array> <string>media-device-protocol.myradio</string> </array> The same id is in the extension’s UTExportedTypeDeclarations → UTTypeIdentifier (conforming to public.media-sharing-protocol) and in the Swift protocolType property. The app’s Info.plist has MDESupportedProtocols with the same id. EXExtensionPointIdentifier = com.apple.media-device-extension in EXAppExtensionAttributes. The capability is enabled on both App IDs in Certificates, Identifiers & Profiles. Problem Opening an AVRoutePickerView does not call startDeviceDiscovery(), and no device appears in the picker. Device log: -FigCustomEndpointManager- manager_shouldAllowDiscoveryForProtocol: Protocol media-device-protocol.myradio has never been activated - allowing discovery for first-time use -FigCustomEndpointManager- manager_SetDiscoveryMode: Failed to launch system casting instance for protocol media-device-protocol.myradio -MXAppExtensionMonitor- -[MXAppExtensionInstance launchExtensionRequiring:]: Extension with identifier: <_EXExtensionIdentity: 0x…> is missing required entitlement, dropping! -MXSystemMediaCasting- -[MXSystemCastingExtensionInstance launch]: Failed to launch <SystemMediaCastingExtension<MyRadio:media-device-protocol.myradio>: …, <EXExtensionProcess: (null) PID: 0>, 0> Question What entitlement is MXAppExtensionMonitor checking for when launching a SystemMediaCastingExtension? It is not named in the log. If an entitlement beyond com.apple.developer.media-device-extension is required, how is it obtained? FB23043277
Replies
1
Boosts
1
Views
198
Activity
2w
Support for webm playback and processing
Hi, Safari supports playing back webm files (such as those recorded by Chrome/Firefox browsers with the MediaRecorder API), but our native app is unable to play these back with AVPlayer. Is there a supported way to do this? Right now we have to use a WebView for this! Would also love to ExtAudioFileURLOpen or process these files with AVAudioEngine. Have I missed something obvious here?
Replies
4
Boosts
0
Views
140
Activity
2w
Auv3 Synth Zippering
I am running into zippering and zipper storms in making my Auv3 synth. Do you have advice? It is a big synthesizer, big cinematic sounds, big presets, dual mode, I am mostly hitting zipper storms when trying to play fast on big preset recipes. Is there something I need to know to avoid zippering?
Replies
1
Boosts
0
Views
112
Activity
2w
RemoteMediaSession device add/remove
Are there plans to extend the session or MediaDevice capabilities to be able to add and remove speakers via the Lock Screen and Control Center? For example, user may have multiple speakers, they start playback on a subset of them and would like to add other speakers or remove some from the session through Control Center.
Replies
1
Boosts
0
Views
144
Activity
2w
My biggest priorities after 9 months of shipping Albums for macOS
Hi everyone! Adam here, the developer of Albums. Bummed not to get to have our yearly WebEx reunion where I beg y’all to add an endpoint to the Apple Music API to allow deleting items from a user’s library. But that’s not what this thread is about. Back in October, thanks to your team’s multi-year efforts, I was finally able to ship Albums for macOS. It’s been a genuine dream come true to use it on the platform I always envisioned it on, and I hear from users all the time who feel the same way. I know it was a significant engineering effort, and I’m genuinely very grateful. I’ll be judicious with my time here (I’d love to chat in more detail about my adventures in MusicKit on the Mac somehow, sometime). These are the main things on my wishlist for MusicKit on the Mac. There are three main things I hear from users that my app “can’t do” that can be dealbreakers for them. The first two are AirPlay support and a volume slider. Users are only able to AirPlay using the control center utility, because AVRoutePickerView does not work with ApplicationMusicPlayer (FB13934910). Relatedly, the MPVolumeView does not work with ApplicationMusicPlayer (FB21042385), so I can’t allow users app-specific volume control for my app. The lack of those two things is a real detriment to my app being able to be taken seriously alongside all of the other music player apps on the platform for which those features are table stakes. I know there’s a challenge here given the playback actually happens in the subprocess, but hoping some progress can be made here. The other issue is that library tracks played in ApplicationMusicPlayer do not update the last played date or play count in Music.app or in the user’s iCloud Music Library (FB17675148). Some people refuse to use the app for that reason, and I can’t say I blame them. I’ve only been able to test this briefly in Golden Gate, but it seems like this is still the case. Are you able to share anything about your work on the music library in macOS this year? Thank you again for all your work on MusicKit! I’m planning to get the last of the load-bearing MediaPlayer code out of my codebase later this year. Hooray!
Replies
1
Boosts
1
Views
156
Activity
2w
Audio Prioritization
Is there anyway that we as users can set apps to not be able to play audio? There have been times that I want to listen to music and then scroll specific social media apps, and the social media apps keep interrupt the audio. I made a feedback related to this last year: FB18494861
Replies
0
Boosts
0
Views
109
Activity
2w
microphone level monitoring
My macOS app provides real-time microphone level monitoring, and I also have a macOS widget that shows recent audio level status. Since WidgetKit widgets are snapshot/timeline based rather than continuously live SwiftUI views, is there any new capability in macOS 27 that allows a widget to show more responsive or animated audio-level changes? If true real-time animation is still not supported in widgets, what is Apple’s recommended approach for displaying frequently changing signal levels, such as audio meters, in a macOS widget?
Replies
2
Boosts
0
Views
180
Activity
2w
CATaps
I’ve noticed that CATap appears to scale the stereo/mono mixdown based on the number of channels in the output audio device. For example, if the selected output device has 4 channels, the tapped audio is reduced by half. This behavior does not seem to be documented anywhere, but I’m assuming it may have been done to reduce the chance of clipping when downmixing. The issue is that users are complaining that the audio level is too low. Is there any workaround for this that does not involve tapping every audio device separately and doing my own mixdown? Ideally, I’d like to keep using the system tap/mix behavior but avoid the automatic level reduction, or at least compensate for it in a reliable way.
Replies
1
Boosts
0
Views
107
Activity
2w
What's the recommended way for a Mac app to receive audio from other apps in 2026?
My app allows users to transcribe and translate audio from other apps running on your Mac. I currently use ScreenCaptureKit to do this, and just ignore the video stream and only use the audio stream. This works, but it's awkward because it requires I ask users for screen recording permission, even though all I want is the audio, not the screen. Is there a better way to do this in modern macOS releases? Thanks!
Replies
1
Boosts
0
Views
101
Activity
2w
New text dictation system available to 3rd party apps?
In the keynote (or maybe SOTU) they demo'd a new text dictation system. I'm wondering if this is being added to the Speech framework as a new transcription mechanism available to 3rd party apps. I couldn't find any videos referring to updates to the Speech framework.
Replies
0
Boosts
0
Views
62
Activity
2w
AudioServerPlugin AddDeviceClient/RemoveDeviceClient calls.
I added logging in an AudioServerPlugin to trace the arrival and removal of a particular client in the AddDeviceClient() and RemoveDeviceClient() methods. To my surprise the app was removed and added as a client often, even when it was in a steady state, for example when using the Audio MIDI Setup to change sample rates. Is this expected behavior, and if so, are there any explanations about this.
Replies
0
Boosts
0
Views
102
Activity
2w
New Siri text-to-speech available to 3rd party apps?
In the keynote (or maybe it was SOTU) they demo'd a new more expressive Siri voice. Is that voice going to be available to 3rd party apps to use for text-to-speech in our apps?
Replies
0
Boosts
0
Views
78
Activity
2w
Optimization of multiple audio track synchronization.
I have an app where multiple audio files of the same length are played and they need to be in sync to produce the full song. My current setup syncs each track to the same position every time the user pauses or plays the song. However it is super laggy when the Audio is played either for the first time or when unpausing/pausing. What's the best way to do this?
Replies
2
Boosts
0
Views
124
Activity
2w
Is there a supported way for apps on iOS to receive audio from other apps?
I make a Mac app where users can transcribe and translate audio from other apps. I want to bring this to iOS, and I'm wondering what APIs are available (if any) to do this. On MacOS I use ScreenCaptureKit currently, but that's not available on iOS.
Replies
0
Boosts
0
Views
101
Activity
2w
How to Seamlessly Handle FIDO QR Codes in Your iOS App
When scanning a FIDO QR code within an iOS app—whether using a custom AVFoundation (AVCaptureSession) implementation or DataScannerViewController—the system displays a native OS confirmation prompt. However, scanning the same QR code using the native system Code Scanner bypasses this prompt entirely. As a developer: Is there a way to suppress or avoid this native prompt when using custom in-app scanners? Alternatively, can I programmatically invoke the system Code Scanner directly from my app and have it deep-link back to the app once the scan is complete?
Replies
0
Boosts
0
Views
116
Activity
2w
Displaying a processed image from AVCaptureVideoDataOutput in a swiftUI view?
What is the recommended way of showing a processed image from AVCaptureVideoDataOutput in a swiftUI view? Currently my chain is AVCaptureVideoDataOutput(SampleBuffer) -> CIImage -> CIFilters -> createCGImage from processed CIImage -> Create swiftUI 'Image' from CGIImage, that is in a view Is there a better way to go from AVCaptureVideoDataOutput to a swiftUI view, with image processing?
Replies
2
Boosts
0
Views
148
Activity
2w
Offscreen drawing to generate video content
We develop software for video broadcasting, including for sporting events, where animated scoreboards play a key role. macOS offers excellent features for such animations—such as CAAnimations—but this holds true only as long as the animations run on visible screens. Distributing the video signal (e.g., via NDI) requires access to the CVPixelBuffers of the individual frames. Currently, we generate the animations on an external screen and create the pixel buffers using ScreenCaptureKit. We are unaware of any way to obtain these pixel buffers without relying on such—strictly speaking, unnecessary—external screens (or multiple screens). If the window is created offscreen, it updates only once per second, which is unusable. Are there alternatives to external screens? If not, why can’t we create a virtual offscreen device for such use cases—one where we specify the required frame rate and rendering frequency—to generate the necessary video frames? This would also be helpful for HTML-based overlays, which are becoming increasingly popular; currently, these are also rendered only once per second when offscreen.
Replies
1
Boosts
0
Views
202
Activity
2w
QuickTime Player will not playback with MediaExtention
Hi Everyone! I am writing a media extension to playback old "Amiga" ANIM files, as a test project. I have build the following objects: MEFormatReader METrackReader MESampleCursor The subclassed objects seem fairly straightforward. Rather than create a Video Decoder, I followed the instructions in the SampleCursor object header, and use the function: loadSampleBufferContainingSamplesToEndCursor:completionHandler: to deliver an BGRA 8-bit image. This works great AVPlayer object in my Swift based test app, but QuickTime Player will not actually play the movie. If I scrub on QuickTime Player's timeline, I can watch the movie just fine. But if I click on the "Play" button. Nothing happens. For fun, I created a custom pixel type, and implemented a MEVideoDecoder object. This also works with my AVPlayer test app, but again, same problem with QuickTime Player. I have even generated a JPEG image in the SampleCursor, and that fails too. I am stumped on this. I do not see any way to have QuickTime Player work properly with my MediaExtension, nor is there any documentation as to what to do. Suggestions? bob
Replies
1
Boosts
0
Views
302
Activity
2w
Automatic sign-in on tvOS
I have a streaming video app in the App Store. I would like to use the automatic sign in feature for tvOS that was introduced in WWDC '25. However, it requires the com.apple.developer.video-subscriber-single-sign-on entitlement. I'm told that I have to join the video partnership program to access that entitlement. I filled out the forms to join the partnership program, but was ghosted. Would you consider making this feature available to everyone, not just those in the video partnership program? It is frustrating that a new and useful feature isn't actually available.
Replies
0
Boosts
0
Views
102
Activity
2w
Apple AUGraphicEQ dead-lock
Hi, I encounter a dead-lock with Apple AUGraphicEQ. I have attached backtrace: thread #44, name = 'com.apple.audio.toolbox.AUScheduledParameterRefresher' frame #0: 0x0000000182059bb0 libsystem_kernel.dylib`semaphore_wait_trap + 8 frame #1: 0x000000018e9e7e00 caulk`caulk::semaphore::timed_wait(double) + 224 frame #2: 0x000000018e9e7cac caulk`caulk::concurrent::details::worker_thread::run() + 32 frame #3: 0x000000018e9e794c caulk`void* caulk::thread_proxy<std::__1::tuple<caulk::thread::attributes, void (caulk::concurrent::details::worker_thread::*)(), std::__1::tuple<caulk::concurrent::details::worker_thread*>>>(void*) + 96 frame #4: 0x000000018209dc58 libsystem_pthread.dylib`_pthread_start + 136 regards, Joël
Replies
2
Boosts
0
Views
528
Activity
2w