Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

Updating downloaded manifest
Following up on: https://feedbackassistant.apple.com/feedback/22501602 We have a downloaded .movpkg (via AVAggregateAssetDownloadTask) where we need to update #EXT-X-DATERANGE tag URLs in the manifest Apple Developer Technical Support (FB15965814) confirmed "it is perfectly fine to update the m3u8 playlist inside the movpkg if that works well for you." Can you confirm this is safe to rely on long-term and won't break in future OS updates? Is there anything we should watch out for (e.g., movpkg integrity checks, asset download storage manager interactions)?
1
0
118
2w
AVAssetResourceLoaderDelegate and error handling
If we are using an AVAssetResourceLoaderDelegate, and performing our own playlist requests using a URLSession, what are some best practices for responding to the resource loading requests with errors? In other words, how best to populate the NSError domain, code, and userInfo values when calling finishLoadingWithError? SImilarly, are there any things to look out for when the URLSession has caching enabled, and we receive HTTP 304s for live playlists? The URLSession appears to transparently handle the 304, and return a 200 with the cached data in the URLSession callbacks. But could there be edge cases where the URLSession’s cache is becoming invalid just as we get a 304?
3
0
205
2w
How many concurrent VTCompressionSession / VTDecompressionSession can an app run, and can the limit be queried?
I'm batch-transcoding a library of clips into downscaled editing proxies. If I kick off two hardware transcodes at once, the encoder reliably falls over, so right now I run everything serially: each clip gets fully decoded, encoded, and muxed before the next one starts. It works, but it's slow, and the decode and encode hardware are mostly idle waiting on each other. A few things I can't pin down from the docs: Is there an actual ceiling on how many VTCompressionSession / VTDecompressionSession instances can be live at once, and does it depend on the device, the codec, or the resolution? Can I query that ceiling at runtime? I'd rather size my concurrency up front than find it by crashing. Decode and encode are separate hardware blocks, so can I safely run a decode session for one clip while the previous clip is still encoding, or does VideoToolbox serialize them anyway? When I do go over the limit, what should I be checking so I can back off cleanly? Right now I just get a crash instead of an error I can catch. Anything that gets me off the fully-serial pipeline would help. Thank you
5
0
294
2w
Reconstruction with Object Mask in PhotogrammetrySession Fails .
Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface
4
0
696
2w
Memory leak on processing stereoscopic video frame, makeMutablePixelBuffer()
Hi, I downloaded and ran https://developer.apple.com/documentation/realitykit/rendering-stereoscopic-video-with-realitykit and noticed that memory usage grows linearly. I replaced the sample video with a different 8k side by side video, and the app crashed almost immediately due to memory leak. it looks like the culprit is from makeMutablePixelBuffer() function and the allocated pixelBuffers are not recycled after being used. screenshot is from a physical device.
1
0
563
2w
VTLowLatencyFrameInterpolationConfiguration supported dimensions
Is there limits on the supported dimension for VTLowLatencyFrameInterpolationConfiguration. Querying VTLowLatencyFrameInterpolationConfiguration.maximumDimensions and VTLowLatencyFrameInterpolationConfiguration.minimumDimensions returns nil. When I try the WWDC sample project EnhancingYourAppWithMachineLearningBasedVideoEffects with a 4k video this statement try frameProcessor.startSession(configuration: configuration) executes but try await frameProcessor.process(parameters: parameters) throws error Error Domain=VTFrameProcessorErrorDomain Code=-19730 "Processor is not initialized" UserInfo={NSLocalizedDescription=Processor is not initialized}. Also, why is VTLowLatencyFrameInterpolationConfiguration able to run while app is backgrounded but VTFrameRateConversionParameters can't (due to gpu usage)?
3
0
653
2w
Issue with Airplay for DRM videos
When I try to send a DRM-protected video via Airplay to an Apple TV, the license request is made twice instead of once as it normally does on iOS. We only allow one request per session for security reasons, this causes the second request to fail and the video won't play. We've tested DRM-protected videos without token usage limits and it works, but this creates a security hole in our system. Why does it request the license twice in function: func contentKeySession(_ session: AVContentKeySession, didProvide keyRequest: AVContentKeyRequest)? Is there a way to prevent this?
1
0
508
2w
CoreMediaErrorDomain error -12848
Good day. A video I created via iOS AVAssetWriter with the following settings: let videoWriterInput = AVAssetWriterInput( mediaType: .video, outputSettings: [ AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoWidthKey: 1080, AVVideoHeightKey: 1920, AVVideoCompressionPropertiesKey: [ AVVideoAverageBitRateKey: 2_000_000, AVVideoMaxKeyFrameIntervalKey: 30 ], ] ) let audioWriterInput = AVAssetWriterInput( mediaType: .audio, outputSettings: [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVNumberOfChannelsKey: 2, AVSampleRateKey: 44100, AVEncoderBitRateKey: 128000 ] ) When It is split into fMP4 HLS format using ffmpeg, the video is unable to be played in iOS with the following error: CoreMediaErrorDomain error -12848 However, the video is played normally in Android, Browser HLS players, and also VLC Media Player. Please assist. Thank you.
2
0
535
2w
SBS and OU ViewPacking
SBS ViewPacking add a half a frame to the opposite eye. Meaning if you look all the way right you can see an extra half frame with left eye and vice versa. OU doesn't work at all, the preview just doesn't show a thumbnail and the video doesn't play. Any hints on how to fix this? I submitted a bug report but haven't heard anything.
2
0
647
2w
Save slow motion video with custom playback speed bar
Background: For iOS, I have built a custom video record app that records video at the highest possible FPS (supposed to be around 240 FPS but more like ~200 in practice). When I save this video to the user's camera roll, I notice this dynamic playback speed bar. This bar will speed up or slow down the video. My question: How can I save my video such that this playback speed bar is constantly slow or plays at real time speed? For reference I have include the playback speed bar that I am talking about in the screenshot, you can find this in the photo app when you record slow motion video.
1
0
358
2w
Most efficient way to upload dozens of videos
In our app, users select many videos (10–100+) that we upload to our server in one batch. Two problems compound each other: Backgrounding: Users almost always leave the app while the batch is in flight. Uploads started with a regular URLSession get suspended. Pre-upload transcoding: To reduce upload duration and cellular data usage, we downscale/re-encode each video (via AVAssetExportSession) before upload. This is resource-intensive and unlike a background URLSession upload task can't continue once the app is suspended, so the pipeline stalls even if the uploads themselves could continue. What's Apple's recommended design here? Is the intended pattern a background URLSession with file-based upload tasks, fed by a BGProcessingTask that drains the transcode queue opportunistically? Are there APIs we're missing, e.g. AVAssetExportSession's background-friendly options, HEVC/preferredTranscoding export presets that are cheaper, or any way to get extended runtime for the encode step (beginBackgroundTask budgets seem too short for dozens of videos)? And is there guidance on the trade-off of just uploading originals and transcoding server-side instead?
2
0
198
2w
How to dynamically update an existing AVComposition when users add a new custom video clip?
I’m building a macOS video editor that uses AVComposition and AVVideoComposition. Initially, my renderer creates a composition with some default video/audio tracks: @Published var composition: AVComposition? @Published var videoComposition: AVVideoComposition? @Published var playerItem: AVPlayerItem? Then I call a buildComposition() function that inserts all the default video segments. Later in the editing workflow, the user may choose to add their own custom video clip. For this I have a function like: private func handlePickedVideo(_ url: URL) { guard url.startAccessingSecurityScopedResource() else { print("Failed to access security-scoped resource") return } let asset = AVURLAsset(url: url) let videoTracks = asset.tracks(withMediaType: .video) guard let firstVideoTrack = videoTracks.first else { print("No video track found") url.stopAccessingSecurityScopedResource() return } renderer.insertUserVideoTrack(from: asset, track: firstVideoTrack) url.stopAccessingSecurityScopedResource() } What I want to achieve is the same behavior professional video editors provide, after the composition has already been initialized and built, the user should be able to add a new video track and the composition should update live, meaning the preview player should immediately reflect the changes without rebuilding everything from scratch manually. How can I structure my AVComposition / AVMutableComposition and my rendering pipeline so that adding a new clip later updates the existing composition in real time (similar to Final Cut/Adobe Premiere), instead of needing to rebuild everything from zero? You can find a playable version of this entire setup at :- https://github.com/zaidbren/SimpleEditor
1
0
577
2w
iOS 26.4 regression: The `.pauses` audiovisual background playback policy does not pause video playback anymore when backgrounding the app
Starting with iOS 26.4 and the iOS 26.4 SDK, the .pauses audiovisual background playback policy is not correctly applied anymore to an AVPlayer having an attached video layer displayed on screen. This means that, when backgrounding a video-playing app (without Picture in Picture support) or locking the device, playback is not paused automatically by the system anymore. This issue affects the Apple TV application as well. We have filed FB22488151 with more information.
1
0
547
2w
AVQueuePlayer best practices
These are follow up questions for: https://feedbackassistant.apple.com/feedback/22779473 For a large AVQueuePlayer queue (50-100+ HLS items), does AVFoundation guarantee it only allocates video decode sessions for the active and next item? Our profiling suggests this is the case, but we want to confirm whether we can rely on this behavior or if we should implement an application-side sliding window to limit concurrent decoders.
1
2
207
2w
PHBackgroundResourceUploadExtension is never scheduled when iCloud Photos is enabled
Feedback: PHBackgroundResourceUploadExtension is never scheduled when iCloud Photos is enabled Summary PHBackgroundResourceUploadExtension's init() and process() methods are never called by the system when iCloud Photos is enabled on the device, even though setUploadJobExtensionEnabled(true) succeeds and uploadJobExtensionEnabled returns true. Environment iOS 26.4 (both devices) Xcode 26.x Tested on iPhone 17 Pro (primary device, 10,000+ photos) and an older iPhone (development device, 200+ photos) Same build deployed to both devices Full photo library access (.authorized) on both devices Steps to Reproduce Create an app with a PHBackgroundResourceUploadExtension (ExtensionKit, extension point com.apple.photos.background-upload) Enable iCloud Photos on the device (Settings > Photos > iCloud Photos) In the host app, request .readWrite photo library authorization and receive .authorized Call PHPhotoLibrary.shared().setUploadJobExtensionEnabled(true) — succeeds without error Verify PHPhotoLibrary.shared().uploadJobExtensionEnabled == true Wait for the system to schedule the extension (tested overnight, with device on charger + WiFi) Expected Behavior The system should call the extension's init() and process() methods to allow the app to create upload jobs, as documented in "Uploading asset resources in the background." Actual Behavior The extension's init() and process() are never called. The extension process is never launched. Investigation via Console.app Using Console.app to monitor system logs on both devices revealed the root cause: When iCloud Photos is DISABLED (extension works correctly): assetsd Checked all submitted library bundles. Result: YES dasd SUBMITTING: com.apple.assetsd.backgroundjobservice.assetresourceuploadextensionrunner:35E7B5 dasd Submitted: ...assetresourceuploadextensionrunner at priority 5 dasd submitTaskRequestWithIdentifier: Submitted BGSystemTask com.apple.assetsd.backgroundjobservice.assetresourceuploadextensionrunner The extension is subsequently launched and init() / process() are called as expected. When iCloud Photos is ENABLED (extension never works): assetsd Checked all submitted library bundles. Result: NO No assetresourceuploadextensionrunner background task is ever submitted to dasd. The extension is never launched. Reproducibility 100% reproducible across two different devices Toggling iCloud Photos off (and waiting for sync to complete) immediately resolves the issue Toggling iCloud Photos on immediately causes the issue to reappear Reinstalling the app does not help The same build works on the same device when iCloud Photos is disabled Impact This effectively makes PHBackgroundResourceUploadExtension unusable for the vast majority of users, as most iPhone users have iCloud Photos enabled. Third-party photo backup apps (Google Photos, Dropbox, OneDrive, etc.) would all be affected by this limitation. The documentation for "Uploading asset resources in the background" does not mention any incompatibility with iCloud Photos being enabled. Requested Resolution Please either: Allow PHBackgroundResourceUploadExtension to be scheduled regardless of iCloud Photos status, or Document this limitation clearly in the API documentation if it is intentional behavior
2
0
445
2w
How to reliably debug PHBackgroundResourceUploadExtension during development?
I'm developing a PHBackgroundResourceUploadExtension and finding it difficult to debug because the system controls when the extension launches. Current experience: The extension starts at unpredictable times (anywhere from 1 minute to several hours after photos are added) By the time I attach the debugger, the upload may have already completed or failed Breakpoints in init() or early lifecycle methods are often missed Questions: Is there a way to force-launch the extension during development (similar to how we can manually trigger Background App Refresh in Xcode)? Are there any launch arguments or environment variables that put the extension in a debug/eager mode? I tried taking photos/videos, but this doesn't trigged app extension in all cases. Any tips for improving the debug cycle would be greatly appreciated. Environment: iOS 26, Xcode 18
2
1
806
2w
Lack of AVCameraCalibrationData parity in RAW Photo workflows
I am requesting technical clarification and a formal feature addition regarding the availability of per-frame calibration data for Standard RAW captures on iPhone 17 and 17 Pro. The Technical Gap: Currently, AVCaptureVideoDataOutput provides a direct path to the AVCameraCalibrationData object, allowing real-time access to the Intrinsic Matrix, Radial/Tangential Distortion coefficients, and Lens Shading Maps. However, this same level of geometric transparency is missing from the AVCapturePhoto RAW/ProRAW delegate. While standard RAW files contain some metadata, they lack the physics sidecar required for precision manual alignment. Because the lens assembly in the iPhone 17 Pro is dynamic shifted constantly by Optical Image Stabilization (OIS) and high-speed Voice Coil Motors (VCM) for focus a static or factor calibration is mathematically insufficient for high-precision workflows. The Problem: Without the 1:1 hardware state at the millisecond of exposure, we cannot perform accurate geometric reconstruction from RAW stills. We are forced to choose between the high dynamic range of a RAW sensor dump and the geometric precision of the video pipeline. Final Questions: Is there a documented, supported method to force the inclusion of the AVCameraCalibrationData object or a raw metadata sidecar in the AVCapturePhoto workflow? If not, can Apple provide parity between the Video and Photo APIs so that the "rawest" data (RAW) is accompanied by the "rawest" physics (Calibration Data)? Providing the pixels without the lens geometry limits the utility of the RAW format for any technical workflow requiring sub-pixel geometric integrity.
1
0
775
2w
RotationCoordinator returns angles 90 degrees lower on iPhone 17 Pro front camera — clarification on contract with AVSampleBufferDisplayLayer
Hi AVFoundation team, I'm seeing a uniform 90° offset in AVCaptureDevice.RotationCoordinator's reported angles between iPhone 17 Pro and iPhone 14 Pro using the front-facing .builtInWideAngleCamera (Center Stage on 17 Pro), and I'd like to confirm whether this is by design and what the recommended consumption pattern is when the rendering surface is an AVSampleBufferDisplayLayer rather than an AVCaptureVideoPreviewLayer. Here is the github repo of sample project. Setup Devices: iPhone 14 Pro (iOS 26.5) and iPhone 17 Pro (iOS 26.4.2) Camera: front, AVCaptureDeviceTypeBuiltInWideAngleCamera Active format: 1920×1080 Three RotationCoordinator instances are created on the same AVCaptureDevice, varying only the previewLayer: argument: - previewLayer: nil - previewLayer: AVSampleBufferDisplayLayer (the surface receiving frames from AVCaptureVideoDataOutput) - previewLayer: AVCaptureVideoPreviewLayer (with .session = captureSession, not displayed) Each instance is KVO-observed for videoRotationAngleForHorizonLevelPreview and videoRotationAngleForHorizonLevelCapture. Observed angles Device / Orientation: 14 Pro · Portrait (interface=1) RC[nil] prev / cap: 0° / 90° RC[AVSampleBufferLayer] prev / cap: 90° / 90° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 90° ──────────────────────────────────────── Device / Orientation: 14 Pro · LandscapeRight (interface=3) RC[nil] prev / cap: 0° / 180° RC[AVSampleBufferLayer] prev / cap: 180° / 180° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 180° ──────────────────────────────────────── Device / Orientation: 17 Pro · Portrait (interface=1) RC[nil] prev / cap: 0° / 0° RC[AVSampleBufferLayer] prev / cap: 0° / 0° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 0° ──────────────────────────────────────── Device / Orientation: 17 Pro · LandscapeRight (interface=3) RC[nil] prev / cap: 0° / 90° RC[AVSampleBufferLayer] prev / cap: 90° / 90° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 90° The −90° offset on 17 Pro is uniform: it appears in every RC variant, in both the preview-angle and the capture-angle properties, at every orientation tested. It is not specific to the previewLayer: argument.
3
1
611
2w
Updating downloaded manifest
Following up on: https://feedbackassistant.apple.com/feedback/22501602 We have a downloaded .movpkg (via AVAggregateAssetDownloadTask) where we need to update #EXT-X-DATERANGE tag URLs in the manifest Apple Developer Technical Support (FB15965814) confirmed "it is perfectly fine to update the m3u8 playlist inside the movpkg if that works well for you." Can you confirm this is safe to rely on long-term and won't break in future OS updates? Is there anything we should watch out for (e.g., movpkg integrity checks, asset download storage manager interactions)?
Replies
1
Boosts
0
Views
118
Activity
2w
AVAssetResourceLoaderDelegate and error handling
If we are using an AVAssetResourceLoaderDelegate, and performing our own playlist requests using a URLSession, what are some best practices for responding to the resource loading requests with errors? In other words, how best to populate the NSError domain, code, and userInfo values when calling finishLoadingWithError? SImilarly, are there any things to look out for when the URLSession has caching enabled, and we receive HTTP 304s for live playlists? The URLSession appears to transparently handle the 304, and return a 200 with the cached data in the URLSession callbacks. But could there be edge cases where the URLSession’s cache is becoming invalid just as we get a 304?
Replies
3
Boosts
0
Views
205
Activity
2w
How many concurrent VTCompressionSession / VTDecompressionSession can an app run, and can the limit be queried?
I'm batch-transcoding a library of clips into downscaled editing proxies. If I kick off two hardware transcodes at once, the encoder reliably falls over, so right now I run everything serially: each clip gets fully decoded, encoded, and muxed before the next one starts. It works, but it's slow, and the decode and encode hardware are mostly idle waiting on each other. A few things I can't pin down from the docs: Is there an actual ceiling on how many VTCompressionSession / VTDecompressionSession instances can be live at once, and does it depend on the device, the codec, or the resolution? Can I query that ceiling at runtime? I'd rather size my concurrency up front than find it by crashing. Decode and encode are separate hardware blocks, so can I safely run a decode session for one clip while the previous clip is still encoding, or does VideoToolbox serialize them anyway? When I do go over the limit, what should I be checking so I can back off cleanly? Right now I just get a crash instead of an error I can catch. Anything that gets me off the fully-serial pipeline would help. Thank you
Replies
5
Boosts
0
Views
294
Activity
2w
Reconstruction with Object Mask in PhotogrammetrySession Fails .
Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface
Replies
4
Boosts
0
Views
696
Activity
2w
Memory leak on processing stereoscopic video frame, makeMutablePixelBuffer()
Hi, I downloaded and ran https://developer.apple.com/documentation/realitykit/rendering-stereoscopic-video-with-realitykit and noticed that memory usage grows linearly. I replaced the sample video with a different 8k side by side video, and the app crashed almost immediately due to memory leak. it looks like the culprit is from makeMutablePixelBuffer() function and the allocated pixelBuffers are not recycled after being used. screenshot is from a physical device.
Replies
1
Boosts
0
Views
563
Activity
2w
VTLowLatencyFrameInterpolationConfiguration supported dimensions
Is there limits on the supported dimension for VTLowLatencyFrameInterpolationConfiguration. Querying VTLowLatencyFrameInterpolationConfiguration.maximumDimensions and VTLowLatencyFrameInterpolationConfiguration.minimumDimensions returns nil. When I try the WWDC sample project EnhancingYourAppWithMachineLearningBasedVideoEffects with a 4k video this statement try frameProcessor.startSession(configuration: configuration) executes but try await frameProcessor.process(parameters: parameters) throws error Error Domain=VTFrameProcessorErrorDomain Code=-19730 "Processor is not initialized" UserInfo={NSLocalizedDescription=Processor is not initialized}. Also, why is VTLowLatencyFrameInterpolationConfiguration able to run while app is backgrounded but VTFrameRateConversionParameters can't (due to gpu usage)?
Replies
3
Boosts
0
Views
653
Activity
2w
Issue with Airplay for DRM videos
When I try to send a DRM-protected video via Airplay to an Apple TV, the license request is made twice instead of once as it normally does on iOS. We only allow one request per session for security reasons, this causes the second request to fail and the video won't play. We've tested DRM-protected videos without token usage limits and it works, but this creates a security hole in our system. Why does it request the license twice in function: func contentKeySession(_ session: AVContentKeySession, didProvide keyRequest: AVContentKeyRequest)? Is there a way to prevent this?
Replies
1
Boosts
0
Views
508
Activity
2w
CoreMediaErrorDomain error -12848
Good day. A video I created via iOS AVAssetWriter with the following settings: let videoWriterInput = AVAssetWriterInput( mediaType: .video, outputSettings: [ AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoWidthKey: 1080, AVVideoHeightKey: 1920, AVVideoCompressionPropertiesKey: [ AVVideoAverageBitRateKey: 2_000_000, AVVideoMaxKeyFrameIntervalKey: 30 ], ] ) let audioWriterInput = AVAssetWriterInput( mediaType: .audio, outputSettings: [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVNumberOfChannelsKey: 2, AVSampleRateKey: 44100, AVEncoderBitRateKey: 128000 ] ) When It is split into fMP4 HLS format using ffmpeg, the video is unable to be played in iOS with the following error: CoreMediaErrorDomain error -12848 However, the video is played normally in Android, Browser HLS players, and also VLC Media Player. Please assist. Thank you.
Replies
2
Boosts
0
Views
535
Activity
2w
SBS and OU ViewPacking
SBS ViewPacking add a half a frame to the opposite eye. Meaning if you look all the way right you can see an extra half frame with left eye and vice versa. OU doesn't work at all, the preview just doesn't show a thumbnail and the video doesn't play. Any hints on how to fix this? I submitted a bug report but haven't heard anything.
Replies
2
Boosts
0
Views
647
Activity
2w
Save slow motion video with custom playback speed bar
Background: For iOS, I have built a custom video record app that records video at the highest possible FPS (supposed to be around 240 FPS but more like ~200 in practice). When I save this video to the user's camera roll, I notice this dynamic playback speed bar. This bar will speed up or slow down the video. My question: How can I save my video such that this playback speed bar is constantly slow or plays at real time speed? For reference I have include the playback speed bar that I am talking about in the screenshot, you can find this in the photo app when you record slow motion video.
Replies
1
Boosts
0
Views
358
Activity
2w
Most efficient way to upload dozens of videos
In our app, users select many videos (10–100+) that we upload to our server in one batch. Two problems compound each other: Backgrounding: Users almost always leave the app while the batch is in flight. Uploads started with a regular URLSession get suspended. Pre-upload transcoding: To reduce upload duration and cellular data usage, we downscale/re-encode each video (via AVAssetExportSession) before upload. This is resource-intensive and unlike a background URLSession upload task can't continue once the app is suspended, so the pipeline stalls even if the uploads themselves could continue. What's Apple's recommended design here? Is the intended pattern a background URLSession with file-based upload tasks, fed by a BGProcessingTask that drains the transcode queue opportunistically? Are there APIs we're missing, e.g. AVAssetExportSession's background-friendly options, HEVC/preferredTranscoding export presets that are cheaper, or any way to get extended runtime for the encode step (beginBackgroundTask budgets seem too short for dozens of videos)? And is there guidance on the trade-off of just uploading originals and transcoding server-side instead?
Replies
2
Boosts
0
Views
198
Activity
2w
<<<< PlayerRemoteXPC >>>> signalled err=-12860 at <>:1519
Avplayer encapsulates a player. After connecting to an Apple Bluetooth headset, it immediately reports an error. Non-Apple Bluetooth headsets can be played, but currently the issue is that the player works normally in one app but not in another. We are an educational app.
Replies
1
Boosts
0
Views
638
Activity
2w
How to dynamically update an existing AVComposition when users add a new custom video clip?
I’m building a macOS video editor that uses AVComposition and AVVideoComposition. Initially, my renderer creates a composition with some default video/audio tracks: @Published var composition: AVComposition? @Published var videoComposition: AVVideoComposition? @Published var playerItem: AVPlayerItem? Then I call a buildComposition() function that inserts all the default video segments. Later in the editing workflow, the user may choose to add their own custom video clip. For this I have a function like: private func handlePickedVideo(_ url: URL) { guard url.startAccessingSecurityScopedResource() else { print("Failed to access security-scoped resource") return } let asset = AVURLAsset(url: url) let videoTracks = asset.tracks(withMediaType: .video) guard let firstVideoTrack = videoTracks.first else { print("No video track found") url.stopAccessingSecurityScopedResource() return } renderer.insertUserVideoTrack(from: asset, track: firstVideoTrack) url.stopAccessingSecurityScopedResource() } What I want to achieve is the same behavior professional video editors provide, after the composition has already been initialized and built, the user should be able to add a new video track and the composition should update live, meaning the preview player should immediately reflect the changes without rebuilding everything from scratch manually. How can I structure my AVComposition / AVMutableComposition and my rendering pipeline so that adding a new clip later updates the existing composition in real time (similar to Final Cut/Adobe Premiere), instead of needing to rebuild everything from zero? You can find a playable version of this entire setup at :- https://github.com/zaidbren/SimpleEditor
Replies
1
Boosts
0
Views
577
Activity
2w
iOS 26.4 regression: The `.pauses` audiovisual background playback policy does not pause video playback anymore when backgrounding the app
Starting with iOS 26.4 and the iOS 26.4 SDK, the .pauses audiovisual background playback policy is not correctly applied anymore to an AVPlayer having an attached video layer displayed on screen. This means that, when backgrounding a video-playing app (without Picture in Picture support) or locking the device, playback is not paused automatically by the system anymore. This issue affects the Apple TV application as well. We have filed FB22488151 with more information.
Replies
1
Boosts
0
Views
547
Activity
2w
Are frames returned in presentation or decode order with AVAssetReader
I read somewhere that the frames are returned in decode order instead of presentation order when using AVAssetReader. The documentation seems sparse on the subject. I have so far failed to find a video file where the frames are not returned in presentation order. Can anyone confirm the frames are actually returned in decode order?
Replies
2
Boosts
1
Views
373
Activity
2w
AVQueuePlayer best practices
These are follow up questions for: https://feedbackassistant.apple.com/feedback/22779473 For a large AVQueuePlayer queue (50-100+ HLS items), does AVFoundation guarantee it only allocates video decode sessions for the active and next item? Our profiling suggests this is the case, but we want to confirm whether we can rely on this behavior or if we should implement an application-side sliding window to limit concurrent decoders.
Replies
1
Boosts
2
Views
207
Activity
2w
PHBackgroundResourceUploadExtension is never scheduled when iCloud Photos is enabled
Feedback: PHBackgroundResourceUploadExtension is never scheduled when iCloud Photos is enabled Summary PHBackgroundResourceUploadExtension's init() and process() methods are never called by the system when iCloud Photos is enabled on the device, even though setUploadJobExtensionEnabled(true) succeeds and uploadJobExtensionEnabled returns true. Environment iOS 26.4 (both devices) Xcode 26.x Tested on iPhone 17 Pro (primary device, 10,000+ photos) and an older iPhone (development device, 200+ photos) Same build deployed to both devices Full photo library access (.authorized) on both devices Steps to Reproduce Create an app with a PHBackgroundResourceUploadExtension (ExtensionKit, extension point com.apple.photos.background-upload) Enable iCloud Photos on the device (Settings > Photos > iCloud Photos) In the host app, request .readWrite photo library authorization and receive .authorized Call PHPhotoLibrary.shared().setUploadJobExtensionEnabled(true) — succeeds without error Verify PHPhotoLibrary.shared().uploadJobExtensionEnabled == true Wait for the system to schedule the extension (tested overnight, with device on charger + WiFi) Expected Behavior The system should call the extension's init() and process() methods to allow the app to create upload jobs, as documented in "Uploading asset resources in the background." Actual Behavior The extension's init() and process() are never called. The extension process is never launched. Investigation via Console.app Using Console.app to monitor system logs on both devices revealed the root cause: When iCloud Photos is DISABLED (extension works correctly): assetsd Checked all submitted library bundles. Result: YES dasd SUBMITTING: com.apple.assetsd.backgroundjobservice.assetresourceuploadextensionrunner:35E7B5 dasd Submitted: ...assetresourceuploadextensionrunner at priority 5 dasd submitTaskRequestWithIdentifier: Submitted BGSystemTask com.apple.assetsd.backgroundjobservice.assetresourceuploadextensionrunner The extension is subsequently launched and init() / process() are called as expected. When iCloud Photos is ENABLED (extension never works): assetsd Checked all submitted library bundles. Result: NO No assetresourceuploadextensionrunner background task is ever submitted to dasd. The extension is never launched. Reproducibility 100% reproducible across two different devices Toggling iCloud Photos off (and waiting for sync to complete) immediately resolves the issue Toggling iCloud Photos on immediately causes the issue to reappear Reinstalling the app does not help The same build works on the same device when iCloud Photos is disabled Impact This effectively makes PHBackgroundResourceUploadExtension unusable for the vast majority of users, as most iPhone users have iCloud Photos enabled. Third-party photo backup apps (Google Photos, Dropbox, OneDrive, etc.) would all be affected by this limitation. The documentation for "Uploading asset resources in the background" does not mention any incompatibility with iCloud Photos being enabled. Requested Resolution Please either: Allow PHBackgroundResourceUploadExtension to be scheduled regardless of iCloud Photos status, or Document this limitation clearly in the API documentation if it is intentional behavior
Replies
2
Boosts
0
Views
445
Activity
2w
How to reliably debug PHBackgroundResourceUploadExtension during development?
I'm developing a PHBackgroundResourceUploadExtension and finding it difficult to debug because the system controls when the extension launches. Current experience: The extension starts at unpredictable times (anywhere from 1 minute to several hours after photos are added) By the time I attach the debugger, the upload may have already completed or failed Breakpoints in init() or early lifecycle methods are often missed Questions: Is there a way to force-launch the extension during development (similar to how we can manually trigger Background App Refresh in Xcode)? Are there any launch arguments or environment variables that put the extension in a debug/eager mode? I tried taking photos/videos, but this doesn't trigged app extension in all cases. Any tips for improving the debug cycle would be greatly appreciated. Environment: iOS 26, Xcode 18
Replies
2
Boosts
1
Views
806
Activity
2w
Lack of AVCameraCalibrationData parity in RAW Photo workflows
I am requesting technical clarification and a formal feature addition regarding the availability of per-frame calibration data for Standard RAW captures on iPhone 17 and 17 Pro. The Technical Gap: Currently, AVCaptureVideoDataOutput provides a direct path to the AVCameraCalibrationData object, allowing real-time access to the Intrinsic Matrix, Radial/Tangential Distortion coefficients, and Lens Shading Maps. However, this same level of geometric transparency is missing from the AVCapturePhoto RAW/ProRAW delegate. While standard RAW files contain some metadata, they lack the physics sidecar required for precision manual alignment. Because the lens assembly in the iPhone 17 Pro is dynamic shifted constantly by Optical Image Stabilization (OIS) and high-speed Voice Coil Motors (VCM) for focus a static or factor calibration is mathematically insufficient for high-precision workflows. The Problem: Without the 1:1 hardware state at the millisecond of exposure, we cannot perform accurate geometric reconstruction from RAW stills. We are forced to choose between the high dynamic range of a RAW sensor dump and the geometric precision of the video pipeline. Final Questions: Is there a documented, supported method to force the inclusion of the AVCameraCalibrationData object or a raw metadata sidecar in the AVCapturePhoto workflow? If not, can Apple provide parity between the Video and Photo APIs so that the "rawest" data (RAW) is accompanied by the "rawest" physics (Calibration Data)? Providing the pixels without the lens geometry limits the utility of the RAW format for any technical workflow requiring sub-pixel geometric integrity.
Replies
1
Boosts
0
Views
775
Activity
2w
RotationCoordinator returns angles 90 degrees lower on iPhone 17 Pro front camera — clarification on contract with AVSampleBufferDisplayLayer
Hi AVFoundation team, I'm seeing a uniform 90° offset in AVCaptureDevice.RotationCoordinator's reported angles between iPhone 17 Pro and iPhone 14 Pro using the front-facing .builtInWideAngleCamera (Center Stage on 17 Pro), and I'd like to confirm whether this is by design and what the recommended consumption pattern is when the rendering surface is an AVSampleBufferDisplayLayer rather than an AVCaptureVideoPreviewLayer. Here is the github repo of sample project. Setup Devices: iPhone 14 Pro (iOS 26.5) and iPhone 17 Pro (iOS 26.4.2) Camera: front, AVCaptureDeviceTypeBuiltInWideAngleCamera Active format: 1920×1080 Three RotationCoordinator instances are created on the same AVCaptureDevice, varying only the previewLayer: argument: - previewLayer: nil - previewLayer: AVSampleBufferDisplayLayer (the surface receiving frames from AVCaptureVideoDataOutput) - previewLayer: AVCaptureVideoPreviewLayer (with .session = captureSession, not displayed) Each instance is KVO-observed for videoRotationAngleForHorizonLevelPreview and videoRotationAngleForHorizonLevelCapture. Observed angles Device / Orientation: 14 Pro · Portrait (interface=1) RC[nil] prev / cap: 0° / 90° RC[AVSampleBufferLayer] prev / cap: 90° / 90° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 90° ──────────────────────────────────────── Device / Orientation: 14 Pro · LandscapeRight (interface=3) RC[nil] prev / cap: 0° / 180° RC[AVSampleBufferLayer] prev / cap: 180° / 180° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 180° ──────────────────────────────────────── Device / Orientation: 17 Pro · Portrait (interface=1) RC[nil] prev / cap: 0° / 0° RC[AVSampleBufferLayer] prev / cap: 0° / 0° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 0° ──────────────────────────────────────── Device / Orientation: 17 Pro · LandscapeRight (interface=3) RC[nil] prev / cap: 0° / 90° RC[AVSampleBufferLayer] prev / cap: 90° / 90° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 90° The −90° offset on 17 Pro is uniform: it appears in every RC variant, in both the preview-angle and the capture-angle properties, at every orientation tested. It is not specific to the previewLayer: argument.
Replies
3
Boosts
1
Views
611
Activity
2w