Posts under Machine Learning & AI topic

Post

Replies

Boosts

Views

Activity

Core AI Debugger Beta Questions
Hi guys, I am a new core developer. Recently I am trying the newly released core ai debugger tool and meet some issues. I am not sure whether the related functions of debugger is ready in beta package. Reference Run: aimodel on device versus intermediate dump I used the intermediate exporting api to export the intermediate dump data of MobileNet2.0 and converted the same MobileNet Model to .aimodel. But the compare button is not usable. the suffix of intermediate file suffix is OK(can be selected). The debugger is a GUI so I cannot figure out whether I exported the intermediate data incorrectly. (1st attached png ) on device model run compare I just simply tried the different computing unit and different input for a simple 'add/sub/mul/div' toy model. Especially set the input different but the comparison results seems to be empty I am not sure whether I modify the debugger tool incorrectly. Hope someone sharing experience. My Gratitude.
0
0
18
6h
Can output images from imagePlaygroundSheet be used as input for third-party video generation APIs?
I'm developing an iOS app that uses imagePlaygroundSheet to generate images based on AI-generated prompts (not user-typed text). I would like to confirm whether the following usage is permitted under the Apple Developer Program License Agreement: Generate an image using imagePlaygroundSheet within my app Pass the resulting image (obtained via the completion handler URL) as an input to a third-party video generation API (e.g., to create an image-to-video animation) Offer this video generation as a paid feature (consumable in-app purchase) within my app I've reviewed Section 3.2(8) of the ADPLA, which prohibits using "output generated from an Apple model to train, fine-tune, or improve another artificial intelligence model." My understanding is that this clause is specifically about training/improving AI models, not about using the output image as a content input for a separate processing step (e.g., animation). Could someone confirm: Whether passing imagePlaygroundSheet output to a third-party API for further processing (not training) is permitted Whether monetizing a feature that uses this combined output is permitted Thank you in advance for any guidance.
0
0
17
6h
I got old Siri UI instead
I’m on a base iPhone 17 and I‘ve been approved from the waitlist over two weeks ago and when I use Siri, I get the old Siri from before Apple Intelligence was a thing (Glowy orb at the bottom). Inside Siri settings, it looks like it’s supposed to be set up for the new SirI as it shows the app settings. Here are my observations I have found from using beta 1 to 2. Beta 1: Whenever I have the new Siri “enabled”, there is no app I can type “siri://“ and it‘ll open the Siri open saying that’s there’s an update in progress. But if I type that in when I’m using the previous Siri, the app is there and it’s blank When new Siri is ”enabled” and whenever I ask Siri a question she’ll either think and never stop or output a blank answer. But when I turn Siri back to its previous version and open the “sir://“, it stores the conversation and when I open it, it actually answered my question. Beta 2: When installing beta 2 I said “new Siri“ enabled Once it was done installing, I went to iPhone storage and saw that “Apple Intelligence“ storage is increasing size. But then after a minute, it went back down to 14GB Still gave me the old Siri UI Now the siri app is not accessible when typing “siri://“ Siri mode is now in the camera app but it says it’s unavailable The ChatGPT extension area is now blocked because assets are downloading Whenever I visual intelligence and press ask it says that siri support is downloading I left siri alone for 3 days and nothing happened. Still stuck with old siri The ask siri option is everywhere now and when I press it, nothing happens Right now I’m using the previous version of Siri and hoping this bug will be patched and I can use the new Siri AI in Beta 3. *the screenshot provided shows what happens when I use Siri in beta 2 when it’s enabled.
0
0
28
6h
LongRunningIntent run from in the app?
If I attempt to use a LongRunningIntent from a SwiftUI Button, using the Button(_:AppIntent:) control, I get the following errors: [LongRunningIntent <<E:<unknown>>>] No IntentContext available performBackgroundTask threw: noContext Intent failed to execute with error: LNPerformActionErrorCodeUnsupportedValueType It runs as expected when run from the Shortcuts app, but fails when run from a button within the app. Feedback ID (with sample app): FB23492034
1
1
16
7h
IndexedEntities and Siri AI
Currently, I have spotlight entities show up when I search for them using Spotlight on iOS 27. These entities are things that are important for users, like campus buildings, accessible entrances, assignments, and more. However, after getting access to Siri AI, it seems that none of this information at all is available to Siri, yet all of it is sitting there in the spotlight index and viewable with a written query. I was told by an Apple Engineer that creating Indexed and EnumerableEntities, and indexing them via the App Intents framework, should expose information about these items to Siri, so if I query: "[Building name] in Ohio State" it would at least show me what the app has for that information. Presently, Siri uses the web for everything and doesn't pull in any spotlight information for my app, despite either creating wrapper entities or using the API associating with spotlight. With Siri AI, it would be so much more helpful for a disabled user to say "Orton Hall accessible entrance" and Siri to know that there's 1 accessible entrance indexed in spotlight in my app, and then show or open it, instead of querying the web or saying it can't answer the question. It has all available information already in spotlight to answer this question. Currently, as far as I'm aware, something like this simply doesn't work, unless your app conforms to the strict use cases of making reminders or calendar events, all of which aren't useful here. Can a Frameworks engineer please clarify precisely when and how IndexedEntities (paired with an a corresponding macro-annotated OpenIntent) eg: @AppIntent(schema: .system.open) struct OpenBuildingIntent: OpenIntent { @Parameter(title: "Building") var Building: BuildingEntity ... will or will not be visible using Siri AI? To me it seems I have wasted a lot of time porting actions within my app to App Intents, and viewable entities with AppEntity, only to have Siri not be able to use any of this information out of the box.
2
0
191
7h
App schema domains for trains, flighs, ferrys and othe public transport
Hi everyone, I've submitted a Feedback request (FB23469644) asking to introduce a generic Transportation Journey schema for App Intents. At the moment, there doesn't seem to be a way to donate structured public transportation trips (train, flight, bus, ferry, etc.) to Siri's semantic index. While many travel apps already have rich journey data, there is no common semantic model that represents: Multi-leg journeys Departure and arrival stations/airports/stops Platforms, tracks, gates, terminals Scheduled and real-time departure/arrival information Delays and cancellations Ticket and reservation information This seems like a natural addition to the existing App Intents schemas and would enable Siri, Spotlight, and Apple Intelligence to better understand upcoming journeys across all transportation providers. If you're building a travel or mobility app and have encountered the same limitation, I'd appreciate hearing your thoughts. Any idea if the old Siri event suggestion API gets is indexed by Siri Semantic Index / Siri AI?
3
6
91
7h
Sensitive Content Error When Using Foundation Models
I am using the following code in my iOS application. #Playground { let session = LanguageModelSession() let response = try await session.respond(to: "List all states of USA.") print(response.content) } And I get the following error: The operation couldn’t be completed. (com.apple.SensitiveContentAnalysisML error 15.) I have turned off Apple Intelligence and turned on again. No use. I am using Xcode 27 beta 2. any ideas?
1
0
51
20h
AppIntents and String catalog: how can we support both singular and plural forms for TypeDisplayRepresentation (used by DeleteIntent in the Shortcuts app for example)
Hello, I’m implementing the AppIntents framework in my app. I want to translate the TypeDisplayRepresentation that is used in the Shortcuts app UI like in a DeleteIntent (see my feedback about this: FB23451186 for more context). In the “Accelerating app interactions with App Intents” sample code we can see that this is done using a .stringsdict file, as follows for the “Trail” key (singular and plural): <key>Trail</key> <dict> <key>NSStringLocalizedFormatKey</key> <string>%#@VARIABLE@</string> <key>VARIABLE</key> <dict> <key>NSStringFormatSpecTypeKey</key> <string>NSStringPluralRuleType</string> <key>one</key> <string>Trail</string> <key>other</key> <string>Trails</string> </dict> </dict> I want to use a String catalog instead of a .stringsdict file because all my strings are in a String catalog. I tried to migrate the AppIntents.stringsdict file manually but it failed with an error: “An error occurred when migrating AppIntentsSampleApp/Resources/AppIntents.stringsdict: This stringsdict cannot be migrated: Missing required key 'NSStringFormatValueTypeKey' inside 'Trail' -> 'VARIABLE’” So I manually added a NSStringFormatValueTypeKey like this in the .stringsdict file: <key>Trail</key> <dict> <key>NSStringLocalizedFormatKey</key> <string>%#@VARIABLE@</string> <key>VARIABLE</key> <dict> <key>NSStringFormatSpecTypeKey</key> <string>NSStringPluralRuleType</string> <key>NSStringFormatValueTypeKey</key> <string>lld</string> <key>one</key> <string>Trail</string> <key>other</key> <string>Trails</string> </dict> </dict> And then I’ve been able to migrate the .stringsdict file into a String catalog. The string catalog looks like this after migration: "Trail" : { "extractionState" : "stale", "localizations" : { "en" : { "stringUnit" : { "state" : "translated", "value" : "%#@VARIABLE@" }, "substitutions" : { "VARIABLE" : { "formatSpecifier" : "lld", "variations" : { "plural" : { "one" : { "stringUnit" : { "state" : "translated", "value" : "Trail (Catalog)" } }, "other" : { "stringUnit" : { "state" : "translated", "value" : "Trails (Catalog)" } } } } } } } } This works, which is nice. But I tried to reproduce the same result by having a %#@VARIABLE@ in my LocalizedStringResource defaultValue like this: TypeDisplayRepresentation( name: LocalizedStringResource( "Flower", defaultValue: "%#@VARIABLE@", table: "AppIntents" ), numericFormat: LocalizedStringResource( "\(placeholder: .int) flower", table: "AppIntents" ) ) But the String catalog doesn’t support that apparently, I can’t get a “substitution” object in my catalog, so I have to manually do it using the source code which is not ideal and painful. Is there a way to support this kind of substitution with no actual plural token in the string as we can see for the %#@VARIABLE@ for Trail? Thank you, Regards, Axel
1
0
79
1d
Bring an LLM provider to the Foundation Models, missing MLX dependencies
On this talk: Bring an LLM provider to the Foundation Models framework URL: https://developer.apple.com/videos/play/wwdc2026/339/ on the coding examples a very peculiar framework is shown: import MLXFoundationModels However I am not able to find it nowhere, there is even a code section with this framework as part of an example. Where is this framework, there are no BETA branches on the MLX framework either. Thanks!
2
0
91
1d
iOS 27 ImagePlaygroundViewController.Delegate not working?
In the WWDC 2026 sessions it was called out in code that the helper functions would still work, however they don't seem to be working either inside a UIViewRepresentable, nor as a UIKit View as below (also tried as a sheet, to no avail). Otherwise it works. Is there something else I'm missing? import SwiftUI import ImagePlayground @available(iOS 27.0, *) final class ImagePlaygroundPopupController: UIViewController { var sourceImage: UIImage var prompt: String var onComplete: (URL) -> Void var onCancel: () -> Void private var didPresent = false private var playgroundVC: ImagePlaygroundViewController? init( sourceImage: UIImage, prompt: String, onComplete: @escaping (URL) -> Void, onCancel: @escaping () -> Void ) { self.sourceImage = sourceImage self.prompt = prompt self.onComplete = onComplete self.onCancel = onCancel super.init(nibName: nil, bundle: nil) view.backgroundColor = .clear } required init?(coder: NSCoder) { fatalError("init(coder:) has not been implemented") } override func viewDidAppear(_ animated: Bool) { super.viewDidAppear(animated) guard !didPresent else { return } didPresent = true let vc = ImagePlaygroundViewController() vc.sourceImage = sourceImage vc.concepts = [.text(prompt)] vc.delegate = self playgroundVC = vc present(vc, animated: true) } } @available(iOS 27.0, *) extension ImagePlaygroundPopupController: ImagePlaygroundViewController.Delegate { func imagePlaygroundViewController( _ imagePlaygroundViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL ) { imagePlaygroundViewController.dismiss(animated: true) { self.playgroundVC = nil self.onComplete(imageURL) } } func imagePlaygroundViewControllerDidCancel( _ imagePlaygroundViewController: ImagePlaygroundViewController ) { imagePlaygroundViewController.dismiss(animated: true) { self.playgroundVC = nil self.onCancel() } } } @available(iOS 27.0, *) struct ImagePlaygroundPopupView: UIViewControllerRepresentable { let sourceImage: UIImage let prompt: String let onComplete: (URL) -> Void let onCancel: () -> Void func makeUIViewController(context: Context) -> ImagePlaygroundPopupController { ImagePlaygroundPopupController( sourceImage: sourceImage, prompt: prompt, onComplete: onComplete, onCancel: onCancel ) } func updateUIViewController( _ uiViewController: ImagePlaygroundPopupController, context: Context ) {} }
0
0
41
1d
AppIntents: how to present a searchable modal picker for an entity relation in an automatic "Find" intent (from EnumerableEntityQuery or EntityPropertyQuery)
Hello, My (fictional) app has multiple app entities like a BookEntity and an AuthorEntity. There is a relationship between them: a book has an author. Because I implement the EntityPropertyQuery for the BookEntityQuery, it creates a “Find Book where...” intent. In this intent, I can filter the data by an author (AuthorEntity). But I can’t find a way to allow the user to select any author using the searchable modal usually presented to “choose” a parameter value from an intent for example. Right now, it shows: a menu with the suggestedEntities() authors if the AuthorEntityQuery implements this function. With no way to search even if the AuthorEntityQuery conforms to the EntityStringQuery. a menu with allEntities() authors if the AuthorEntityQuery conforms to EnumerableEntityQuery and does not implement the suggestedEntities(). With no way to search even if the AuthorEntityQuery conforms to the EntityStringQuery. nothing if suggestedEntities() is not implemented and allEntities() is also not implemented even if the AuthorEntityQuery conforms to the EntityStringQuery. So this means if I implement suggestedEntities(), my users have no way to choose any author not in the suggestions. It’s too limited. if I implement allEntities() but not suggestedEntities(), this could potentially be a lot of entities and it’s not always relevant because then the Shortcuts app with an AuthorEntity parameter will be created for allEntities(), and the UI with always show the full list of allEntities() when I have to pick an author (unless I build a specific DynamicOptionsProvider but this would mean I lose the suggestedEntities() for example. if I just conforms to EntityStringQuery, I can’t select any item at all for this filter (no menu to choose). You can check the attached sample code provided. I checked a bit other apps, for example the Photos app allows you to find a photo based on an album but it lists ALL the albums (which is a lot) with no way to search using string. I also check the Sofa app, it shows the list of SofaListEntity (which implements suggestedEntities() and conforms to EntityStringQuery and EntityQuery). It actually shows all the lists (confirmed by the developer) with no way to search. See attachements from the Sofa developer. Is there a way to support a search for a filter using a relationship conforming to the EntityStringQuery? Is there a way to specify a provider for a @Property of an entity like we can for a @Parameter? For example, I could provide a custom DynamicOptionsProvider for the @Property(title: “Author”) var author which would be independent from the AuthorEntityQuery used elsewhere (parameter intents + Shortcuts). Maybe you can suggest another way to implement this? Maybe I should not rely on a relationship but instead flatten the relationship in the BookEntity to add an “authorName” or “authorID”? I filed this feedback with a sample code: FB23453134 Thanks, Regards, Axel
0
1
48
2d
DeleteIntent: the singular typeDisplayRepresentation of the AppEntity is used instead of the entities parameter title (plural)
Hello, I'm using a DeleteIntent in my app, but I can't get it to display "Delete {Books}" in the Shortcut apps. It always displays "Delete {Book}". It apparently uses the BookEntity typeDisplayRepresentation instead of using the intent entities parameter title ("Books") in the parameterSummary. For a "normal" AppIntent, the Shortcuts app uses the entities parameter title ("Books") in the parameterSummary. I filed a feedback: FB23451186 Regards, Axel
0
0
37
2d
Is UISceneAppIntent supported in Designed for iPad apps on macOS?
I'm seeing what appears to be different UISceneAppIntent behavior between iOS and Designed for iPad on macOS, and I'd like to confirm whether this is expected. I'm working on an iOS app that defines an AppIntent conforming to UISceneAppIntent from the AppIntents framework. The intent is handled by a scene delegate conforming to both UIWindowSceneDelegate and AppIntentSceneDelegate. On iOS, everything works as expected: If the app is launched for the first time from Shortcuts, the intent is available via connectionOptions.appIntent in scene(_:willConnectTo:options:). If the app is already running, scene(_:willPerformAppIntent:) is called. However, when running the same iOS app on macOS in Designed for iPad mode, the behavior is different: If the app is launched from Shortcuts, connectionOptions.appIntent is always nil in scene(_:willConnectTo:options:). If the app is already running, scene(_:willPerformAppIntent:) is never called, even though the application is successfully activated. Is this expected behavior? I noticed that the AppIntents framework explicitly marks both AppIntentSceneDelegate and UISceneAppIntent as unavailable on macOS: @available(iOS 26.0, tvOS 26.0, *) @available(macOS, unavailable) @available(watchOS, unavailable) public protocol AppIntentSceneDelegate : UISceneDelegate Since the app is running on macOS in Designed for iPad mode and still uses the iOS binary, I wasn't sure whether these scene-based APIs are expected to work in this environment or whether they are intentionally unsupported. Has anyone from Apple or the community been able to confirm whether this behavior is by design, or whether it should be considered a bug? I'd appreciate any clarification.
1
0
113
4d
Receiving an on‑screen image from another app via App Intents / Siri (app has no photo library)
I have a photo editing app that owns no photo library. I want a user viewing an image in another app (e.g. Photos) to say "filter this image in MyApp" and have Siri hand that on‑screen image to my intent. Targeting iOS 27. What I've tried, and the result in each case: • App Shortcut + @Parameter var image: IntentFile — Siri resolves my other parameters (a filter AppEnum) by voice, but never binds the image; the run fails. • @AppIntent(schema: .photos.setFilter) with a .photos.asset entity — never routes from Photos. • @AppIntent(schema: .system.open): OpenIntent with a custom AppEntity target — "Open this image in MyApp" just launches the app by name; perform() is never called, and the entity query never runs. My understanding from WWDC26 "Build intelligent Siri experiences with App Schemas" (session 240) and "Discover new capabilities in the App Intents framework" (session 345): • Cross‑app content transfer (Transferable + IntentValueRepresentation) seems limited to system value types (IntentPerson, PlaceDescriptor); IntentFile is not a _SystemIntentValue, so an image can't ride that rail. • Onscreen awareness (NSUserActivity.appEntityIdentifier, View Annotations) appears to expose only the foreground app's own content — which here is Photos, not me. Question: Is there a supported way for a third‑party app to receive another app's on‑screen image (vs. a contact/place) through Siri/App Intents today? If so, which API carries the pixels — an IntentFile parameter, @UnionValue, IntentValueQuery, something else — and what must the source app do to make it available? Or is asking "do X to this image in <third‑party app>" simply not supported yet outside Shortcuts?
1
1
111
4d
Adopting AppIntents and AppEntities for Apple Intelligence
In my app, I adopt ‘App Intents’ and ‘App Entities’ for various items and actions in my app. These work in the Shortcuts app, and users can use Find Intents to lookup information or use the intents to act up on them, even though they don’t map on to any of the specific App ‘Schemes’ and ‘Domains’ that are listed. If I'm using just App Entities and Intents on their own, would these participate with the new Siri AI, without any additional work? For example, in my app, I have entities like Contacts, Tags, and ‘Visit Logs’. I have intents for 'create tag’, ‘add tags to a contact’, add a ‘Log’ to a 'contact' etc. These are specified in the AppShortcutsProvider list, but I have more than 10 intents. How would these work with the new Apple Intelligence and Siri? Or does Siri integration require AppSchema adoption?
8
5
431
5d
FoundationModels Framework on watchOS 27 Beta 2
When importing FoundationModels in watchOS 27 Beta 2 this error appears: /Applications/Xcode-beta.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS27.0.sdk/System/Library/Frameworks/FoundationModels.framework/Modules/FoundationModels.swiftmodule/arm64e-apple-watchos.swiftinterface:6:15 Unable to resolve module dependency: 'CoreImage' Does anybody else have this issue?
1
0
113
5d
Feedback on Foundation Models context management wrapper
I’ve been experimenting with Foundation Models and built a small Swift package that wraps LanguageModelSession with simple context management. The current approach checks the transcript token count using tokenCount(for:), compacts the transcript when it reaches a threshold, and retries once if exceededContextWindowSize is thrown. I’d appreciate feedback on whether this is a sensible use of Foundation Models APIs, especially around rebuilding a session from a compacted Transcript. GitHub: https://github.com/ricky-stone/FoundationContext
1
0
103
5d
Core AI Debugger Beta Questions
Hi guys, I am a new core developer. Recently I am trying the newly released core ai debugger tool and meet some issues. I am not sure whether the related functions of debugger is ready in beta package. Reference Run: aimodel on device versus intermediate dump I used the intermediate exporting api to export the intermediate dump data of MobileNet2.0 and converted the same MobileNet Model to .aimodel. But the compare button is not usable. the suffix of intermediate file suffix is OK(can be selected). The debugger is a GUI so I cannot figure out whether I exported the intermediate data incorrectly. (1st attached png ) on device model run compare I just simply tried the different computing unit and different input for a simple 'add/sub/mul/div' toy model. Especially set the input different but the comparison results seems to be empty I am not sure whether I modify the debugger tool incorrectly. Hope someone sharing experience. My Gratitude.
Replies
0
Boosts
0
Views
18
Activity
6h
Can output images from imagePlaygroundSheet be used as input for third-party video generation APIs?
I'm developing an iOS app that uses imagePlaygroundSheet to generate images based on AI-generated prompts (not user-typed text). I would like to confirm whether the following usage is permitted under the Apple Developer Program License Agreement: Generate an image using imagePlaygroundSheet within my app Pass the resulting image (obtained via the completion handler URL) as an input to a third-party video generation API (e.g., to create an image-to-video animation) Offer this video generation as a paid feature (consumable in-app purchase) within my app I've reviewed Section 3.2(8) of the ADPLA, which prohibits using "output generated from an Apple model to train, fine-tune, or improve another artificial intelligence model." My understanding is that this clause is specifically about training/improving AI models, not about using the output image as a content input for a separate processing step (e.g., animation). Could someone confirm: Whether passing imagePlaygroundSheet output to a third-party API for further processing (not training) is permitted Whether monetizing a feature that uses this combined output is permitted Thank you in advance for any guidance.
Replies
0
Boosts
0
Views
17
Activity
6h
I got old Siri UI instead
I’m on a base iPhone 17 and I‘ve been approved from the waitlist over two weeks ago and when I use Siri, I get the old Siri from before Apple Intelligence was a thing (Glowy orb at the bottom). Inside Siri settings, it looks like it’s supposed to be set up for the new SirI as it shows the app settings. Here are my observations I have found from using beta 1 to 2. Beta 1: Whenever I have the new Siri “enabled”, there is no app I can type “siri://“ and it‘ll open the Siri open saying that’s there’s an update in progress. But if I type that in when I’m using the previous Siri, the app is there and it’s blank When new Siri is ”enabled” and whenever I ask Siri a question she’ll either think and never stop or output a blank answer. But when I turn Siri back to its previous version and open the “sir://“, it stores the conversation and when I open it, it actually answered my question. Beta 2: When installing beta 2 I said “new Siri“ enabled Once it was done installing, I went to iPhone storage and saw that “Apple Intelligence“ storage is increasing size. But then after a minute, it went back down to 14GB Still gave me the old Siri UI Now the siri app is not accessible when typing “siri://“ Siri mode is now in the camera app but it says it’s unavailable The ChatGPT extension area is now blocked because assets are downloading Whenever I visual intelligence and press ask it says that siri support is downloading I left siri alone for 3 days and nothing happened. Still stuck with old siri The ask siri option is everywhere now and when I press it, nothing happens Right now I’m using the previous version of Siri and hoping this bug will be patched and I can use the new Siri AI in Beta 3. *the screenshot provided shows what happens when I use Siri in beta 2 when it’s enabled.
Replies
0
Boosts
0
Views
28
Activity
6h
LongRunningIntent run from in the app?
If I attempt to use a LongRunningIntent from a SwiftUI Button, using the Button(_:AppIntent:) control, I get the following errors: [LongRunningIntent <<E:<unknown>>>] No IntentContext available performBackgroundTask threw: noContext Intent failed to execute with error: LNPerformActionErrorCodeUnsupportedValueType It runs as expected when run from the Shortcuts app, but fails when run from a button within the app. Feedback ID (with sample app): FB23492034
Replies
1
Boosts
1
Views
16
Activity
7h
IndexedEntities and Siri AI
Currently, I have spotlight entities show up when I search for them using Spotlight on iOS 27. These entities are things that are important for users, like campus buildings, accessible entrances, assignments, and more. However, after getting access to Siri AI, it seems that none of this information at all is available to Siri, yet all of it is sitting there in the spotlight index and viewable with a written query. I was told by an Apple Engineer that creating Indexed and EnumerableEntities, and indexing them via the App Intents framework, should expose information about these items to Siri, so if I query: "[Building name] in Ohio State" it would at least show me what the app has for that information. Presently, Siri uses the web for everything and doesn't pull in any spotlight information for my app, despite either creating wrapper entities or using the API associating with spotlight. With Siri AI, it would be so much more helpful for a disabled user to say "Orton Hall accessible entrance" and Siri to know that there's 1 accessible entrance indexed in spotlight in my app, and then show or open it, instead of querying the web or saying it can't answer the question. It has all available information already in spotlight to answer this question. Currently, as far as I'm aware, something like this simply doesn't work, unless your app conforms to the strict use cases of making reminders or calendar events, all of which aren't useful here. Can a Frameworks engineer please clarify precisely when and how IndexedEntities (paired with an a corresponding macro-annotated OpenIntent) eg: @AppIntent(schema: .system.open) struct OpenBuildingIntent: OpenIntent { @Parameter(title: "Building") var Building: BuildingEntity ... will or will not be visible using Siri AI? To me it seems I have wasted a lot of time porting actions within my app to App Intents, and viewable entities with AppEntity, only to have Siri not be able to use any of this information out of the box.
Replies
2
Boosts
0
Views
191
Activity
7h
App schema domains for trains, flighs, ferrys and othe public transport
Hi everyone, I've submitted a Feedback request (FB23469644) asking to introduce a generic Transportation Journey schema for App Intents. At the moment, there doesn't seem to be a way to donate structured public transportation trips (train, flight, bus, ferry, etc.) to Siri's semantic index. While many travel apps already have rich journey data, there is no common semantic model that represents: Multi-leg journeys Departure and arrival stations/airports/stops Platforms, tracks, gates, terminals Scheduled and real-time departure/arrival information Delays and cancellations Ticket and reservation information This seems like a natural addition to the existing App Intents schemas and would enable Siri, Spotlight, and Apple Intelligence to better understand upcoming journeys across all transportation providers. If you're building a travel or mobility app and have encountered the same limitation, I'd appreciate hearing your thoughts. Any idea if the old Siri event suggestion API gets is indexed by Siri Semantic Index / Siri AI?
Replies
3
Boosts
6
Views
91
Activity
7h
Sensitive Content Error When Using Foundation Models
I am using the following code in my iOS application. #Playground { let session = LanguageModelSession() let response = try await session.respond(to: "List all states of USA.") print(response.content) } And I get the following error: The operation couldn’t be completed. (com.apple.SensitiveContentAnalysisML error 15.) I have turned off Apple Intelligence and turned on again. No use. I am using Xcode 27 beta 2. any ideas?
Replies
1
Boosts
0
Views
51
Activity
20h
AppIntents and String catalog: how can we support both singular and plural forms for TypeDisplayRepresentation (used by DeleteIntent in the Shortcuts app for example)
Hello, I’m implementing the AppIntents framework in my app. I want to translate the TypeDisplayRepresentation that is used in the Shortcuts app UI like in a DeleteIntent (see my feedback about this: FB23451186 for more context). In the “Accelerating app interactions with App Intents” sample code we can see that this is done using a .stringsdict file, as follows for the “Trail” key (singular and plural): <key>Trail</key> <dict> <key>NSStringLocalizedFormatKey</key> <string>%#@VARIABLE@</string> <key>VARIABLE</key> <dict> <key>NSStringFormatSpecTypeKey</key> <string>NSStringPluralRuleType</string> <key>one</key> <string>Trail</string> <key>other</key> <string>Trails</string> </dict> </dict> I want to use a String catalog instead of a .stringsdict file because all my strings are in a String catalog. I tried to migrate the AppIntents.stringsdict file manually but it failed with an error: “An error occurred when migrating AppIntentsSampleApp/Resources/AppIntents.stringsdict: This stringsdict cannot be migrated: Missing required key 'NSStringFormatValueTypeKey' inside 'Trail' -> 'VARIABLE’” So I manually added a NSStringFormatValueTypeKey like this in the .stringsdict file: <key>Trail</key> <dict> <key>NSStringLocalizedFormatKey</key> <string>%#@VARIABLE@</string> <key>VARIABLE</key> <dict> <key>NSStringFormatSpecTypeKey</key> <string>NSStringPluralRuleType</string> <key>NSStringFormatValueTypeKey</key> <string>lld</string> <key>one</key> <string>Trail</string> <key>other</key> <string>Trails</string> </dict> </dict> And then I’ve been able to migrate the .stringsdict file into a String catalog. The string catalog looks like this after migration: "Trail" : { "extractionState" : "stale", "localizations" : { "en" : { "stringUnit" : { "state" : "translated", "value" : "%#@VARIABLE@" }, "substitutions" : { "VARIABLE" : { "formatSpecifier" : "lld", "variations" : { "plural" : { "one" : { "stringUnit" : { "state" : "translated", "value" : "Trail (Catalog)" } }, "other" : { "stringUnit" : { "state" : "translated", "value" : "Trails (Catalog)" } } } } } } } } This works, which is nice. But I tried to reproduce the same result by having a %#@VARIABLE@ in my LocalizedStringResource defaultValue like this: TypeDisplayRepresentation( name: LocalizedStringResource( "Flower", defaultValue: "%#@VARIABLE@", table: "AppIntents" ), numericFormat: LocalizedStringResource( "\(placeholder: .int) flower", table: "AppIntents" ) ) But the String catalog doesn’t support that apparently, I can’t get a “substitution” object in my catalog, so I have to manually do it using the source code which is not ideal and painful. Is there a way to support this kind of substitution with no actual plural token in the string as we can see for the %#@VARIABLE@ for Trail? Thank you, Regards, Axel
Replies
1
Boosts
0
Views
79
Activity
1d
Bring an LLM provider to the Foundation Models, missing MLX dependencies
On this talk: Bring an LLM provider to the Foundation Models framework URL: https://developer.apple.com/videos/play/wwdc2026/339/ on the coding examples a very peculiar framework is shown: import MLXFoundationModels However I am not able to find it nowhere, there is even a code section with this framework as part of an example. Where is this framework, there are no BETA branches on the MLX framework either. Thanks!
Replies
2
Boosts
0
Views
91
Activity
1d
iOS 27 ImagePlaygroundViewController.Delegate not working?
In the WWDC 2026 sessions it was called out in code that the helper functions would still work, however they don't seem to be working either inside a UIViewRepresentable, nor as a UIKit View as below (also tried as a sheet, to no avail). Otherwise it works. Is there something else I'm missing? import SwiftUI import ImagePlayground @available(iOS 27.0, *) final class ImagePlaygroundPopupController: UIViewController { var sourceImage: UIImage var prompt: String var onComplete: (URL) -> Void var onCancel: () -> Void private var didPresent = false private var playgroundVC: ImagePlaygroundViewController? init( sourceImage: UIImage, prompt: String, onComplete: @escaping (URL) -> Void, onCancel: @escaping () -> Void ) { self.sourceImage = sourceImage self.prompt = prompt self.onComplete = onComplete self.onCancel = onCancel super.init(nibName: nil, bundle: nil) view.backgroundColor = .clear } required init?(coder: NSCoder) { fatalError("init(coder:) has not been implemented") } override func viewDidAppear(_ animated: Bool) { super.viewDidAppear(animated) guard !didPresent else { return } didPresent = true let vc = ImagePlaygroundViewController() vc.sourceImage = sourceImage vc.concepts = [.text(prompt)] vc.delegate = self playgroundVC = vc present(vc, animated: true) } } @available(iOS 27.0, *) extension ImagePlaygroundPopupController: ImagePlaygroundViewController.Delegate { func imagePlaygroundViewController( _ imagePlaygroundViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL ) { imagePlaygroundViewController.dismiss(animated: true) { self.playgroundVC = nil self.onComplete(imageURL) } } func imagePlaygroundViewControllerDidCancel( _ imagePlaygroundViewController: ImagePlaygroundViewController ) { imagePlaygroundViewController.dismiss(animated: true) { self.playgroundVC = nil self.onCancel() } } } @available(iOS 27.0, *) struct ImagePlaygroundPopupView: UIViewControllerRepresentable { let sourceImage: UIImage let prompt: String let onComplete: (URL) -> Void let onCancel: () -> Void func makeUIViewController(context: Context) -> ImagePlaygroundPopupController { ImagePlaygroundPopupController( sourceImage: sourceImage, prompt: prompt, onComplete: onComplete, onCancel: onCancel ) } func updateUIViewController( _ uiViewController: ImagePlaygroundPopupController, context: Context ) {} }
Replies
0
Boosts
0
Views
41
Activity
1d
AppIntents: how to present a searchable modal picker for an entity relation in an automatic "Find" intent (from EnumerableEntityQuery or EntityPropertyQuery)
Hello, My (fictional) app has multiple app entities like a BookEntity and an AuthorEntity. There is a relationship between them: a book has an author. Because I implement the EntityPropertyQuery for the BookEntityQuery, it creates a “Find Book where...” intent. In this intent, I can filter the data by an author (AuthorEntity). But I can’t find a way to allow the user to select any author using the searchable modal usually presented to “choose” a parameter value from an intent for example. Right now, it shows: a menu with the suggestedEntities() authors if the AuthorEntityQuery implements this function. With no way to search even if the AuthorEntityQuery conforms to the EntityStringQuery. a menu with allEntities() authors if the AuthorEntityQuery conforms to EnumerableEntityQuery and does not implement the suggestedEntities(). With no way to search even if the AuthorEntityQuery conforms to the EntityStringQuery. nothing if suggestedEntities() is not implemented and allEntities() is also not implemented even if the AuthorEntityQuery conforms to the EntityStringQuery. So this means if I implement suggestedEntities(), my users have no way to choose any author not in the suggestions. It’s too limited. if I implement allEntities() but not suggestedEntities(), this could potentially be a lot of entities and it’s not always relevant because then the Shortcuts app with an AuthorEntity parameter will be created for allEntities(), and the UI with always show the full list of allEntities() when I have to pick an author (unless I build a specific DynamicOptionsProvider but this would mean I lose the suggestedEntities() for example. if I just conforms to EntityStringQuery, I can’t select any item at all for this filter (no menu to choose). You can check the attached sample code provided. I checked a bit other apps, for example the Photos app allows you to find a photo based on an album but it lists ALL the albums (which is a lot) with no way to search using string. I also check the Sofa app, it shows the list of SofaListEntity (which implements suggestedEntities() and conforms to EntityStringQuery and EntityQuery). It actually shows all the lists (confirmed by the developer) with no way to search. See attachements from the Sofa developer. Is there a way to support a search for a filter using a relationship conforming to the EntityStringQuery? Is there a way to specify a provider for a @Property of an entity like we can for a @Parameter? For example, I could provide a custom DynamicOptionsProvider for the @Property(title: “Author”) var author which would be independent from the AuthorEntityQuery used elsewhere (parameter intents + Shortcuts). Maybe you can suggest another way to implement this? Maybe I should not rely on a relationship but instead flatten the relationship in the BookEntity to add an “authorName” or “authorID”? I filed this feedback with a sample code: FB23453134 Thanks, Regards, Axel
Replies
0
Boosts
1
Views
48
Activity
2d
DeleteIntent: the singular typeDisplayRepresentation of the AppEntity is used instead of the entities parameter title (plural)
Hello, I'm using a DeleteIntent in my app, but I can't get it to display "Delete {Books}" in the Shortcut apps. It always displays "Delete {Book}". It apparently uses the BookEntity typeDisplayRepresentation instead of using the intent entities parameter title ("Books") in the parameterSummary. For a "normal" AppIntent, the Shortcuts app uses the entities parameter title ("Books") in the parameterSummary. I filed a feedback: FB23451186 Regards, Axel
Replies
0
Boosts
0
Views
37
Activity
2d
Where is my new siri??
still no sign of the new siri no app no nothing im on the ios 27 beta 2 and iphone 15 pro max what is this apple
Replies
0
Boosts
0
Views
76
Activity
4d
Is UISceneAppIntent supported in Designed for iPad apps on macOS?
I'm seeing what appears to be different UISceneAppIntent behavior between iOS and Designed for iPad on macOS, and I'd like to confirm whether this is expected. I'm working on an iOS app that defines an AppIntent conforming to UISceneAppIntent from the AppIntents framework. The intent is handled by a scene delegate conforming to both UIWindowSceneDelegate and AppIntentSceneDelegate. On iOS, everything works as expected: If the app is launched for the first time from Shortcuts, the intent is available via connectionOptions.appIntent in scene(_:willConnectTo:options:). If the app is already running, scene(_:willPerformAppIntent:) is called. However, when running the same iOS app on macOS in Designed for iPad mode, the behavior is different: If the app is launched from Shortcuts, connectionOptions.appIntent is always nil in scene(_:willConnectTo:options:). If the app is already running, scene(_:willPerformAppIntent:) is never called, even though the application is successfully activated. Is this expected behavior? I noticed that the AppIntents framework explicitly marks both AppIntentSceneDelegate and UISceneAppIntent as unavailable on macOS: @available(iOS 26.0, tvOS 26.0, *) @available(macOS, unavailable) @available(watchOS, unavailable) public protocol AppIntentSceneDelegate : UISceneDelegate Since the app is running on macOS in Designed for iPad mode and still uses the iOS binary, I wasn't sure whether these scene-based APIs are expected to work in this environment or whether they are intentionally unsupported. Has anyone from Apple or the community been able to confirm whether this behavior is by design, or whether it should be considered a bug? I'd appreciate any clarification.
Replies
1
Boosts
0
Views
113
Activity
4d
Receiving an on‑screen image from another app via App Intents / Siri (app has no photo library)
I have a photo editing app that owns no photo library. I want a user viewing an image in another app (e.g. Photos) to say "filter this image in MyApp" and have Siri hand that on‑screen image to my intent. Targeting iOS 27. What I've tried, and the result in each case: • App Shortcut + @Parameter var image: IntentFile — Siri resolves my other parameters (a filter AppEnum) by voice, but never binds the image; the run fails. • @AppIntent(schema: .photos.setFilter) with a .photos.asset entity — never routes from Photos. • @AppIntent(schema: .system.open): OpenIntent with a custom AppEntity target — "Open this image in MyApp" just launches the app by name; perform() is never called, and the entity query never runs. My understanding from WWDC26 "Build intelligent Siri experiences with App Schemas" (session 240) and "Discover new capabilities in the App Intents framework" (session 345): • Cross‑app content transfer (Transferable + IntentValueRepresentation) seems limited to system value types (IntentPerson, PlaceDescriptor); IntentFile is not a _SystemIntentValue, so an image can't ride that rail. • Onscreen awareness (NSUserActivity.appEntityIdentifier, View Annotations) appears to expose only the foreground app's own content — which here is Photos, not me. Question: Is there a supported way for a third‑party app to receive another app's on‑screen image (vs. a contact/place) through Siri/App Intents today? If so, which API carries the pixels — an IntentFile parameter, @UnionValue, IntentValueQuery, something else — and what must the source app do to make it available? Or is asking "do X to this image in <third‑party app>" simply not supported yet outside Shortcuts?
Replies
1
Boosts
1
Views
111
Activity
4d
Joined waitlist for Siri AI
Actually trick to change language to English UK, restart and change back to English US worked for me. After this in an hour notification showed up and Siri AI was ready.
Replies
0
Boosts
0
Views
97
Activity
4d
Siri AI Waitlist 100+ Hours
I live in Canada and I called apple support, they said the new Siri ai isn’t available in some parts of Canada. I’ve changed my region to united states and now it’s just a waiting game. If i now get it fast and quick I’ll let u guys know
Replies
1
Boosts
0
Views
134
Activity
5d
Adopting AppIntents and AppEntities for Apple Intelligence
In my app, I adopt ‘App Intents’ and ‘App Entities’ for various items and actions in my app. These work in the Shortcuts app, and users can use Find Intents to lookup information or use the intents to act up on them, even though they don’t map on to any of the specific App ‘Schemes’ and ‘Domains’ that are listed. If I'm using just App Entities and Intents on their own, would these participate with the new Siri AI, without any additional work? For example, in my app, I have entities like Contacts, Tags, and ‘Visit Logs’. I have intents for 'create tag’, ‘add tags to a contact’, add a ‘Log’ to a 'contact' etc. These are specified in the AppShortcutsProvider list, but I have more than 10 intents. How would these work with the new Apple Intelligence and Siri? Or does Siri integration require AppSchema adoption?
Replies
8
Boosts
5
Views
431
Activity
5d
FoundationModels Framework on watchOS 27 Beta 2
When importing FoundationModels in watchOS 27 Beta 2 this error appears: /Applications/Xcode-beta.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS27.0.sdk/System/Library/Frameworks/FoundationModels.framework/Modules/FoundationModels.swiftmodule/arm64e-apple-watchos.swiftinterface:6:15 Unable to resolve module dependency: 'CoreImage' Does anybody else have this issue?
Replies
1
Boosts
0
Views
113
Activity
5d
Feedback on Foundation Models context management wrapper
I’ve been experimenting with Foundation Models and built a small Swift package that wraps LanguageModelSession with simple context management. The current approach checks the transcript token count using tokenCount(for:), compacts the transcript when it reaches a threshold, and retries once if exceededContextWindowSize is thrown. I’d appreciate feedback on whether this is a sensible use of Foundation Models APIs, especially around rebuilding a session from a compacted Transcript. GitHub: https://github.com/ricky-stone/FoundationContext
Replies
1
Boosts
0
Views
103
Activity
5d