Posts under Machine Learning & AI topic

Post

Replies

Boosts

Views

Activity

siri waitlist
How are some people getting access within 10 minutes while I've been waiting for over a week? In fact, it's been 48 hrs since I joined the waitlist. This is honestly ridiculous. I've met all the requirements, enabled every necessary setting, and still have no access. Meanwhile, others are getting approved almost instantly. If the rollout is based on a waitlist, then it should be handled fairly and consistently. Waiting this long for a feature update is extremely frustrating, especially when there has been little to no communication about the delay.
0
0
52
2w
Use different model in foundation model
Hi everyone, I’m working with the WWDC26 Foundation Models framework and would like to know how to precisely control which model is used. Specifically: On-Device: How can I force SystemLanguageModel() to use AFM 3 Core Advanced (the 20B sparse multimodal variant) instead of automatically falling back to the 3B Core? Is there an API to query or explicitly specify the on-device model variant? Private Cloud Compute (PCC): When using PrivateCloudComputeLanguageModel(), how can I ensure it uses AFM 3 Cloud Pro instead of the regular Cloud model? Does setting ContextOptions.reasoningLevel = .deep guarantee the Pro model, or is it still determined automatically by the backend? So far I can only check model.capabilities, but there’s no clear way to confirm which exact model variant is actually running. Are there more granular APIs, DynamicProfile modifiers, or Instruments methods to achieve precise control? Any insights, official documentation, or WWDC session references would be greatly appreciated!
1
0
224
2w
SpotlightSearchTool arguments: description vs. JSON Schema mismatch → “Failed to parse generated content”
Using SpotlightSearchTool with a custom LanguageModel backend (Apple’s ChatCompletionsLanguageModel from foundation-models-utilities, pointed at an OpenAI-compatible server), every tool call fails with ToolCallError → "Failed to parse generated content." The model follows the tool’s documented "Call format" and emits { root, modelComposition, … }. But the generated parameters schema (FullArguments) requires { "query": { "type": "search", "value": { root, modelComposition, … } } }. Query is a QueryType union and a search must be wrapped in DiscriminatedSearch. Wrapping the args manually makes it parse and search correctly. So the description omits the query + type:"search" envelope the schema demands, which makes the tool uninvokable by any model that follows the documentation (it presumably works only with the on-device model trained on the real format). Is this a known issue / intended? Anyone gotten SpotlightSearchTool working with a non-Apple model? Secondary: CoreSpotlightSource.fetchAttributes seems to have no effect on returned attributes. kMDItemDescription only comes back when the in-query fetchAttributes requests it. Bug or expected?
1
0
119
2w
MLX,MLX LM, MLX LM Server -> Is there a bootstrap repo?
theres a MLX, a MLX LM and a MLX LM Server mentioned. Is there a Bootstrap GitHub repo out there that can be used to directly, and quickly, set up an example of this, without the hassle of setting up, kind of like a bootstrap for "us mere mortals"? And what is the feasibility of using these on a M3 Pro with 18Gb of memory? - can these be bounced between a local M3 Pro and a Tailscale-linked M2 Pro with 36Gb memory? Do both need to be on macOS27 for it to work?
1
0
78
2w
How to obtain more value out of a generic "FoundationModels.LanguageModelError error -1"
I created a tiny sample: provided a session with a tiny tool to tell the date and time to the model. Asked the model to stream response to "What time is it" and just get this error: The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.) There is no relevant output on Xcode console. And profiling with the new Foundation Models instrument brings zero more insight into the issue. I know it could be many things, but it there may be more information the tools could surface in debug?
4
0
83
2w
Cannot pattern match LanguageModelError from a response stream
The LanguageModelSession.GenerationErrors seems to be deprecated in favor of LanguageModelError for the most part. Now... when iterating through the ResponseStream<String> of a LanguageModelSession.streamResponse(to:options:), with a good old for await, the async iterator .next() can throws. Leaving aside that it is not very conspicuous at the call site it will throw... in the do/catch, the error thrown does not see to be able to be pattern matched to the new LanguageModelError with something like catch let error as LanguageModelError. It was able to patten match the GenerationErrors before just fine, so may be an oversight/bug?
3
0
97
2w
App Intent photos createAssets can't receive user selected files
When implementing @AppIntent(schema: .photos.createAssets) in my app I can accept files to import as assets into my app. Therefore I would expect be able to use this in Shortcuts and pass in a selected file to my createAssets intent. For example using the Files app Select Files action and pass the selected files on to my photos.createAssets action. However doesn't seem to be possible in any way. The only thing I can do in Shortcuts is select a file for the create assets action at the time I make the shortcut. Which removes all flexibility. Is this a bug or a missing feature? Or am I missing something how to set this up correctly? I've been testing this with the new "Integrating your photo app with Apple Intelligence" Apple Sample and it's not working with that in any way. FB23066510
1
0
85
2w
Nearly 70 hours on Siri waitlist
Downloaded the moment WWDC presentation ended at 11a PT. Phone 17 Air, English (US) Plugged in over (3) nights Still indexing Rebooted 5+ times Still waitlisted. The only feature anyone cares about is gated. So much for “available immediately“. Losing trust with Apple fast here.
0
0
244
2w
Sandbox-Bypassing IPC between App Intents and Launchd Daemons on macOS
We are designing a macOS utility that runs a local background agent via launchd (managing a local SQLite database and Unix socket). We want to expose controls (start/stop, status checks, CLI command invocation) to Siri via the App Intents framework. Since App Intents typically execute within a sandboxed App Extension or a sandboxed App wrapper container: What is the recommended IPC mechanism (e.g., Unix domain sockets, local HTTP/TCP ports, XPC) to securely communicate between a sandboxed App Intent extension and a non-sandboxed launchd helper daemon on macOS? Are there specific Entitlements (com.apple.security.temporary-exception.files.absolute-path.read-write or network exceptions) required to allow App Intents to talk to local UNIX sockets or loopback interfaces (127.0.0.1) without triggering sandbox violations? Can an App Intent directly invoke a helper command-line tool or launch a plist-configured service without bringing up the main application UI?
2
0
144
2w
Meal nutrition data/Siri AI and iOS27 vision framework
I'm an app developer with an interest in nutrition data. While watching the Keynote segment on Monday, there was a brief section about Siri AI and iOS 27 Vision framework being used to scan a meal with an iPhone camera and display the meal nutritional content. I have two questions: Do the displayed data include GI (glycemic index) and/or GL (glycemic load)? and Are these nutrition data surfaced anywhere, especially in HealthKit, or — if not — will Apple make a public API available so developers can use these data directly in iOS27 apps?
3
0
139
2w
How can I get Siri to recognize "43rd and Broadway"?
Hi all, This is a simplification of an AppIntent where I need my phrases to recognize three "regions" and a specific set of numbers associated with those regions. I am having trouble defining the AppEntity and associated AppShortcut phrases that works well with Siri, which is having trouble extracting numbers for some reason... Now the example: Imagine my app provides information surrounding Starbucks establishments on three major thoroughfares in NYC, such as: Broadway & 26, 40, 43, 54, 63, 75, 86, 95 Madison & 44, 50, 84, 96 Lexington & 43, 48, 55, 63, 67, 77, 85, 96 I need AppEntities, AppShortcuts, and friends that best work with Siri when launched with phrases such as: "Show 'example' for Madison and 50th in \(.applicationName)", "Show 'example' on 43rd and Broadway in \(.applicationName)", Long story short, Siri is picking up my phrases well, but then asking me to answer whether the value is 44, 50, 84, or 96, instead of just hearing that I said: "Madison and 50th" I've defined streets like this: public let nycStreetDisplayRepresentations: [DisplayRepresentation] = [ DisplayRepresentation(title: "26", synonyms: ["twenty six", "twenty sixth"]), DisplayRepresentation(title: "40", synonyms: ["forty", "fortieth"]), DisplayRepresentation(title: "43", synonyms: ["forty three", "forty third"]), DisplayRepresentation(title: "48", synonyms: ["forty eight", "forty eighth"]), DisplayRepresentation(title: "50", synonyms: ["fifty", "fiftieth"]), DisplayRepresentation(title: "54", synonyms: ["fifty four", "fifty fourth"]), DisplayRepresentation(title: "55", synonyms: ["fifty five", "fifty fifth", "five five"]), DisplayRepresentation(title: "63", synonyms: ["sixty three", "sixty third"]), DisplayRepresentation(title: "67", synonyms: ["sixty seven", "sixty seventh"]), DisplayRepresentation(title: "75", synonyms: ["seventy five", "seventy fifth"]), DisplayRepresentation(title: "77", synonyms: ["seventy seven", "seventy seventh", "seven seven"]), DisplayRepresentation(title: "84", synonyms: ["eighty four", "eighty fourth"]), DisplayRepresentation(title: "85", synonyms: ["eighty five", "eighty fifth"]), DisplayRepresentation(title: "86", synonyms: ["eighty six", "eighty sixth"]), DisplayRepresentation(title: "95", synonyms: ["ninety five", "ninety fifth"]), DisplayRepresentation(title: "96", synonyms: ["ninety six", "ninety sixth"]) ] And an entity like this: public struct MadisonThings: AppEntity, Identifiable { public let id: Int public var displayRepresentation: DisplayRepresentation { nycStreetDisplayRepresentations[id-1] } public static var typeDisplayRepresentation = TypeDisplayRepresentation( name: "Madison" , synonyms: ["Madison Ave"] ) public static var defaultQuery = MadisonTHingsQuery() } An EntityQuery: public struct MadisonThingsQuery: EntityQuery, EntityStringQuery { ...attempts to find synonyms in the phrase } Intent: public struct ShowMadisonThingsIntent: AppIntent { { @Parameter( title: "Street Number", description: "The street number (e.g., 5, 12, 44)" ) var streetNumber: MadisonThings // I have also tried just plain 'int' to no avail } And finally the shortcuts: static var appShortcuts: [AppShortcut] { return [ AppShortcut( intent: ShowMadisonThingsIntent(), phrases: [ "Show example for Madison and \(\.$streetNumber) in \(.applicationName)", "Show example on \(\.$streetNumber) in \(.applicationName)", ... This should be very simple, I think, and I've tried multiple ways of extracting numbers from my speech, but for some reason this is not working well at all. Thanks for your help!
3
0
165
2w
Can I import images into my app with App Intent Schemas?
I'm trying to create an App Intent that takes in an image and copies it to a destination within my app. Sure, this can be done with shortcuts, but that's lame, I want siri to grab an image off the screen and invoke the action! Without an app schema, it doesn't seem like siri can tell what's possible in the app, so I'm trying to find a schema that fits? I've started trying to do this with the photos.addAssetsToAlbum App Intent schema, but it doesn't seem like Siri is onboard with the concept. It's replied many times that it is unable to add things to my App and that I should try via other means like opening my app and importing directly. Should be I be trying this with a different intent schema? I was thinking about files.open, but documentation says that Schema is not available to Siri.
3
0
124
2w
siri waitlist
How are some people getting access within 10 minutes while I've been waiting for over a week? In fact, it's been 48 hrs since I joined the waitlist. This is honestly ridiculous. I've met all the requirements, enabled every necessary setting, and still have no access. Meanwhile, others are getting approved almost instantly. If the rollout is based on a waitlist, then it should be handled fairly and consistently. Waiting this long for a feature update is extremely frustrating, especially when there has been little to no communication about the delay.
Replies
0
Boosts
0
Views
52
Activity
2w
Use different model in foundation model
Hi everyone, I’m working with the WWDC26 Foundation Models framework and would like to know how to precisely control which model is used. Specifically: On-Device: How can I force SystemLanguageModel() to use AFM 3 Core Advanced (the 20B sparse multimodal variant) instead of automatically falling back to the 3B Core? Is there an API to query or explicitly specify the on-device model variant? Private Cloud Compute (PCC): When using PrivateCloudComputeLanguageModel(), how can I ensure it uses AFM 3 Cloud Pro instead of the regular Cloud model? Does setting ContextOptions.reasoningLevel = .deep guarantee the Pro model, or is it still determined automatically by the backend? So far I can only check model.capabilities, but there’s no clear way to confirm which exact model variant is actually running. Are there more granular APIs, DynamicProfile modifiers, or Instruments methods to achieve precise control? Any insights, official documentation, or WWDC session references would be greatly appreciated!
Replies
1
Boosts
0
Views
224
Activity
2w
SpotlightSearchTool arguments: description vs. JSON Schema mismatch → “Failed to parse generated content”
Using SpotlightSearchTool with a custom LanguageModel backend (Apple’s ChatCompletionsLanguageModel from foundation-models-utilities, pointed at an OpenAI-compatible server), every tool call fails with ToolCallError → "Failed to parse generated content." The model follows the tool’s documented "Call format" and emits { root, modelComposition, … }. But the generated parameters schema (FullArguments) requires { "query": { "type": "search", "value": { root, modelComposition, … } } }. Query is a QueryType union and a search must be wrapped in DiscriminatedSearch. Wrapping the args manually makes it parse and search correctly. So the description omits the query + type:"search" envelope the schema demands, which makes the tool uninvokable by any model that follows the documentation (it presumably works only with the on-device model trained on the real format). Is this a known issue / intended? Anyone gotten SpotlightSearchTool working with a non-Apple model? Secondary: CoreSpotlightSource.fetchAttributes seems to have no effect on returned attributes. kMDItemDescription only comes back when the in-query fetchAttributes requests it. Bug or expected?
Replies
1
Boosts
0
Views
119
Activity
2w
Spoken Locale Exposure (Dynamic Language Routing)
Does the App Intents framework expose the user's active spoken Siri locale (e.g., ja-JP, fr-FR) directly within the perform() context, or must the extension rely on the system's global locale setting? If a user switches Siri's language dynamically, how is that locale string propagated to the intent execution block?
Replies
1
Boosts
0
Views
104
Activity
2w
Siri Ai
I don’t think this Siri waitlist is normal, I am on iPhone 16 Pro Max in the US set to English and I’ve been on the new Siri waitlist for >48 hours. Is this a bug?
Replies
0
Boosts
0
Views
83
Activity
2w
MLX,MLX LM, MLX LM Server -> Is there a bootstrap repo?
theres a MLX, a MLX LM and a MLX LM Server mentioned. Is there a Bootstrap GitHub repo out there that can be used to directly, and quickly, set up an example of this, without the hassle of setting up, kind of like a bootstrap for "us mere mortals"? And what is the feasibility of using these on a M3 Pro with 18Gb of memory? - can these be bounced between a local M3 Pro and a Tailscale-linked M2 Pro with 36Gb memory? Do both need to be on macOS27 for it to work?
Replies
1
Boosts
0
Views
78
Activity
2w
Photos grid layout for .photos.asset entities in Spotlight
When using the .photos.asset schema for an AppEntity conforming to IndexedEntity, is there a way to trigger the rich grid layout in Spotlight results that native apps like Photos and Messages currently utilize?
Replies
1
Boosts
0
Views
102
Activity
2w
How to obtain more value out of a generic "FoundationModels.LanguageModelError error -1"
I created a tiny sample: provided a session with a tiny tool to tell the date and time to the model. Asked the model to stream response to "What time is it" and just get this error: The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.) There is no relevant output on Xcode console. And profiling with the new Foundation Models instrument brings zero more insight into the issue. I know it could be many things, but it there may be more information the tools could surface in debug?
Replies
4
Boosts
0
Views
83
Activity
2w
Cannot pattern match LanguageModelError from a response stream
The LanguageModelSession.GenerationErrors seems to be deprecated in favor of LanguageModelError for the most part. Now... when iterating through the ResponseStream<String> of a LanguageModelSession.streamResponse(to:options:), with a good old for await, the async iterator .next() can throws. Leaving aside that it is not very conspicuous at the call site it will throw... in the do/catch, the error thrown does not see to be able to be pattern matched to the new LanguageModelError with something like catch let error as LanguageModelError. It was able to patten match the GenerationErrors before just fine, so may be an oversight/bug?
Replies
3
Boosts
0
Views
97
Activity
2w
App Intent photos createAssets can't receive user selected files
When implementing @AppIntent(schema: .photos.createAssets) in my app I can accept files to import as assets into my app. Therefore I would expect be able to use this in Shortcuts and pass in a selected file to my createAssets intent. For example using the Files app Select Files action and pass the selected files on to my photos.createAssets action. However doesn't seem to be possible in any way. The only thing I can do in Shortcuts is select a file for the create assets action at the time I make the shortcut. Which removes all flexibility. Is this a bug or a missing feature? Or am I missing something how to set this up correctly? I've been testing this with the new "Integrating your photo app with Apple Intelligence" Apple Sample and it's not working with that in any way. FB23066510
Replies
1
Boosts
0
Views
85
Activity
2w
Nearly 70 hours on Siri waitlist
Downloaded the moment WWDC presentation ended at 11a PT. Phone 17 Air, English (US) Plugged in over (3) nights Still indexing Rebooted 5+ times Still waitlisted. The only feature anyone cares about is gated. So much for “available immediately“. Losing trust with Apple fast here.
Replies
0
Boosts
0
Views
244
Activity
2w
Dynamic Runtime App Shortcuts
Can App Shortcuts (and their trigger phrases) be generated or translated dynamically at runtime based on user-defined configurations (e.g. download-on-demand language models or custom voice aliases), or must all voice shortcuts and translations be statically declared in the application's compiled String Catalogs (.xcstrings)?
Replies
0
Boosts
2
Views
183
Activity
2w
Voice-Streaming & Text-to-Speech (TTS) Latency
How does the App Intents framework support streaming spoken voice (TTS) output for long-form text responses? Is there an API (such as an asynchronous sequence or buffer stream) that allows Siri to begin speaking a response while the underlying AI engine is still generating the remainder of the text?
Replies
0
Boosts
0
Views
84
Activity
2w
Sandbox-Bypassing IPC between App Intents and Launchd Daemons on macOS
We are designing a macOS utility that runs a local background agent via launchd (managing a local SQLite database and Unix socket). We want to expose controls (start/stop, status checks, CLI command invocation) to Siri via the App Intents framework. Since App Intents typically execute within a sandboxed App Extension or a sandboxed App wrapper container: What is the recommended IPC mechanism (e.g., Unix domain sockets, local HTTP/TCP ports, XPC) to securely communicate between a sandboxed App Intent extension and a non-sandboxed launchd helper daemon on macOS? Are there specific Entitlements (com.apple.security.temporary-exception.files.absolute-path.read-write or network exceptions) required to allow App Intents to talk to local UNIX sockets or loopback interfaces (127.0.0.1) without triggering sandbox violations? Can an App Intent directly invoke a helper command-line tool or launch a plist-configured service without bringing up the main application UI?
Replies
2
Boosts
0
Views
144
Activity
2w
Shortcut-specific domains?
Under the App Schemas list, some of the entries are listed under "Shortcut-specific domains" (like Books and Journaling). What's the difference? Will those domains not be used with Siri AI?
Replies
1
Boosts
0
Views
95
Activity
2w
Deployment & Entitlements
Does the Foundation Models framework support notarized, non-App Store apps on macOS, and are there specific entitlements required to access on-device system models in that environment?
Replies
1
Boosts
1
Views
54
Activity
2w
Meal nutrition data/Siri AI and iOS27 vision framework
I'm an app developer with an interest in nutrition data. While watching the Keynote segment on Monday, there was a brief section about Siri AI and iOS 27 Vision framework being used to scan a meal with an iPhone camera and display the meal nutritional content. I have two questions: Do the displayed data include GI (glycemic index) and/or GL (glycemic load)? and Are these nutrition data surfaced anywhere, especially in HealthKit, or — if not — will Apple make a public API available so developers can use these data directly in iOS27 apps?
Replies
3
Boosts
0
Views
139
Activity
2w
How can I get Siri to recognize "43rd and Broadway"?
Hi all, This is a simplification of an AppIntent where I need my phrases to recognize three "regions" and a specific set of numbers associated with those regions. I am having trouble defining the AppEntity and associated AppShortcut phrases that works well with Siri, which is having trouble extracting numbers for some reason... Now the example: Imagine my app provides information surrounding Starbucks establishments on three major thoroughfares in NYC, such as: Broadway & 26, 40, 43, 54, 63, 75, 86, 95 Madison & 44, 50, 84, 96 Lexington & 43, 48, 55, 63, 67, 77, 85, 96 I need AppEntities, AppShortcuts, and friends that best work with Siri when launched with phrases such as: "Show 'example' for Madison and 50th in \(.applicationName)", "Show 'example' on 43rd and Broadway in \(.applicationName)", Long story short, Siri is picking up my phrases well, but then asking me to answer whether the value is 44, 50, 84, or 96, instead of just hearing that I said: "Madison and 50th" I've defined streets like this: public let nycStreetDisplayRepresentations: [DisplayRepresentation] = [ DisplayRepresentation(title: "26", synonyms: ["twenty six", "twenty sixth"]), DisplayRepresentation(title: "40", synonyms: ["forty", "fortieth"]), DisplayRepresentation(title: "43", synonyms: ["forty three", "forty third"]), DisplayRepresentation(title: "48", synonyms: ["forty eight", "forty eighth"]), DisplayRepresentation(title: "50", synonyms: ["fifty", "fiftieth"]), DisplayRepresentation(title: "54", synonyms: ["fifty four", "fifty fourth"]), DisplayRepresentation(title: "55", synonyms: ["fifty five", "fifty fifth", "five five"]), DisplayRepresentation(title: "63", synonyms: ["sixty three", "sixty third"]), DisplayRepresentation(title: "67", synonyms: ["sixty seven", "sixty seventh"]), DisplayRepresentation(title: "75", synonyms: ["seventy five", "seventy fifth"]), DisplayRepresentation(title: "77", synonyms: ["seventy seven", "seventy seventh", "seven seven"]), DisplayRepresentation(title: "84", synonyms: ["eighty four", "eighty fourth"]), DisplayRepresentation(title: "85", synonyms: ["eighty five", "eighty fifth"]), DisplayRepresentation(title: "86", synonyms: ["eighty six", "eighty sixth"]), DisplayRepresentation(title: "95", synonyms: ["ninety five", "ninety fifth"]), DisplayRepresentation(title: "96", synonyms: ["ninety six", "ninety sixth"]) ] And an entity like this: public struct MadisonThings: AppEntity, Identifiable { public let id: Int public var displayRepresentation: DisplayRepresentation { nycStreetDisplayRepresentations[id-1] } public static var typeDisplayRepresentation = TypeDisplayRepresentation( name: "Madison" , synonyms: ["Madison Ave"] ) public static var defaultQuery = MadisonTHingsQuery() } An EntityQuery: public struct MadisonThingsQuery: EntityQuery, EntityStringQuery { ...attempts to find synonyms in the phrase } Intent: public struct ShowMadisonThingsIntent: AppIntent { { @Parameter( title: "Street Number", description: "The street number (e.g., 5, 12, 44)" ) var streetNumber: MadisonThings // I have also tried just plain 'int' to no avail } And finally the shortcuts: static var appShortcuts: [AppShortcut] { return [ AppShortcut( intent: ShowMadisonThingsIntent(), phrases: [ "Show example for Madison and \(\.$streetNumber) in \(.applicationName)", "Show example on \(\.$streetNumber) in \(.applicationName)", ... This should be very simple, I think, and I've tried multiple ways of extracting numbers from my speech, but for some reason this is not working well at all. Thanks for your help!
Replies
3
Boosts
0
Views
165
Activity
2w
Can I import images into my app with App Intent Schemas?
I'm trying to create an App Intent that takes in an image and copies it to a destination within my app. Sure, this can be done with shortcuts, but that's lame, I want siri to grab an image off the screen and invoke the action! Without an app schema, it doesn't seem like siri can tell what's possible in the app, so I'm trying to find a schema that fits? I've started trying to do this with the photos.addAssetsToAlbum App Intent schema, but it doesn't seem like Siri is onboard with the concept. It's replied many times that it is unable to add things to my App and that I should try via other means like opening my app and importing directly. Should be I be trying this with a different intent schema? I was thinking about files.open, but documentation says that Schema is not available to Siri.
Replies
3
Boosts
0
Views
124
Activity
2w
Integrating Siri with AppIntent that doesn't fit a pre-defined schema/domain
I have an app that controls speakers. Among others it has an action to change various settings. The intent has three parameters, setting, value (dependent on the setting selected) and speaker. Is it possible for Siri to understand phrases such as 'Turn On Night Mode in Bedroom' or 'Set Bass to 2 in Bedroom'?
Replies
1
Boosts
0
Views
121
Activity
2w