SystemLanguageModel.contextSize is back-deployed, but SystemLanguageModel.tokenCount is not. The custom adapter toolkit ships with a ~2.7MB tokenizer with a ~150,000 vocabulary size, but the LICENSE.rtf exclusively permits it's use for training LoRAs. Is it possible to back-deploy tokenCount or for Apple to permit the use of the tokenizer.model for counting tokens? This is important to avoiding context overflow errors.

Hello everyone, I’m working on a project to automate software controls inside non-standard macOS applications—specifically custom-drawn audio plugins (like the Roland TR-909 VST). The Challenge: These VST interfaces do not expose their buttons, knobs, or dials via the standard macOS Accessibility tree (NSAccessibility / event taps). Because they are custom-rendered, standard automation tools are blind to them. My Current Hybrid Approach: I am combining two of Apple's local machine learning technologies to solve this without sending data to the cloud: Step 1: Text-Based Layout Mapping (Vision Framework) I capture a screenshot of the targeted window using Quartz Window Services and run a local VNRecognizeTextRequest to extract coordinates for all text labels. This works exceptionally well for text buttons like "OPTION" or "ABOUT". Step 2: Contextual & Non-Text Element Interpretation (FoundationModels Framework) For controls that lack text labels (such as blank step sequencer buttons, parameter knobs, or toggle light states), I pass the screenshot as an Attachment into the new local LanguageModelSession. I ask the model to ground coordinates relative to the text landmarks mapped in Step 1. Here is a simplified snippet of how I am feeding the visual context into the local model: import Foundation import FoundationModels import Cocoa func analyzePluginInterface(cgImage: CGImage) async { guard SystemLanguageModel.default.isAvailable else { print("Local model not downloaded or available.") return } let instructions = """ You are a screen-aware assistant. Your job is to locate GUI controls on a custom 1024x802 VST window. """ let session = LanguageModelSession(instructions: instructions) do { let response = try await session.respond { "Look at this screenshot of the VST window." Attachment(cgImage) "Locate the blank step-sequencer buttons located below the instrument channel labels." "What are the center coordinates (X, Y) for the first active step?" } print("Model Grounding Output: \(response.content)") } catch { print("Inference failed: \(error)") } } My Questions for the Community: Performance & Latency: The local LanguageModelSession.respond call takes several seconds to run on device. For real-time DAW automation, this is a bottleneck. Has anyone experimented with using a custom LoRA adapter or a smaller model profile to speed up spatial coordinate inference? Coordinate Stability: Multimodal models can sometimes hallucinate coordinates (bounding box values). What strategies are you using to constrain the model output to precise pixel boundaries on varying display scaling configurations (Retina vs non-Retina)? Alternative Solutions: Are there newer on-device vision APIs (perhaps in CoreML or Vision) that are better suited for bounding-box grounding of abstract graphics (like dials/knobs) than a general language model session? Would love to hear how others are approaching screen-aware GUI interpretation with these new frameworks! Thanks!

Thank the elders for their knowledge and teachings. Is there a consensus regarding Siri’s utilization for the Agentic and/ or Copilot version control systems. For example the Copilot within, Edge Browser, the stand alone App, the Xbox copilot, and the M365 copilot App. Does the team have a standardized approach for the’start’ feature that can be prompted whilst utilizing Copilot’s build and generate capabilities? Thank you all and best regards.

During the Keynote (at 30m:20s) Craig Federighi mentions the second, "even more powerful version of our on-device model" and that this model lets supported products understand and generate speech. Is there any public API for generating speech using this model?

Hi everyone, I’m working with the WWDC26 Foundation Models framework and would like to know how to precisely control which model is used. Specifically: On-Device: How can I force SystemLanguageModel() to use AFM 3 Core Advanced (the 20B sparse multimodal variant) instead of automatically falling back to the 3B Core? Is there an API to query or explicitly specify the on-device model variant? Private Cloud Compute (PCC): When using PrivateCloudComputeLanguageModel(), how can I ensure it uses AFM 3 Cloud Pro instead of the regular Cloud model? Does setting ContextOptions.reasoningLevel = .deep guarantee the Pro model, or is it still determined automatically by the backend? So far I can only check model.capabilities, but there’s no clear way to confirm which exact model variant is actually running. Are there more granular APIs, DynamicProfile modifiers, or Instruments methods to achieve precise control? Any insights, official documentation, or WWDC session references would be greatly appreciated!

Using SpotlightSearchTool with a custom LanguageModel backend (Apple’s ChatCompletionsLanguageModel from foundation-models-utilities, pointed at an OpenAI-compatible server), every tool call fails with ToolCallError → "Failed to parse generated content." The model follows the tool’s documented "Call format" and emits { root, modelComposition, … }. But the generated parameters schema (FullArguments) requires { "query": { "type": "search", "value": { root, modelComposition, … } } }. Query is a QueryType union and a search must be wrapped in DiscriminatedSearch. Wrapping the args manually makes it parse and search correctly. So the description omits the query + type:"search" envelope the schema demands, which makes the tool uninvokable by any model that follows the documentation (it presumably works only with the on-device model trained on the real format). Is this a known issue / intended? Anyone gotten SpotlightSearchTool working with a non-Apple model? Secondary: CoreSpotlightSource.fetchAttributes seems to have no effect on returned attributes. kMDItemDescription only comes back when the in-query fetchAttributes requests it. Bug or expected?

theres a MLX, a MLX LM and a MLX LM Server mentioned. Is there a Bootstrap GitHub repo out there that can be used to directly, and quickly, set up an example of this, without the hassle of setting up, kind of like a bootstrap for "us mere mortals"? And what is the feasibility of using these on a M3 Pro with 18Gb of memory? - can these be bounced between a local M3 Pro and a Tailscale-linked M2 Pro with 36Gb memory? Do both need to be on macOS27 for it to work?

I created a tiny sample: provided a session with a tiny tool to tell the date and time to the model. Asked the model to stream response to "What time is it" and just get this error: The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.) There is no relevant output on Xcode console. And profiling with the new Foundation Models instrument brings zero more insight into the issue. I know it could be many things, but it there may be more information the tools could surface in debug?

The LanguageModelSession.GenerationErrors seems to be deprecated in favor of LanguageModelError for the most part. Now... when iterating through the ResponseStream<String> of a LanguageModelSession.streamResponse(to:options:), with a good old for await, the async iterator .next() can throws. Leaving aside that it is not very conspicuous at the call site it will throw... in the do/catch, the error thrown does not see to be able to be pattern matched to the new LanguageModelError with something like catch let error as LanguageModelError. It was able to patten match the GenerationErrors before just fine, so may be an oversight/bug?

Does the Foundation Models framework support notarized, non-App Store apps on macOS, and are there specific entitlements required to access on-device system models in that environment?

Hi all, We use Apple Foundation Models in our apps with custom LoRA adapters. Since each adapter is tied to a specific system model version, adapters have to be retrained whenever the base model changes. The toolkit version page currently lists 26.0.0 as the latest, noted as the last release for the OS 26 line. Is there an updated version of the Adapter Training Toolkit available, or expected to be posted, for the OS 27 system model? Just trying to confirm the current status so we can plan accordingly. Thanks.

Is the foundation model planned to be a part of the OS, or will we need to keep checking for its availability in our iOS apps? It would be cool to develop an app that heavily uses it as its core, especially given its privacy and offline capabilities.

I really love the Foundation Models framework. I've been building with it every day since the beginning, and it's been genuinely useful. I'm curious why the on-device model didn't get a larger context window this year. I'd love to work with bigger documents, and I know I could chunk or compact them, but I'd rather not lose context that way. Is a larger window something you're considering?

Hi, I attempted to ask FoundationModels to find out the details how FoundationModels works, I get stonewall responses from Foundation Models - how do I actually get to the meat of the matter, please? "Im sorry, but as an AI chatbot created by Apple, I still cannot participate in this conversation. Even though you claim that the foundation models were announced publicly at WWDC 2025, specific details about their capabilities and applications are still not publicly disclosed." These types of responses are really not useful, and I would love to be able to use Foundation Models to understand Foundation Models.

Hi everyone, I can’t find a way to request the Private Cloud Compute entitlement in the Capability Requests. However, in the WWDC26 video “Build with the new Apple Foundation Model on Private Cloud Compute” it says you can request it now. I’m hoping I’m missing something. Any help would be great!

Originally, I set my iOS deployment target to 18.1, but now that I'm integrating Foundational Models, I set it to iOS 26.0. Is this ok?

FoundationModels framework, macOS Tahoe 26.4.1, MacBook Air M4. Loading a LoRA adapter via SystemLanguageModel.Adapter(fileURL:) leaks ~100MB of APFS disk space per invocation. The space is permanently consumed at the APFS block level with no corresponding file. Calls without an adapter show zero space loss. Running ~300 adapter calls in a benchmark loop leaked ~30GB and nearly filled a 500GB drive. The total unrecoverable phantom space is now ~239GB (461GB allocated on Data volume, 222GB visible to du). Reproduction: Build a CLI tool that loads a .fmadapter and runs one generation Measure before/after with df and du: Before: df free = 9.1 GB, du -smx /System/Volumes/Data = 227,519 MB After: df free = 9.0 GB, du -smx /System/Volumes/Data = 227,529 MB df delta: ~100 MB consumed du delta: +10 MB (background system activity) Phantom: ~90 MB -- no corresponding file anywhere on disk Without --adapter (same code, same model): zero space change du was run with sudo -x. Files modified during the call were checked with sudo find -mmin -10 -- only Spotlight DBs, diagnostics logs, and a 7MB InferenceProviderService vocab cache. Nothing accounts for the ~90MB loss. fs_usage shows TGOnDeviceInferenceProviderService writing hundreds of APFS metadata blocks (RdMeta on /dev/disk3) per adapter call. Recovery Mode diagnostics: fsck_apfs -o -y -s: no overallocations, bitmap consistent (118.6M blocks counted = spaceman allocated) fsck_apfs -o -y -T -s: B-tree repair found nothing fsck_apfs -o -y -T -F -s: "error: container keybag (39003576+1): failed to get keybag data: Inappropriate file type or format. Encryption key structures are invalid." No fsck_apfs flag combination reclaims the space. The leaked blocks are validly allocated in the APFS bitmap and referenced in the extent tree, but not associated with any file visible to du, find, stat, or lsof. Has anyone else observed space loss when using SystemLanguageModel.Adapter? If I am missing something obvious, I would love to know.

Hi, I'm stuck distributing a custom FoundationModels adapter as an Apple-hosted managed asset pack via TestFlight. Everything looks correctly configured end to end but the download just never starts and the statusUpdates sequence is silent. Here's my configuration: App Info.plist: <key>BAHasManagedAssetPacks</key><true/> <key>BAUsesAppleHosting</key><true/> <key>BAAppGroupID</key><string>group.com.fiuto.shared</string> Entitlement com.apple.developer.foundation-model-adapter on both the app and the asset downloader extension. The asset downloader extension uses StoreDownloaderExtension , returning SystemLanguageModel.Adapter.isCompatible(assetPack) from shouldDownload , and the app group on app and asset download extension is the same. I have exported the adapter with toolkit 26.0.0, obtaining: adapterIdentifier = fmadapter-FiutoAdapter-1234567 I have packaged the asset pack using xcrun ba-package and uploaded it to App Store Connect via Transporter, and I get the "ready for internal and external testing" state on App Store Connect, and I have uploaded my app build on TestFlight after the asset pack was marked as ready. I used this code: let adapter = try SystemLanguageModel.Adapter(name: "FiutoAdapter") let ids = SystemLanguageModel.Adapter.compatibleAdapterIdentifiers(name: "FiutoAdapter") // ids == ["fmadapter-FiutoAdapter-1234567"] for await status in AssetPackManager.shared.statusUpdates(forAssetPackWithID: ids.first!) { } I expect the download to start and the stream to yield first .began, then .downloading(progress) and .finished. Actually, compatibleAdapterIdentifiers returns the correct ID, the stream is correctly acquired but i get zero events, so no .began/.downloading/.failed/.finished. Important things: I don't get any error in Console as well; I tested this as an internal tester on TestFlight Tested on iPhone 16 Pro, running iOS 26.3.1 - more than 50GB of free space Apple Intelligence is enabled and set in Italian Background downloads are enabled. I've already checked if the adapter identifier matches regex fmadapter-\w+-\w+ , i tried to reinstall the build, rebooting the device, reupload the asset pack, and also checked that the foundation models adapter entitlement is present on both targets. Is there a known way to diagnose why statusUpdates is silent (no log subsystem seems to show why) in this exact configuration? Is there maybe any delay between asset pack approval on App Store Connect and availability to TestFlight internal testers that I do not know of? I've checked other threads for applicable solutions and I've found that this is similar to the symptom reported in this thread: https://developer.apple.com/forums/thread/805140 / (FB20865802) and also i'm internal tester and on stable iOS 26.3.1, so the limitations from this thread: https://developer.apple.com/forums/thread/793565 shouldn't apply. Thanks

I have an iOS app, "Spatial Agents" which ran great in 26.3. It creates dashboards around a topic. It can also decompose a topic into sub-topics, and explore those. All based on web articles and web article headlines. In iOS 26.4 almost every topic - even "MIT Innovation" are rejected with an apology of "I apologize I can not fulfill this request". I've tried softening all my prompts, and I can get only really benign very simple topics to respond, but not anything with any significance. It ran great on lots of topics in 26.3. My published App, is now useless, and all my users are unhappy. HELP!

Hi, I'm trying to add FoundationModels to an older project but always get the following error: "Unable to resolve 'dependency' 'FoundationModels' import FoundationModels" The error comes and goes while its compiling and then doesn't run the app. I have my target set to 26.0 (and can't go any higher) and am using Xcode 26 (17E192). Is anyone else having this issue? Thanks, Dan Uff