Hi everyone,
I've been working on a problem that I think many of us building AI in Swift are facing: how do you debug why an AI agent behaves differently between runs?
Traditional logging tells you what happened. But when a model skips a step, changes its reasoning order, or produces a different output with identical input — you're left staring at walls of logs with no clear answer to why.
I've released DProvenanceKit — a reasoning observability framework for Swift that lets you:
Record every reasoning step an agent takes (non-blocking, async-safe)
Query for reasoning patterns ("find runs where X happened but Y didn't")
Diff two executions to see structural differences
Detect regressions automatically with rule-based validation
Think of it as Git for AI logic.
Example:
Swift
// Record an execution
try await DProvenanceKit.run(contextID: "case-123", store: store) {
DProvenanceKit.record(.documentEvaluated(documentID: "DocA", score: 0.95))
DProvenanceKit.record(.conflictDetected(reason: "timeline_inconsistency"))
DProvenanceKit.record(.finalDecisionMade(approved: false))
}
// Query for suspicious patterns
let suspiciousRuns = try await store.queryRuns(
TraceQueryDSL()
.requiring(step: "conflictDetected")
.missing(step: "documentEvaluated") // Find runs where conflict was reported but no docs evaluated
)
// Diff two runs
let diff = engine.diff(base: runA, comparison: runB)
print(diff.changes) // See exactly which steps appeared, disappeared, or moved
The design:
Built specifically for on-device AI (macOS/iOS) with Apple Foundation Models, MLX, or Core ML
Non-blocking recording (touches only in-memory buffer)
Durable, crash-safe persistence with SQLite WAL
Works with async/await context propagation
Status:
Experimental (core engine complete, actively evolving). Free for development/testing under BSL 1.1.
GitHub: https://github.com/Therealdk8890/DProvenanceKit
I'm curious if this resonates with anyone here building AI in Swift. What debugging/observability challenges are you facing with AI systems?
