Exploring Foundation Models Framework
Overview
"This book has allowed me to be what I saw myself as being with Foundation Models. I feel I can legitimately use them expressively now. I needed this.
~Adrian Eves
This book teaches iOS, macOS, and visionOS developers how to integrate Apple's Foundation Models directly into their apps. Build features like streaming interfaces, structured data extraction, tool calling, and more.
All running on-device with Apple's privacy guarantees.
What you get:
- 169 pages PDF
- EPUB for Kindle
- LLM text for AI (~52,000 tokens)
- Based on the most popular open-source example repository with 36+ playground examples
Foundation Models with Apple Intelligence
1. Introduction to Foundation Models
Apple's framework for accessing on-device Apple Intelligence models, including guided generation, tool calling, and stateful sessions for iOS 26.0+
- The Model
- Guided Generation
- Streaming with Snapshots
- Tool Calling
- Stateful Sessions and Multi-Turn Conversations
- Foundation Models vs MLX Swift
- Developer Experience
- Limitations and Considerations
2. Getting Started with Sessions
Setting up your first AI session and understanding model availability, error handling, and different generation configuration options
- Project Setup
- Checking Availability
- Exploring with Playgrounds
- Your First Session
- Physiqa Example: Workout Assistant Session
- Instructions vs Prompts
- Prompt Engineering Best Practices
- Understanding the Model's Capabilities
- Basic Error Handling
- Session Safety
- Performance Optimization
3. Streaming and Snapshots
Building responsive UIs with partial results and understanding how Foundation Models streams complete object snapshots instead of token deltas
- Why Snapshots Instead of Token Deltas?
- Streaming Plain Text
- Streaming Structured Results
- Cancellation and Backpressure
- Case Study: Pokémon Snapshot Streaming
- When to Stream vs Generate Once
4. Generation Options and Sampling Control
Shaping model behavior with temperature, penalties, and other parameters to control output quality and creativity
- Understanding Generation Options
- Sampling Modes
- Foundation Models vs MLX Swift Parameters
- Parameter Tuning Tips
5. Structured Generation with Schemas
Creating type-safe, structured output directly from AI responses using the @Generable framework
- Traditional AI Responses
- Understanding the @Generable Macro
- Understanding the @Guide Macro
- Nested Structures
- Optional Fields and Collections
- Advanced @Guide Features
- Journaling and Structured Analysis
- Nutrition Data Parsing
- Streaming with Structured Generation
6. Basic Tool Use
Enabling your AI to perform actions and access real-world data with practical examples of working with web search APIs
- How Tools Work
- Understanding the Tool Protocol
- API Integration: Search Tool
- Using Tools in Your App
- Tool Building Best Practices
- Health Data Integration with Zenther
7. Advanced Chat Patterns
Building production-ready conversation interfaces with context management, memory handling, and graceful error recovery
- Working with Conversation Memory
- Streaming Responses
- Sliding Window Context Management
- Learning from Users
8. Safety and Best Practices
Implementing responsible AI features with proper guardrails, user protection, and Apple's safety principles for trustworthy experiences
- Apple's Safety Philosophy
- Understanding Model Limitations
- Built-in Safety Layers
- Working with Guardrails in Depth
- Safety in Instructions
- Input Patterns with Risk Management
- Using Guided Generation for Safety
- Domain-Specific Safety Considerations
- Building Trust
9. Integrating External JSON APIs
Reusing your Foundation Models schemas with external providers like OpenAI, Anthropic, and Google through JSON Schema compatibility
- Architecture and Streaming
- Defining the @Generable Structure
- Including the JSON Schema
- Making a Request to OpenAI with JSON Schema
- Using AIProxy or Other Packages
- Nutrition JSON to Foundation Models
11. Supported Languages and Internationalization
Working with Foundation Models across 14 supported locales, handling multilingual conversations, and session management strategies for global apps
- Building Multilingual Experiences
- Querying Available Languages
- Creating Language Selection
- Language Support Matrix
- Generating Responses in Multiple Languages
- Handling Multilingual Scenarios
- Physiqa Example: Production Language Detection and Localization
12. Training Custom Adapters
Specializing Foundation Models for your app's domain and writing style using Apple's adapter training toolkit with cost-effective cloud GPU workflows
- Understanding Foundation Models Adapters
- When to Consider Custom Adapters
- Training on Resource-Constrained Hardware
- Scaling Up with the M5 MacBook Pro
- Setting Up the Training Environment
- Training Your First Adapter
- Testing Your Adapter: Before and After
- Using Adapter Studio CLI Wrapper for Easier Training
- Exporting the Adapter
- Entitlements and Device Support
- Adapter Studio: Side-by-Side Evaluation on macOS
- Loading Adapters in Your App
- Compile the Draft Model (optional)
- Locale and Language Support
- Integration Checklist
Upcoming chapters:
13. Foundation Models Beyond the Main App
Using Foundation Models in app extensions, widgets, shortcuts, and system integrations while maintaining session-based patterns
14. Advanced Tool Patterns
Production integrations with external APIs and services, including retry strategies, fallback handling, and security considerations
15. Dynamic Generation Schemas
Runtime schema construction and complex data modeling for advanced structured generation patterns
16. Performance Optimization and Profiling
Using Foundation Models instruments in Xcode for production deployment, including memory management, token optimization, and performance monitoring
17. Voice-First AI Experiences
Building conversational interfaces and voice-driven interactions with Foundation Models
Use the code "STUDENT" for a discount if you are a student.
© Copyright 2025 — Rudrank Riyam's Academy