product image

Exploring Foundation Models Framework

Overview

"This book has allowed me to be what I saw myself as being with Foundation Models. I feel I can legitimately use them expressively now. I needed this.
~Adrian Eves

This book teaches iOS, macOS, and visionOS developers how to integrate Apple's Foundation Models directly into their apps. Build features like streaming interfaces, structured data extraction, tool calling, and more.

All running on-device with Apple's privacy guarantees.

What you get:


Foundation Models with Apple Intelligence

1. Introduction to Foundation Models

Apple's framework for accessing on-device Apple Intelligence models, including guided generation, tool calling, and stateful sessions for iOS 26.0+

  • The Model
  • Guided Generation
  • Streaming with Snapshots
  • Tool Calling
  • Stateful Sessions and Multi-Turn Conversations
  • Foundation Models vs MLX Swift
  • Developer Experience
  • Limitations and Considerations

2. Getting Started with Sessions

Setting up your first AI session and understanding model availability, error handling, and different generation configuration options

  • Project Setup
  • Checking Availability
  • Exploring with Playgrounds
  • Your First Session
  • Physiqa Example: Workout Assistant Session
  • Instructions vs Prompts
  • Prompt Engineering Best Practices
  • Understanding the Model's Capabilities
  • Basic Error Handling
  • Session Safety
  • Performance Optimization

3. Streaming and Snapshots

Building responsive UIs with partial results and understanding how Foundation Models streams complete object snapshots instead of token deltas

  • Why Snapshots Instead of Token Deltas?
  • Streaming Plain Text
  • Streaming Structured Results
  • Cancellation and Backpressure
  • Case Study: Pokémon Snapshot Streaming
  • When to Stream vs Generate Once

4. Generation Options and Sampling Control

Shaping model behavior with temperature, penalties, and other parameters to control output quality and creativity

  • Understanding Generation Options
  • Sampling Modes
  • Foundation Models vs MLX Swift Parameters
  • Parameter Tuning Tips

5. Structured Generation with Schemas

Creating type-safe, structured output directly from AI responses using the @Generable framework

  • Traditional AI Responses
  • Understanding the @Generable Macro
  • Understanding the @Guide Macro
  • Nested Structures
  • Optional Fields and Collections
  • Advanced @Guide Features
  • Journaling and Structured Analysis
  • Nutrition Data Parsing
  • Streaming with Structured Generation

6. Basic Tool Use

Enabling your AI to perform actions and access real-world data with practical examples of working with web search APIs

  • How Tools Work
  • Understanding the Tool Protocol
  • API Integration: Search Tool
  • Using Tools in Your App
  • Tool Building Best Practices
  • Health Data Integration with Zenther

7. Advanced Chat Patterns

Building production-ready conversation interfaces with context management, memory handling, and graceful error recovery

  • Working with Conversation Memory
  • Streaming Responses
  • Sliding Window Context Management
  • Learning from Users

8. Safety and Best Practices

Implementing responsible AI features with proper guardrails, user protection, and Apple's safety principles for trustworthy experiences

  • Apple's Safety Philosophy
  • Understanding Model Limitations
  • Built-in Safety Layers
  • Working with Guardrails in Depth
  • Safety in Instructions
  • Input Patterns with Risk Management
  • Using Guided Generation for Safety
  • Domain-Specific Safety Considerations
  • Building Trust

9. Integrating External JSON APIs

Reusing your Foundation Models schemas with external providers like OpenAI, Anthropic, and Google through JSON Schema compatibility

  • Architecture and Streaming
  • Defining the @Generable Structure
  • Including the JSON Schema
  • Making a Request to OpenAI with JSON Schema
  • Using AIProxy or Other Packages
  • Nutrition JSON to Foundation Models

11. Supported Languages and Internationalization

Working with Foundation Models across 14 supported locales, handling multilingual conversations, and session management strategies for global apps

  • Building Multilingual Experiences
  • Querying Available Languages
  • Creating Language Selection
  • Language Support Matrix
  • Generating Responses in Multiple Languages
  • Handling Multilingual Scenarios
  • Physiqa Example: Production Language Detection and Localization

12. Training Custom Adapters

Specializing Foundation Models for your app's domain and writing style using Apple's adapter training toolkit with cost-effective cloud GPU workflows

  • Understanding Foundation Models Adapters
  • When to Consider Custom Adapters
  • Training on Resource-Constrained Hardware
  • Scaling Up with the M5 MacBook Pro
  • Setting Up the Training Environment
  • Training Your First Adapter
  • Testing Your Adapter: Before and After
  • Using Adapter Studio CLI Wrapper for Easier Training
  • Exporting the Adapter
  • Entitlements and Device Support
  • Adapter Studio: Side-by-Side Evaluation on macOS
  • Loading Adapters in Your App
  • Compile the Draft Model (optional)
  • Locale and Language Support
  • Integration Checklist

Upcoming chapters:

13. Foundation Models Beyond the Main App

Using Foundation Models in app extensions, widgets, shortcuts, and system integrations while maintaining session-based patterns

14. Advanced Tool Patterns

Production integrations with external APIs and services, including retry strategies, fallback handling, and security considerations

15. Dynamic Generation Schemas

Runtime schema construction and complex data modeling for advanced structured generation patterns

16. Performance Optimization and Profiling

Using Foundation Models instruments in Xcode for production deployment, including memory management, token optimization, and performance monitoring

17. Voice-First AI Experiences

Building conversational interfaces and voice-driven interactions with Foundation Models


Use the code "STUDENT" for a discount if you are a student.

© Copyright 2025 Rudrank Riyam's Academy

Terms of Service / Privacy Policy