Rybczak describes how Vercel’s AI SDK offers a shared interface for providers such as Anthropic, Gemini, and OpenAI. He set out to connect that interface to React Native and chose MLC LLM as a cross-platform engine. A demo ran a downloaded model on an iPhone 15 Pro in airplane mode, though users first had to download several gigabytes.
How to Run Any LLM On-Device With React Native
Speaker
Szymon Rybczak, senior React Native developer at Callstack, explains React Native AI and on-device language models at React Advanced London 2025.
Rybczak says he wanted models that users would not need to download. After Apple announced its Foundation Models framework at WWDC 2025, he connected it to React Native AI that same day. The model ran offline on an iPhone 15 Pro and was available through the library’s provider and generation functions.
The library supports text generation and streaming, tool calling, structured outputs, embeddings, transcription, speech-to-text, and text-to-speech. Tool calls can connect a model to native features, such as reading a calendar or creating an event offline. Rybczak says developers can also define system prompts and use device data through tools when users grant the required permissions.
Rybczak points to privacy, fast responses, and offline access as reasons to run models on-device, including summarizing private medical PDFs. He says cloud models remain more capable, so developers should choose based on the task. In his account, downloaded models could use around 4 GB of RAM, while Apple’s built-in model used about 200 to 300 MB.
In the Q&A, Rybczak says Pixel 9 Pro devices shipped with built-in Gemini models and that the team was working on support. Apple’s models can run in simulators with macOS 26 and Apple Intelligence enabled; Android requires a physical Pixel Pro device. He says MLC is the current universal engine, while MLX was under consideration, and the library can check whether a device supports a model.
Takeaways
- Use on-device models when privacy, offline availability, or low-latency responses matter more than the strongest model capability.
- A downloaded model can work offline after installation, but its multi-gigabyte size and memory use may limit adoption.
- Built-in operating system models avoid a separate model download and, in Rybczak’s figures, use substantially less RAM.
- Connect model tool calls only to device features the app needs, and request the relevant user permissions.
- Check model support on each device and provide fallback behavior for hardware that cannot run it.
Mentioned in this talk
Search inside this talk
What was claimed 13
- role atSzymon Rybczak is Senior React Native developer at
Callstack“I'm a senior React Native developer at Callstack.”
0:00inIntelligent Fallbacks: Using Apple Intelligence Seamlessly With @react-native-ai/appleYouTube Verified6 sources· Talk
- integrates withReact Native AI integrates with Vercel AI SDK
“With all packages under the React Native AI family, we have full support for the AI SDK”
0:18inBuild Smarter Apps: Tool Calling & AI Orchestration ExplainedYouTube Verified4 sources· Talk
- built
“I've created many React Native libraries; React Native AI is one of them.”
YouTube Verified2 sources· Talk
- usesReact Native AI uses MLC LLM
“I found an engine called MLC LLM.”
YouTube Verified3 sources· Talk
- featureMLC LLM: Runs models across multiple platforms
“You have one model that you can also run on multiple platforms.”
YouTube Unverified1 source· Talk
- featureApple Foundation Models framework: Provides an Apple-created on-device LLM on iOS, macOS, and iPadOS
“A way to run an on-device LLM created by Apple on iOS, macOS, and iPadOS.”
YouTube Unverified1 source· Talk
- integrates withReact Native AI integrates with Apple Foundation Models framework
“I managed to plug Apple’s Foundation Models framework into React Native.”
YouTube Verified2 sources· Talk
- featureReact Native AI: Supports first-class vLLM community provider integration
“We support... first-class vLLM support... we're an official community provider.”
YouTube Unverified1 source· Talk
- featureReact Native AI: Supports structured outputs, streaming text generation, tool calling, embeddings, transcription, TTS, and STT
“We support structured outputs... text generation streaming, tool calling, embeddings, and transcription.”
YouTube Unverified1 source· Talk
- usesGemini models on Google Pixel devices
“Google Pixel 9 Pro actually shipped with them, and every new Google Pixel we ship will ship with Gemini models.”
YouTube Unverified1 source· Talk
- featureReact Native AI: Supports embeddings for compatible engines
“There's support for embeddings, depending on the engine you're using.”
YouTube Unverified1 source· Talk
- featureApple Foundation Models framework: Supports system prompts, tool calling, and structured outputs
“Apple supports system prompts, tool calling, and structured outputs.”
YouTube Unverified1 source· Talk
- metricApple Foundation Models framework: Around 200–300 megabytes of RAM
“The models shipped with the OS are very performant, using around 200 or 300 megabytes of RAM.”
YouTube Unverified1 source· Talk
