The presenter compares a cloud API with on-device inference in a health app during a subway commute. With a local model, a response can stream without waiting for a network connection.
What Is the React Native AI SDK? A Complete Intro & Quickstart
An unnamed Callstack presenter introduces React Native AI and its approach to running language models on mobile devices.
The presenter says local inference keeps user input on the device instead of sending it to a third-party server. They argue this can reduce privacy and compliance concerns, though the transcript does not detail specific legal requirements.
Local models can keep working when a user loses signal or a cloud service has an outage. The presenter says that after a model is downloaded, inference does not incur per-token API charges.
Callstack’s library supports built-in Apple Foundation Models through its Apple package on iOS 26, and MLC LLM through a package for Android and iOS. The presenter says the MLC engine can run custom models.
React Native AI provides a Vercel AI SDK provider for both Apple and MLC packages. The presenter says this lets developers set up and run on-device models with a small amount of code, with setup details planned for later episodes.
Takeaways
- On-device inference can keep app features available when network access is unreliable.
- Keeping sensitive inputs on the device can reduce data sent to third parties, but does not by itself establish regulatory compliance.
- Budget for model downloads and device support; local inference avoids per-token API charges after setup.
- Choose an inference engine based on platform and model needs: Apple Foundation Models or MLC LLM.
- A Vercel AI SDK provider can give JavaScript developers a consistent interface across the supported engines.
Mentioned in this talk
Search inside this talk
What was claimed 6
- uses
“On the JavaScript side of React Native AI, we're using the Vercel AI SDK.”
0:18inThe Hard Way vs. The React Native AI SDK Way: Stop Writing Custom Modules for Every ModelYouTube Verified2 sources· Talk
- integrates with
React Native AI integrates with MLC LLM“We integrate native modules for Apple LLM and MLC LLM.”
0:34inThe Hard Way vs. The React Native AI SDK Way: Stop Writing Custom Modules for Every ModelYouTube Verified2 sources· Talk
- integrates withReact Native AI integrates with Apple LLM
“We integrate native modules for Apple LLM and MLC LLM.”
0:34inThe Hard Way vs. The React Native AI SDK Way: Stop Writing Custom Modules for Every ModelYouTube Verified2 sources· Talk
- featureReact Native AI: Configured for local inference, input stays on the device.
“that input never leaves the device”
YouTube Unverified1 source· Talk
- featureReact Native AI: Supports built-in OS models and third-party models.
“support for both built-in models shipped with the OS and third-party models”
YouTube Unverified1 source· Talk
- builtCallstack built React Native AI
“This is why we built React Native AI at Callstack”
YouTube Verified2 sources· Talk
