How JSI powers the most advanced Camera library (VisionCamera Frame Processing)

Rousavy introduces VisionCamera and demonstrates a camera app with frame processors. These functions run on camera frames, and native plugins can perform tasks such as image labeling, object detection, and QR code scanning. His example uses ML Kit to label frames and return the results to JavaScript.

Rousavy describes the React Native bridge as a batched communication layer that serializes messages, which adds delay and overhead. JSI provides a C++ interface to JavaScript runtimes, letting native code create values and functions that JavaScript can call synchronously. He demonstrates a native `add` function and explains that JSI values can be installed in the runtime’s global namespace.

Using React Native MMKV as an example, Rousavy shows how a module can install JSI functions before the JavaScript bundle runs, using the JavaScript thread and a runtime pointer passed into C++. He then explains VisionCamera’s frame object: a C++ host object exposes properties such as width and height by retrieving them from the native camera frame when JavaScript accesses them. This avoids copying or serializing the full image into JavaScript.

Rousavy explains that sending a camera frame through the bridge would require a large JSON message for every frame, and calling a native plugin would send data back across the bridge again. VisionCamera instead passes a host object to a frame processor running in a secondary JavaScript runtime created with Worklets. Plugins can be written in Swift or Objective-C on iOS and Java or Kotlin on Android, with JSI values converted to platform types.