WhoBuilt

Stepping Into New Dimension With React Native VisionOS

May 31, 2024· 54 min· 444 views·Watch on YouTube

Speakers

Callstack senior React Native developer Oscar and 3D graphics engineer Mario discuss React Native for visionOS, its new immersive and multi-window capabilities, and the work behind rendering 3D content, with host Jakub on Callstack’s React Native Show.

From flat apps to immersive spaces

Oscar says React Native for visionOS has grown from rendering flat interfaces to supporting fully or partly immersive spaces. Its SwiftUI lifecycle enables visionOS features and lets JavaScript request multiple root views on demand. Separate windows have separate React roots, though they can share data through approaches such as a state manager.

Multi-window support and ecosystem updates

The window manager can open windows with initial props and update React Native or SwiftUI windows. Oscar says tracking which windows remain open is still planned, while app state already accounts for multiple windows. The team also added documentation, and community libraries including React Navigation, Reanimated, and Gesture Handler gained visionOS support. Hermes support was merged, which Oscar says should allow prebuilding Hermes and cut build times in half.

Making 3D rendering work

Mario explains that immersive 3D scenes currently need Swift APIs, which can be exposed to JavaScript. The team adapted Babylon Native and bgfx for visionOS, reworking parts of the swap chain and framebuffer because immersive spaces do not use a conventional window. Mario notes that VR rendering has roughly 11 milliseconds per frame at 90 frames per second.

Where the work can lead

Mario describes adding multi-window and immersive features to existing React Native apps, such as a streaming app with a 3D theater. The hosts also discuss furniture previews and 3D data visualization as possible uses. Oscar invites developers to apply the rendering work to other engines and libraries.

A working app and immersive demo

Oscar demonstrates an existing React Native app running in the Vision Pro simulator, with a familiar list and detail flow. A separate window displays a downloadable 3D model, animated with Reanimated and controlled with Gesture Handler. He also shows a RealityKit immersive scene with adjustable immersion, from 30% to full immersion.

Takeaways

  • A React Native app can reuse familiar navigation, animation, and gesture libraries on visionOS while adding platform-specific layouts and APIs.
  • Treat each window as its own React root; plan how separate roots will receive and share data.
  • For immersive 3D, expect Swift integration work and platform-specific rendering constraints.
  • The visionOS work produced reusable ideas and APIs for other React Native platforms, including iPadOS and brownfield iOS apps.
  • Existing apps with 3D or spatial features may be good candidates for visionOS extensions.

Mentioned in this talk

Search inside this talk

What was claimed 12

Related talks