Skip to content

feat(ios): compile the RealityKit layer on tvOS 26 - #108

Merged
jbingham merged 1 commit into
google:mainfrom
shaaaaawn:feat/tvos-realitykit-sdk
Nov 18, 2026
Merged

jbingham merged 1 commit into
google:mainfrom
shaaaaawn:feat/tvos-realitykit-sdk

Conversation

@shaaaaawn

Copy link
Copy Markdown
Contributor

Summary

Apple TV has no WebKit, so native rendering is the only way to show three-live scenes there. ThreeLiveCompiler already declares tvOS 15 and ships tvOS XCFramework slices, but on tvOS only the platform-neutral code builds (ThreeCompiler, XRLiveRuntimeSession and friends): the RealityKit layer sits behind #if canImport(RealityKit) && !os(tvOS). tvOS 26 ships RealityKit, so this PR enables the RealityKit layer on tvOS 26+ while ThreeCompiler-only apps keep the tvOS 15 floor.

A follow-up PR adds an Apple TV target to the SwiftUI example (Siri Remote orbit and select).

API

  • RealityKitHelper, XRLiveRealityKitSession, XRSpatialRealityKitFeatures and XRSimRealityKitBridge compile on tvOS with Swift 6.2+ and are @available(tvOS 26.0, *). With Xcode < 26 the layer is compiled out, as today.
  • configureStudioEnvironment(of:) and handleTap(at:in:) also cover tvOS's ARView, so a tvOS host selects with the same call as iOS (e.g. at a center reticle). Post-processing stays iOS-only.
  • No trait, schema, compiler or C ABI change, so the Web / Android / Apple parity contract is unaffected.

Platform differences

No existing caller changes: the two RealityKit APIs tvOS lacks are handled in the #else of the code that needs them.

  • loadEntityWithoutBlocking, the fallback for OSes without the async Entity initializer, uses that initializer on tvOS (never reached there; tvOS 26 always has it).
  • makeTexture(rgba:size:semantic:), now shared by the splat and contact-shadow textures, uses TextureResource(image:options:) on tvOS instead of generate(from:options:).
  • applyOutline asks a small cullFrontFaces helper whether front-face culling is available, which keeps its hover fallback without an unreachable-code warning on tvOS.

Docs and security

Testing

a773f55 plus this commit, Xcode 27.2, Swift 6.4.2 (rebased unchanged onto 1517c3f; the two commits since touch no Swift):

  • swift test (macOS) and xcodebuild test -scheme ThreeLiveCompiler-Package on iOS 26.5, tvOS 26.5, tvOS 27.0 and Apple Vision Pro (visionOS 27.2) simulators: 120 tests pass on each (118 in ThreeLiveCompilerTests, 2 in ThreeLiveKernelsTests), none skipped. New coverage on tvOS includes:
    • the RealityKit tests, e.g. testWorkbenchAndBallpitLoadInRealityKit and the outline test from fix(ios): render outline traits in XRLiveRealityKitSession #102, whose guard now names tvOS 26
    • new: testOutlineAddsFrontFaceCulledShellAndRemovesIt, and testMakeTextureKeepsRawBytesAndColorAlpha (raw textures read back through Metal byte for byte; color textures keep alpha)
  • No new compiler warnings on any platform; swift-format lint reports nothing new.
  • CI (Xcode 26.6): the existing Compile for tvOS step now builds the five RealityKit files for tvOS, with no new warnings, so the Xcode 26 floor is covered too.
  • handleTap(at:in:) on tvOS 27.0: the follow-up's Siri Remote UI test selects LatticeEnergyCore through it.
  • Texture parity: a throwaway A/B test (not committed) found makeTexture byte-identical to main's code on iOS, macOS and visionOS, and tvOS's TextureResource(image:) gives the same bytes.
  • On screen, the Robotic Arm IK preset on iOS 26.5 and tvOS 27.0 simulators renders the same: outline rim and contact shadow (the tvOS TextureResource(image:) path). The tvOS half is from that follow-up's Apple TV app.

Robotic Arm IK on iOS 26.5 and tvOS 27.0 simulators: the same arm and green orb, each with a bright green outline rim and a soft contact shadow

The package declares tvOS 15 and ships tvOS XCFramework slices, but on
tvOS only the platform-neutral code (ThreeCompiler, XRLiveRuntimeSession
and friends) is built: the RealityKit layer is behind
`canImport(RealityKit) && !os(tvOS)`. tvOS 26 (Xcode 26) ships
RealityKit, so compile the existing layer there, with tvOS as one more
platform rather than a special path:

- RealityKitHelper, XRLiveRealityKitSession, XRSpatialRealityKitFeatures
  and XRSimRealityKitBridge build on tvOS with Swift 6.2+ and are
  available from tvOS 26.0. The package keeps its tvOS 15 floor; with
  older Xcode the layer is compiled out as before.
- tvOS lacks two legacy APIs. No existing caller changes; the tvOS
  variant sits in the `#else` of the code that needs it:
  loadEntityWithoutBlocking (never reached on tvOS, which always has the
  async Entity initializer) and makeTexture, now shared by the splat and
  contact-shadow textures (TextureResource(image:options:) instead of
  generate(from:options:)).
- applyOutline asks a small cullFrontFaces helper whether front-face
  culling is available, so tvOS builds without an unreachable-code
  warning.
- configureStudioEnvironment(of:) and handleTap(at:in:) also cover
  tvOS's ARView, so a tvOS host selects with the same call as iOS (for
  example at a center reticle). Post-processing stays iOS-only.

The existing RealityKit tests now also run on tvOS (skipping below tvOS
26), including the session outline test, plus new tests for the outline
shell and for makeTexture (raw textures read back byte for byte, color
textures keep alpha). 120 tests pass on the macOS host and on iOS 26.5,
tvOS 26.5, tvOS 27.0 and visionOS 27.2 simulators.
@jbingham

Copy link
Copy Markdown
Collaborator

Thanks for putting together such a clean PR and thorough simulator/Metal read-back test coverage, @shaaaaawn!

Keeping the .tvOS(.v15) package floor for ThreeCompiler / ThreeLiveKernels while gating the RealityKit layer on (!os(tvOS) || compiler(>=6.2)) + @available(..., tvOS 26.0, *) is the right split, and consolidating the .color / .raw texture paths in makeTexture(rgba:size:semantic:) (plus cullFrontFaces for warning-free tvOS 26 compilation) is very clean.

I've kicked off the GitHub Actions CI run—once the checks finish green, we'll get this merged. Looking forward to the Apple TV SwiftUI example follow-up!

@jbingham
jbingham merged commit 9504393 into google:main Nov 18, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants