RealityKit – Loading Reality Composer scenes with SwiftUI - swiftui

Question

I'm trying to load different models on face using SwiftUI, RealityKit and ARKit.
struct AugmentedRealityView: UIViewRepresentable {
#Binding var modelName: String
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
let configuration = ARFaceTrackingConfiguration()
arView.session.run(configuration, options: [.removeExistingAnchors,
.resetTracking])
loadModel(name: modelName, arView: arView)
return arView
}
func updateUIView(_ uiView: ARView, context: Context) { }
private func loadModel(name: String, arView: ARView) {
var cancellable: AnyCancellable? = nil
cancellable = ModelEntity.loadAsync(named: name).sink(
receiveCompletion: { loadCompletion in
if case let .failure(error) = loadCompletion {
print("Unable to load model: \(error.localizedDescription)")
}
cancellable?.cancel()
},
receiveValue: { model in
let faceAnchor = AnchorEntity(.face)
arView.scene.addAnchor(faceAnchor)
faceAnchor.addChild(model)
model.scale = [1, 1, 1]
})
}
}
This is how I load them but when the camera view opens and loads one model then the other models won't be loaded. Can someone help me out?

Answer 1

When the value of your Binding changes, SwiftUI is calling your updateUIView(_:,context:) implementation, which does noting.
Additionally, you are not storing the AnyCancellable. When the token returned by sink gets deallocated the request will be cancelled. That could result in unexpected failures when trying to load lager models.
To fix both of these issue, use a Coordinator.
import UIKit
import RealityKit
import SwiftUI
import Combine
import ARKit
struct AugmentedRealityView: UIViewRepresentable {
class Coordinator {
private var token: AnyCancellable?
private var currentModelName: String?
fileprivate func loadModel(_ name: String, into arView: ARView) {
// Only load model if the name is different from the previous one
guard name != currentModelName else {
return
}
currentModelName = name
// This is optional
// When the token gets overwritten
// the request gets cancelled
// automatically
token?.cancel()
token = ModelEntity.loadAsync(named: name).sink(
receiveCompletion: { loadCompletion in
if case let .failure(error) = loadCompletion {
print("Unable to load model: \(error.localizedDescription)")
}
},
receiveValue: { model in
let faceAnchor = AnchorEntity(.camera)
arView.scene.addAnchor(faceAnchor)
faceAnchor.addChild(model)
model.scale = [1, 1, 1]
})
}
fileprivate func cancelRequest() {
token?.cancel()
}
}
#Binding var modelName: String
func makeCoordinator() -> Coordinator {
Coordinator()
}
static func dismantleUIView(_ uiView: ARView, coordinator: Coordinator) {
coordinator.cancelRequest()
}
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
let configuration = ARFaceTrackingConfiguration()
arView.session.run(configuration, options: [.removeExistingAnchors,
.resetTracking])
context.coordinator.loadModel(modelName, into: arView)
return arView
}
func updateUIView(_ uiView: ARView, context: Context) {
context.coordinator.loadModel(modelName, into: uiView)
}
}
We create a nested Coordinator class that holds the AnyCancellable token and move the loadModel function into the Coordinator.
Other than a SwiftUI View, the Coordinator is a class that lives while your view is visible (always a remember that SwiftUI might create and destroy your View at will, its lifecycle is not related to the actual "view" that is shown on screen).
In out loadModel class we double check that the value of our Binding actually changed so that we don't cancel an ongoing request for the same model when SwiftUI updates our View, e.g. because of a change in the environment.
Then we implement the makeCoordinator function to construct one of our Coordinator objects.
Both in makeUIView and in updateUIView we call the loadModel function on our Coordinator.
The dimantleUIView method is optional. When the Coordinator gets deconstructed our token gets released as well, which will trigger Combine into canceling ongoing requests.