The Nested Transparent Objects Issue

Similar to other game engines, RealityKit has a separate processing pipeline for transparent objects. While this works fine for single transparent objects, issues can arise when multiple transparent objects are grouped together, especially when nested.

For example, Apple’s SwiftSplash demo exhibits this issue: with a transparent fish tank containing transparent water, and fish wearing transparent goggles. When rotating or moving, the water and goggles inside can disappear at certain angles.

Simplified Example

For better demonstration, let’s create a simpler example in Reality Composer Pro. We’ll create three nested transparent spheres of decreasing size: the outermost is white, the middle is red, and the innermost is blue. To make the effect more noticeable, we’ll offset them slightly horizontally to create an eccentric arrangement:

When rotating horizontally, we’ll notice that the inner spheres sometimes disappear and reappear:

This anomaly is caused by the rendering order of transparent objects.

Transparent Object Rendering Order

In the real world, when two transparent objects overlap, their order affects the resulting color. This is why rendering engines need to sort transparent objects during processing.

The formula is:

color = foreground.alpha * foreground.color + (1 - foreground.alpha) * background.color

For performance reasons, RealityKit sorts transparent objects based on their coordinate origin (usually the center point) and renders them back-to-front.

However, to achieve correct coloring, we actually need to render based on actual pixel depth from back to front. In fortunate cases, when viewing from certain angles, the pixel depth order matches the center distance order, resulting in correct rendering.

But when viewing from different angles, the actual pixel depth order may not match the center distance order. We render the white sphere first because its center is furthest, but due to its larger radius, its surface is actually closest.

As a result, the engine blends the white sphere with the real background, expecting the next color to have less depth for normal formula blending. However, the red sphere to be rendered next actually has greater surface depth, making normal formula blending impossible. By default, the engine discards later pixels, making the red sphere invisible.

Is there a way to modify the rendering order or the color blending formula? Yes! That’s what ModelSortingGroup is for.

ModelSortingGroup

ModelSortingGroup can be created and ordered either through code in Xcode or by adding and dragging in Reality Composer Pro.

Using ModelSortingGroup in Reality Composer Pro

Click the ”+” in the bottom left to create a ModelSortingGroup, then select it and drag the objects to be sorted into the panel that appears on the right to complete the ordering.

Now, we can rotate and move the scene, and all three spheres render correctly.

Note: Reality Composer Pro sometimes has a bug where the ModelSortingGroup’s effect persists after deletion, requiring a restart to restore default rendering.

Using ModelSortingGroup in Code

Note that in code, it’s called ModelSortGroup. We can create a Group directly and implement sorting by adding ModelSortGroupComponent to ModelEntities that need sorting. In ModelSortGroupComponent, we can specify which Group it belongs to and its order.

fileprivate func setEntityDrawOrder(_ entity: Entity, _ sortOrder: Int32, _ sortGroup: ModelSortGroup) {
    entity.forEachDescendant(withComponent: ModelComponent.self) { modelEntity, model in
        logger.info("Setting sort order of \(sortOrder) of \(entity.name), child entity: \(modelEntity.name)")
        let component = ModelSortGroupComponent(group: sortGroup, order: sortOrder)
        modelEntity.components.set(component)
    }
}

/// Manually specifies sort ordering for the transparent start piece meshes.
func handleStartPieceTransparency(_ startPiece: Entity) {
    let group = ModelSortGroup()

    // Opaque fish parts.
    if let entity = startPiece.findEntity(named: fishIdleAnimModelName) {
        setEntityDrawOrder(entity, 1, group)
    }
    if let entity = startPiece.findEntity(named: fishRideAnimModelName) {
        setEntityDrawOrder(entity, 2, group)
    }
}

How do we access ModelSortingGroup in code if it’s already in a USDZ file? We can either iterate to find ModelSortGroupComponent directly or iterate through all Entities with ModelComponent to check for ModelSortGroupComponent, as both need to be present to take effect.

extension Entity {
    var descendentsWithModelComponent: [Entity] {
        var descendents = [Entity]()
        for child in children {
            if child.components[ModelComponent.self] != nil {
                descendents.append(child)
            }
            descendents.append(contentsOf: child.descendentsWithModelComponent)
        }
        return descendents
    }
}
// Find ModelEntity with ModelSortGroupComponent
let entityWithSortComponent = balls.descendentsWithModelComponent.first { entity in
    entity.components[ModelSortGroupComponent.self] != nil
}

// Get the Group from ModelSortGroupComponent
if let sortComponent = entityWithSortComponent?.components[ModelSortGroupComponent.self] {
    let group = sortComponent.group

    .....

}

Other Considerations

While ModelSortingGroup solves our nested spheres issue, real-world transparent objects can have complex shapes that may intersect, potentially causing rendering issues. What should we do then?

Also, what’s the purpose of the Depth Pass option in ModelSortingGroup?

We’ll address these questions in Part 2, stay tuned!

Author

XanderXu