Introduction

In 3D rendering engines, transparent object rendering has always been a challenge. As we discussed in our previous articles “Solving Nested Transparent Objects in RealityKit with Rendering Ordering Part 1 and Part 2”, there’s no perfect solution. Among transparent objects, glass is the most common. In this article, we’ll discuss two approximate simulation methods for glass objects in RealityKit.

Simulating Glass with Transparent Objects

The most obvious approach is using transparent objects to simulate glass, since glass is naturally semi-transparent in reality. Besides adjusting opacity, you can also modify metallicness and roughness, and add an extra Fresnel effect (shown in the red box below) to enhance edge reflections.

Metallicness: Too low metallicness will cause glass to lose all reflections, while too high will make it look too dark. In reality, glass has relatively low metallicness.

Opacity: Too high opacity will make glass look too solid, losing transparency; too low will make it too transparent, losing reflections.

However, glass objects created this way look artificial. Have you noticed? It doesn’t look like a solid glass ball but rather like a hollow bubble, because it lacks refraction effects and directly shows the background color.

Simulating Glass Refraction with Environment Radiance

For better glass simulation, we need to refract the background. However, RealityKit still uses traditional rasterization rather than full ray tracing, making accurate refraction impossible. But we can simulate this effect using Environment Radiance.

Environment Radiance was originally designed for handling reflection effects, reflecting surrounding environment map information (Skybox and IBL), like the reflection sphere below:

However, we can achieve refraction-like effects by controlling reflection normals, making it appear as if we’re seeing through to objects behind:

As shown below, the left side demonstrates its normal operation, reflecting the environment map (Skybox) behind the eye. The right side shows our desired result - we just need to calculate appropriate normals to make the sphere display the environment in front of the eye, add some perturbation, and we get a refraction-like effect:

There are multiple ways to obtain such a normal. My first thought was Cross Product, but Apple provides a good approximation method in their official demo Implementing adjustable material: using refraction functions for calculation.

The calculation process is as follows: use the refraction function to get the refracted vector (Refract), add it to -ViewDirection to get an approximate new normal (New Normal), then use this to obtain environment information.

Limitations

However, note that its principle is the same as Las Vegas’s spherical display screen - displaying environment information on the object rather than true perspective and refraction. Therefore, it can’t truly show other objects behind it, only environment maps (Skybox) that change based on your viewing angle, creating an illusion of transparency.

For example, when we rotate the angle, we’ll find: you can’t see the dragon through the sphere, nor can you see the sphere through the dragon - they can only display the environment behind them.

Moreover, its refraction isn’t realistic - it just looks transparent with refraction-like effects but doesn’t match reality. Real glass balls not only distort but also invert images, though considering other shapes generally don’t have inversion effects, this effect is sufficient for most purposes.

Choose Based on Your Needs

Until rendering engines fully adopt ray tracing, rendering transparent and glass objects will always be challenging, with no perfect solution. We can only use various techniques to achieve approximate simulation effects.

References

• Sample Code: Implementing adjustable material

Author

XanderXu