XR World Weekly 016

1/14/2024

This Issue’s Cover

This issue’s cover features the TVA Timedoor, designed by Max using Spline. Indeed, for Spline, creating such photorealistic, interactive 3D model scenes is no longer a challenge.

BigNews

Release date for Apple Vision Pro confirmed!

Idea

Seasons—Immersive Weather Experience
Doing chores with Quest, making chores no longer boring!

Tool

UEVR—Bringing VR support to Unreal Engine games in your Steam library
Rokid—Free export of spatial videos to SBS format
Genie—Luma.ai’s Text-Generated 3D Tool

Video

Polyspatial Quick Start / Play to Device
Making a Cel / Toon Shader in Reality Composer Pro’s Shader Graph

SmallNews

At CES, Sony launches new XR headset

BigNews

Release Date for Apple Vision Pro Confirmed!

Keywords: Apple Vision Pro

Long-awaited, Apple officially announced in this press release that the Apple Vision Pro will be officially on sale in the US on February 2, 2024, with pre-orders starting on Friday, January 19, at 5 AM Pacific Time (corresponding to 9 PM on Friday, January 19, Beijing time).

Not just the release date, from the official announcement (combined with some of our exclusive news), we can also glean previously undisclosed information, including:

The starting price of Apple Vision Pro is $3,499, equipped with 16 GB of memory and 256GB of storage space (versions with higher capacity will be available)
In terms of productivity, apps like Fantastical, Freeform, JigSpace, Microsoft 365, and Slack have been adapted specifically (as Apple’s announcement wouldn’t mention an app for no reason)
In gaming, Blackbox, Game Room, What the Golf?, and Super Fruit Ninja have been adapted or newly developed for spatial computing scenarios (Apple’s official term for these games is Spatial Games)
In entertainment, content providers like Apple TV+, Disney+, and Max are offering HDR-supported video content, and in Apple TV, 150 3D movies are already prepared (as we mentioned in 012, there were 19 movies supporting 3D format on Apple TV as of November 2023); furthermore, Apple revealed a new format for 180-degree panoramic videos—Apple Immersive Video, a 3D format with a clarity of 8K covering 180 degrees (the official website even showcased the scenes of Apple shooting these Immersive Videos)

There are some more details in the announcement, such as this fleeting video from the visionOS App Store in the announcement, which gives us a sneak peek at some of the native visionOS apps that will soon be on the Vision Pro App Store homepage.

For example, the most prominent on the homepage is Disney+. You can watch Star Wars on a Star Destroyer, watch the NBA from a God’s eye view, and maybe even become the Watcher Uatu, experiencing What if…

Another app that made an early appearance on the App Store homepage in the official introduction is Sky Guide — this AR software used for astrophotography and constellation observation will be a whole new experience on Vision Pro:

Other first-launch native visionOS apps include: the Apple Design Award-winning Blackbox (Blackbox has been entirely redesigned for the visionOS platform!), JigSpace showcased as part of the ARKit feature, Microsoft Office software (including Microsoft Word and Microsoft Excel), and a new Apple Arcade game, Game Rooms.

JigSpace and another app PTC, not featured in the App store showcase video, were chosen by Apple as exemplars of enterprise-level app design and development.

The video’s finale showcases native apps like Zoom, newly launched Onshape and Super Fruit Ninja, and of course, djay and Complete HeartX, which were showcased on the first day of Vision Pro’s announcement at Keynote.

djay, in particular, frequently appears in the official Developer News, making it a highly anticipated app!

The latest report on how djay achieved their vision was published on January 11, 2024 here.

Also of note, though briefly glimpsed and easily overlooked in the Keynote, is the anticipation for what will be presented in visionOS’s Keynote. Previous media reports also covered what Keynote will look like on visionOS, including features such as simulating presentations in spatial environments.

Based on our summary of various sources, the current Apple Vision Pro package contents are (prescription lenses are an additional purchase):

The specific contents of the package and all Chinese translations of terms are subject to the final release by Apple

In addition, in anticipation of the Apple Vision Pro, a revolutionary product, Apple officially released an advertisement: Get Ready. In this ad, Apple featured scenes from various classic movies where different characters put on blindfolds, as if to say, wearing the Apple Vision Pro, you too can become “Some One”. From a user’s perspective, this indeed might change the minds of many who think “wearing a head cover is silly.”

It’s worth noting that 17 years ago, when the iPhone was first released, Apple used a similar filming technique for an advertisement. In this ad, scenes from various classic movies were featured, with different characters picking up the phone and saying “Hello”:

It seems Apple is fully comparing the Apple Vision Pro with the iPhone. Additionally, if you’re a developer in the Apple ecosystem, you’ve likely received an email from Apple pushing the latest guide on how to submit apps to the visionOS app store here. With the Apple Vision Pro’s release date set, Xcode has officially added support for visionOS in version 15.2 (yes, one less barrier to starting to learn visionOS).

One last thing to remember, if you plan to purchase the Apple Vision Pro, make sure to download the Apple Store app on a device equipped with Face ID in advance, as facial scanning is required before purchase to determine the size of the headband. And according to Mark Gurman’s report, we can reasonably guess that in the future, the iPhone might also be needed to calibrate some Vision Pro accessories.

Tips

If you haven’t yet experienced the magic of visionOS apps, you’re welcome to try them out in a simulator using the Let’s xrOS project.

Let’s xrOS is a platform for exchanging visionOS apps, allowing developers to distribute their visionOS apps to enthusiasts around the world. Rest assured, the distribution mechanism is as secure as Apple’s TestFlight, only requiring the app’s binary, no source code required.

Idea

Seasons—Immersive Weather Experience

Keywords: visionOS, Apple Vision Pro, WeatherKit

If you’re lucky enough to get an Apple Vision Pro after its release, be sure to visit the visionOS AppStore to try this app called Seasons.

This app is a prime example of Apple’s convenient developer tools. By leveraging Apple’s WeatherKit combined with SwiftUI + Reality Composer Pro, Seasons successfully displays 34 types of weather through immersive scenes. In Seasons, we can see:

Cloudy days with drifting leaves:

Drizzly rainy days:

And of course, snowy days with falling snowflakes:

It’s noteworthy that this app was developed by a Chinese developer, Cosmo, and his team from Shenzhen. We’re very excited to see more great works from Chinese developers in the world of spatial computing.

Doing Chores with Quest, Making Chores No Longer Boring!

Keywords: Quest Hackathon Winner, MR, Quest

What if doing chores felt like playing a game? Would you start enjoying household tasks?

The following product concept comes from a four-person team at the Meta Quest Hackathon. By utilizing Meta’s Presence Platform, they transformed the mundane act of vacuuming into an exciting “weed-clearing” experience.

When spatial applications move beyond simply displaying 3D models to integrating more with real-world scenarios, they create wonderful chemical reactions.

Tool

UEVR—Bringing VR Support to Unreal Engine Games in Your Steam Library

Keywords: Steam, VR, UEVR, Unreal

UEVR, developed by praydog, is a mod for Unreal Engine games. By installing UEVR, you can convert games in your Steam library, developed with Unreal Engine (versions 4.8-5.3), into VR games playable with various VR headsets.

As a mod rather than a framework requiring developer integration, UEVR is incredibly user-friendly. The basic usage involves:

Opening UEVR on your PC.
Launching the game you want to inject.
Configuring the injection parameters, including the target game process and selecting the runtime framework (OpenVR/OpenXR).
Starting the game!

The actual gameplay experience varies with the game. For example, in the game illWill, you can play in first-person view, and use VR headset controllers to operate weapons realistically:

In other games, you might only be able to play in third-person view with a traditional game controller:

Currently, the list of games supported by UEVR is continually expanding. If you want to experience your favorite games in VR, you can check the official spreadsheet to see if they’re included. Additionally, you can learn more about getting started with UEVR through these two videos: UEVR - Getting Started Basics and UEVR Mod: An Idiots Guide.

Rokid—Free Export of Spatial Videos to SBS Format

Keywords: Rokid, Spatial Video

In our previous post How to Play Spatial Video on iOS 17.2, we recommended Halocline and Spatialify for converting spatial videos into SBS format for viewing on devices like PICO/Quest. Considering the dedication behind the Spatialify app, its 18 Yuan price tag is quite reasonable, especially with the latest version supporting additional features like spatial photo capture!

However, if you just want to simply export an SBS format video, the Rokid AR App is also a great option.

In the latest 2.0.8 version, Rokid has updated the app to include a feature for exporting spatial videos, with a basic operation logic similar to Spatialify. Just select the spatial video and export it:

Genie—Luma.ai’s Text-Generated 3D Tool

Keywords: AI, Luma.ai, Text to 3D

Luma.ai, frequently mentioned in our previous issues, has recently launched their Text to 3D tool—Genie. Our trials found that the tool generally generates models quite quickly, taking about half a minute to create a basic usable model. For example, we entered “A cute cat”, and in about half a minute, a (somewhat rough) cat model was generated.

It’s worth mentioning that besides Luma.ai’s Genie, another startup company, tripo3d.ai, has also launched a similar rapid 3D model generation tool, providing an API for enthusiasts to experiment with. For instance, a netizen created an online PK game, where the winner of the PK takes the loser’s model keyword, merges prompts using GPT, and regenerates (evolves) into a new model using tripo3d.ai. Here, a cat successfully PKed a little girl and evolved into a cool Catwoman.

Those who follow related technology may notice that compared to CSM.ai introduced in Issue 012, the field has evolved from minute or hour-long generations to generating four sets in just tens of seconds over just a few months.

Further analysis shows that early methods, like zero123 and related technologies, used a combination of 2D diffusion models and NeRF reconstruction, mainly limited by the reconstruction speed of NeRF itself. In contrast, Luma.ai and tripo3d.ai trained a large Transformer-based reconstruction model (like LRM or TriplaneGaussian shown above), directly inferring the final NeRF or 3D Gaussians reconstruction result from 2D images.

Video

Polyspatial Quick Start / Play to Device

Keywords: Unity, visionOS, Polyspatial

If you’re eager to quickly learn how to develop visionOS apps using Unity with Polyspatial, these two videos by Dilmer Valecillos are highly recommended for beginners:

The first video primarily introduces how to incorporate the Polyspatial SDK into a project and debug in visionOS. The second video mainly explains how to use Play to Device, a method that allows for real-time preview of effects in the Unity Editor upon saving.

Making a Cel / Toon Shader in Reality Composer Pro’s Shader Graph

Keywords: visionOS, RealityKit, Reality Composer Pro, Shader

In this video, Mark Robinson explains how to use Reality Composer Pro to build a Cel/Toon Shader. The end result is the effect seen on the spheres, skull, and airplane in the image above.

It’s important to note that this tutorial isn’t aimed at complete beginners. Therefore, before watching this video, you should have some understanding of Reality Composer Pro’s Shader Graph (or a basic understanding of Unity’s Shader Graph).

SmallNews

CES - Sony Launches New XR Headset

Keywords: CES, Sony

At the recent CES, Siemens CEO Roland Busch announced that they will be launching a new XR headset later in 2024, in collaboration with Sony. This headset will feature a 4K OLED screen and a Qualcomm XR2+ Gen 2 chip. It’s primarily aimed at professional engineers, meaning it’s designed for those who can use CAD software, rather than for general consumers interested in movies and gaming.

The headset features a flip-up design for easy on and off.