Cover

For this issue’s cover, we’re sharing the Spline scene Lighting Bulb created by Max.

Editorial

The most significant news this week should be the debut of the Orion prototype. Although it’s more expensive than the Apple Vision Pro and people have had longer and less expected hands-on time (yes, Meta hasn’t given a clear expectation of when Orion will be officially available to consumers), the release of Orion has still caused quite a stir both within and outside the industry. Looking at the WeChat index data, this release of Orion has received more attention than the Apple Vision Pro’s launch in February.

Many articles and videos might say that the future is already here, blah blah blah, but personally, I think the greatest significance of Orion’s release at this stage is to give people hope. It shows people that AR glasses are not an impossible product, and existing technology can still create a product that barely meets hardware specifications. At the very least, the day when we expect everyone to be able to use AR glasses should not be more than ten years away (🤣). Just like the light bulb on this issue’s cover, we can now see some faint but steady light.

Table of Contents

News

• Meta Connect Conference
• Meta Released Meta Spatial SDK
• Apple Provided New visionOS Sample Code

Tools

• PICO Emulator(Beta)

App

• Story: Spatial Storytelling

Code

• SGMExamples: Shader Graph Material Examples in visionOS
• RealityMixerVisionPro: Mixed Reality Capture app for the Vision Pro

News

Meta Connect Conference

On September 25th, local time in the US, the Meta Connect conference officially opened. At this conference, Meta unsurprisingly released the Meta Quest 3s that everyone was already very familiar with in advance.

There’s not much to say about the Meta Quest 3s. You can think of it as a cheaper version of the Quest 3. The CPU and lenses haven’t changed much (the lenses have even slightly decreased, returning to the Quest 2’s resolution level). The 128GB version is priced at $299, and the 256GB version at $399, which is about 2000 RMB. It’s available for pre-order on the day of release and will go on sale on October 15th. Buyers will receive a Batman: Arkham game.

By the way, the name Quest 3s is indeed confusing. It might have been better to call it Quest 3 lite.

In this release, Meta’s description of Quest 3s, as well as the operating system Horizon OS, all describe it as a mixed reality product.

The two bigger points of this conference were Orion—a product that Meta claims to have been developing for 10 years. This is an AR glasses that is not yet available to consumers. It uses silicon carbide lenses, has a 70° field of view, a magnesium frame, a self-developed chip, supports eye tracking and gesture recognition, has about two hours of battery life, and weighs only 90g. It also comes with a portable computing unit called the Wireless Compute Puck (which seems to have a camera on it, but it’s not yet known what this camera is specifically used for), and a wristband that can sense myoelectric signals, which can be used when Orion can’t detect the hand.

Currently, in addition to the pinching of the thumb and index finger, this wristband can also sense a sliding gesture, which seems perfect for up and down scrolling scenarios:

From the information released at the conference, the cost of an Orion AR glasses is about $10,000 (based on 2022 mass production data). Due to the high material cost, Meta had to shelve the official launch plan for this product and only manufactured 1,000 pairs for internal company demonstrations and partner trials.

Personally, I think the greatest significance of Orion’s appearance at this stage is to let everyone realize that existing hardware technology can actually achieve a lightweight, 6DOF AR glasses that supports eye and hand tracking. Although the current technology level can only achieve a 70° FOV (which is already a very large FOV in AR glasses), this is already a very good start. (If you look carefully at the reviews from foreign media, you’ll find that Orion’s current 70° FOV is still clearly visible with a truncated edge when worn, which will at least give people a very obvious feeling of “breaking immersion”.)

Many excellent articles have already provided detailed introductions to more details about Orion. If you’re interested, you can click to read:

• The Strongest AR Glasses Are Here! Meta’s Ten-Year Secret Project, Your Next Phone Might Be Glasses
• The Verge: Meta’s big tease
• My First Dive Into Meta’s Orion AR Glasses And Neural Wristband

Meta Released Meta Spatial SDK

Developers familiar with the love-hate history of Android and iOS will be no stranger to this scene: Meta has released a brand new SDK for development on Quest—Meta Spatial SDK. With this SDK, you can use Android Studio and Kotlin to develop spatial applications on Quest (these two things are very familiar to Android developers).

In Android Studio, you can edit the UI of spatial applications based on XML:

You can also edit 3D scenes in the Meta Spatial Editor, similar to Reality Composer Pro used in visionOS development:

The concept is truly consistent, and the official has also released 10 Sample Codes. Interested friends can go directly to Github and quickly get started according to the documentation.

Apple Provided New visionOS Sample Code

Recently, Apple provided new visionOS Code Samples. Compared to the previous relatively complete large project Sample Code, this time’s Samples are all relatively simple small projects, suitable for beginners to get started.

Tools

PICO Emulator(Beta)

PICO recently released a new debug tool on their official website: PICO Emulator. Its main function is to simulate running applications that originally needed to run on PICO real machines on PC and Mac. You can install it via adb or by directly dragging and dropping the apk. At the same time, the emulator itself also supports keyboard and handle mapping.

App

Story: Spatial Storytelling

Story is an App that can bring an extremely immersive video experience to Apple Vision Pro users. Here, you can not only watch 360° immersive videos composed of pictures, models, and text like this:

You can also immersively view landscape videos with 16K clarity:

Currently, there are five videos in the App, and more videos will be released in the future.

As an immersive video playback application, Story has two surprising points. The first point is the video clarity of up to 16K (this is a perfect match with the resolution of Apple Vision Pro). When viewing landscapes, the extreme clarity will really make your eyes light up.

The second point is the vision behind Story: to enable more people to create such spatial videos. Therefore, we can say that Story is more of a content platform. Let’s take a look at the editor that can be used in Vision Pro:

Code

SGMExamples: Shader Graph Material Examples in visionOS

To create cool and beautiful Apps, Shader Graph is definitely a must-have skill for visionOS developers. But unfortunately, the official only has documentation and lacks some more advanced examples, which brings some difficulties for beginners.

But now, with SGMExamples, you can better learn how to use Shader Graph through this project. This project includes many advanced Shader Graph usage scenarios, and this water ripple is one of them:

Many thanks to Yasuhito for open-sourcing such a great project

RealityMixerVisionPro: Mixed Reality Capture app for the Vision Pro

All friends using Apple Vision Pro may have encountered a problem: people around you can’t see the content in your glasses. Although we can solve this problem through screen casting, this solution is not optimal.

Many thanks to Fabio Dela Antonio for his attempt on this issue. Through RealityMixerVisionPro, you can project what Vision Pro sees onto your phone in a three-dimensional form, so your friends can see the content in your glasses.

Contributors to this issue

SpatialOnee

xChester

SwiftSiqi