XR World Weekly 026

8/26/2024

Cover of this issue

The cover of this issue is about Black Myth · Wukong from Game Science .

Publication preface

This week, whether you are a gamer or not, I believe your various timelines will be occupied by “Black Myth: Wukong”. Moreover, the producer Hideo Kojima, who holds a high position in the gaming industry, posted a thread when Black Myth sold 10 million copies:

The road to the West has always been a difficult road, and Meta has many trade-offs on this road. For example, this week it is said that Meta has cancelled Quest Pro 2 (compared to last week’s cancellation of the GTA VR version, this news seems to have a slightly smaller impact on players).

News

PICO releases PICO 4 Ultra

Idea

Use water droplets for transitions

Tool

3D Icons: An Open Source Collection of 3D Icons
WWDC Notes

Video

Blender Tutorial 4.0
Meta Immersive Debugger

Code

HLS 360 Video Material for visionOS

News

PICO releases PICO 4 Ultra

Keywords: PICO, MR

This Tuesday, PICO officially released the PICO 4 Ultra, which is PICO’s first device defined by MR (previous PICO series products were all VR All in One).

In terms of hardware, in conjunction with the positioning of “MR devices”, PICO has also equipped Ultra with four environmental tracking cameras, two color perspective cameras, and depth perception cameras.

In terms of price, the PICO 4 Ultra is priced at 4299 yuan (12GB + 256GB), with pre-orders starting on August 20th and official sales starting at 10:00 on September 2nd. During the pre-sale period, PICO can provide a subsidy of 300 yuan for old users (referring to those who have activated PICO machines), and provide a subsidy for exchanging old machines for new ones.

However, for creators in the XR industry, it is obvious that updates to the system and application ecology have a greater impact on us. Therefore, we will focus most of our attention on the system and application ecology.

In terms of software functionality, PIOC 4 Ultra, like Meta, is increasingly approaching mature system designs like Apple Vision Pro.

PICO Connect allows you to cast your Mac, Windows, iOS/Android phone to PICO 4 Ultra:

In addition, for computer screen casting, you can customize the window size, and even set it to the size of the fish screen. (Well, the functions of visionOS 2 have also caught up one by one), and when we can use PICO 4 Ultra to cast the phone to the device, a small pain point that has been unable to play with the phone for a long time has also been solved (there are similar third-party apps doing this on visionOS, see Bezel of XR World Weekly 019 ), the system implementation of PICO is not only screen casting, but also includes operation.

In addition, this time PICO 4 Ultra supports running multiple windows of native Android applications simultaneously (including the projected window seen earlier), which was called the Panoramic Screen Workbench at the PICO launch event. It supports up to 20 applications on the same screen and surrounds the user in a 360 ° manner.

These windows can be interacted with in the form of MultiModal Machine Learning (including keyboard, mouse, controller, gestures).

In terms of spatial video (hey, if you don’t know what spatial video is, don’t forget to check out iPhone What is spatial video on the 15 Pro/Apple Vision Pro? ), the PICO 4 Ultra supports 60FPS spatial video shooting, and you can also view the spatial video shot by the iPhone/Apple Vision Pro, and then use the editor on the PICO 4 Ultra to edit, score, and publish (no need to think, it must be published to Douyin—TikTok’s Chinese version)

But, the functions of mobile screen projection, Spatial Audio recording of spatial videos, 3D special effects, and keyboard and mouse interaction for panoramic screens have not been officially released yet, and we need to wait for the OTA upgrade of the subsequent system.

Idea

Use water droplets for transitions

Keywords: Portal, Transition

Switching from one scene to another very immersive scene, instead of directly using a portal like MoonPortal , maybe we can also try to use something like Jason Cooper designed here, a transition of a water droplet from small to large :

Tool

3D Icons: An Open Source Collection of 3D Icons

Keywords: 3D, Icon, Blender

3D Icons is an open source collection of 3D Icons, providing a large number of free (commercial) 3D Icon resources, we can directly use these 3D Icons in a variety of formats (including Blender-based secondary modification).

WWDC Notes

Key words: WWDC, Sessions, Swift-Doc

As we all know, one of the best ways to learn about Apple Vision Pro and Apple-related framework development is to directly watch the Session videos of WWDC over the years. However, with so many videos, it is always a bit tiring to watch them one by one. Sometimes, for efficiency, you can directly come to WWDC Notes This website first to see if there are friends in the community who have helped you watch it and made some notes and summaries of the sessions you want to watch.

The entire WWDC Notes website is based on Apple’s Swift-Doc , and the entire note content is maintained by the open source community. If you are interested in this project, you can also click here to contribute to this project.

Video

Blender Tutorial 4.0

Keywords: Blender, Tutorial

If you want to learn Blender, then Blender Guru ‘s classic Blender 4.0 donut tutorial is a set of tutorials you must not miss. This tutorial has been widely praised since the Blender 2.x era.

Meta Immersive Debugger

Keywords: Meta, Unity, Debugger

All developers who have debugged immersive VR games may have a common pain: when debugging immersive game scenes, they cannot see their own monitor, and then see the console output on the IDE, or turn on and off some options through the debugging tool.

Fortunately, Meta’s recent Meta Immersive Debugger solves this problem to some extent. This Debugger provides some built-in debugging options, and developers can also set the switches themselves. Dilmer Valecillos also explains this tool in detail in his new video .

Code

HLS 360 Video Material for visionOS

Keywords: Apple Vision Pro, Material, HLS

If you are an Apple fan, I believe you must have seen the term HLS in some of Apple’s official LIVE events. This is a video LIVE protocol launched by Apple.

In the HLS 360 Video Material for visionOS project, Yasuhito Nagatomo implemented an HLS version of Material based on RealityKit, enabling it to play HLS LIVE videos in the sky sphere.