Cover of this issue

In this issue's cover, we share the hottest AI Code Editor in the past few weeks - Cursor

Publication preface

Just in the past week, Apple officially announced that it will hold a special event at 1 am Beijing time on September 10, 2024, with the theme of "It's Glowtime" and the following very psychedelic video:

Anyone with discerning eyes knows the meaning of this video - it's too similar to the effect of the new Siri released by Apple at this year's WWDC. It seems that Apple's new hardware product (although we all know what it is, let's pretend not to know 😄) will have a deep relationship with AI - Apple Intelligence.

We have always believed that XR and AI must be a good pair of partners, so this year's WWDC saw Apple Intelligence and visionOS was a bit disappointed (although soon after, unable to use the Swift Assistant sad 😭 was cured by Cursor 😆, test Cursor wrote a lot of new visionOS RealityKit code is also very good)

But after disappointment, think carefully, this is also in line with Apple's usual style - new technologies are first tested on mature platforms and then expanded to other platforms. ARKit and RealityKit, which are now the backbone of visionOS, have also been polished for many years on iOS platforms before finally shining on Apple Vision Pro (you can take a look back at the story behind this Before developing visionOS, you need to know the panoramic view of Apple AR technology ). Let's be patient and look forward to next year's WWDC together~

Table of Contents

Recap

• The gesture matching framework HandVector on Vision Pro has been updated to 2.0, with the addition of FingerShape function

News

• Meta decides to shut down Spark AR effects platform
• Surreal Touch is now available on Kickstarter!

App

• Mixed Snow Worlds: Turn your room into an ice and snow world in an instant

Tool

• Swift-translate: A CLI tool to help you translate Internationalization files in your app

Code

• Quest Display Access Demo: A New Way to Get Quest Camera Data

Recap

The gesture matching framework HandVector on Vision Pro has been updated to 2.0, with the addition of FingerShape function

In this article , we introduce you to the major updates of HandVector 2.0, including:

1. Updated the calculation method of cosine similarity algorithm, changing from hand Anchor space to matrix space based on parent joint points, improving the similarity calculation of fingertip vectors.
2. Added FingerShape function, simplifying finger shape into 5 parameters, including curl and separation, for easy understanding and adjustment.
3. Updated the emulator debugging function SimHand to solve the problem that Google MediaPipes 3D gesture tracking only has joint position without matrix information.
4. Adjusted project structure and 3D joint style, code structure is clearer, joint visualization is clearer.

News

Meta decides to shut down Spark AR effects platform

Recently, Meta's "difficult decisions" have come one after another. On August 27th of the past week, Meta announced that it will shut down its AR effect platform Spark .

Specifically, starting from January 14, 2025, third-party creators will no longer be able to create new AR effects on the Spark platform (including Meta Spark Studio, Meta Spark Hub, and Meta Spark Player), and AR effects created by third-party creators will also be removed from the Meta platform (including Facebook, Instagram, and Messenger).

We have explained Lens Studio and Meta Spark as competing products in the 002 issue . Like the netizens on X , we are also sighing at the end of an era.

Surreal Touch is now available on Kickstarter!

The world's first gesture controller, Surreal Touch, tailored specifically for the Apple Vision Pro, has been officially launched on Kickstarter and has reached its minimum fundraising amount.

With Surreal Touch, you can use Apple Vision Pro to play VR games on SteamVR at 4K 90Hz, including classics like Alyx. If you are interested in playing VR games with Apple Vision Pro, you can purchase it now for as little as 249 dollars, with estimated delivery time in December 2024.

App

Mixed Snow Worlds: Turn your room into an ice and snow world in an instant

We have always believed that simply allowing players to see the real world cannot be called Mixed Reality - at least not excellent Mixed Reality.

In Mixed Snow Worlds , when players play the game, they will see the game scan the player's surrounding environment and transform the corresponding environment into an ice and snow world before starting the next step of the game, which undoubtedly greatly enhances the Mixed Reality. The sense of on-site experience.

The game is currently available for free download on the Quest app store. Interested players can download and give it a try.

Tool

Swift-translate: A CLI tool to help you translate Internationalization files in your app

If you want to do internationalization when you are making a visionOS-related app, you can try using the tool swift-translate to try to simplify the workflow. This is a command line program that can help you send multilingual strings from Xcode to the ChatGPT backend and replace the characters.

Code

Quest Display Access Demo: A New Way to Get Quest Camera Data

As we all know, the camera screen on Quest cannot be obtained by developers normally, but it is also well known that developers still have a way to obtain it. Previously, some developers shared their trick - to obtain it by opening a Quest Mirror page in the background.

This time, trev3d came up with a good solution - directly using the Android MediaProjector API to obtain the camera image on Quest. Although this method is feasible, there are also some small limitations:

• Quest v68 and above system
• Invalid when using QuestLink
• Cannot record properly, need to use a tool like scrcpy for screen recording
• Because it is not normal camera data acquisition, all virtual objects will obscure the entities on the image

The complete code can be viewed on GitHub

Contributors of this issue

SpatialOnee

SwiftSiqi