XRealityZone is a community of creators focused on the XR space, and our goal is to make XR development easier!

If you are familiar with English, you can find us in the following ways: Official Website(EN), X(twitter), Medium.

On November 10, 2023, Apple released the second Beta version of iOS 17.2. Although initially available to a limited number of developers, many joined the Beta upgrade program to experience Spatial Video early. The good news is: iOS 17.2 has been officially released! This means that as long as you own an iPhone 15 Pro series phone and upgrade to the latest version, you can start shooting spatial videos anytime and anywhere.

Back in June, when Apple Vision Pro was announced at WWDC, there was much speculation about the form that spatial video would take. Some guessed it involved reconstructing scenes using multi-angle images, others thought it expressed depth with multi-planar images, and some believed it involved rendering scenes using neural radiance fields (though some of these guesses were debunked with the subsequent release of the iPhone 15, as current smartphones don’t have the computational power for such scene expression). Now, we understand that the principle of spatial video is the same as the 3D movies we see in cinemas: using binocular disparity to restore scene depth (we have also introduced this in detail in this article).

Previously, we only had theoretical introductions to spatial video, but now we have a plethora of tools to play with it. Next, we’ll explore these aspects, allowing you to experience the charm of spatial video before you can purchase Apple Vision Pro:

1. How to shoot spatial videos
2. How to create spatial videos that meet Apple’s standards
3. How to watch spatial videos currently
4. One More Thing 😉

How to Shoot Spatial Videos with the iPhone 15 Pro Series

If you’re new to spatial video and have an iPhone 15 Pro series, first make sure your system is upgraded to iOS 17.2. Then follow these steps to start recording spatial video.

Next, enter the camera app and click on video recording to see the settings related to spatial video recording. Here are some basic operational considerations:

1. Pay attention to system prompts. To ensure a better viewing experience on Apple Vision Pro, avoid recording in low-light scenarios like night scenes and avoid recording too close to objects.
2. Don’t worry too much about shooting shake. When shooting spatial videos, the camera will automatically enable stabilization (hence the larger crop), so normal hand tremors won’t cause much shake. If you want a smoother shooting experience, refer to the horizontal reference lines in the middle of the camera.
3. Don’t worry about the recording size. Compared to specifications like RAW recording, spatial video recording is currently limited to 1080P 30 frames due to the physical parameters of the ultra-wide lens. With the efficient MV-HEVC encoding format, a minute-long video is about 200 MB.
4. Don’t cover the wide, ultra-wide, and LiDAR cameras.

How to Create Spatial Videos that Meet Apple’s Standards

If you were an early adopter of binocular video shooting (especially those who joined during the VR live-streaming craze), don’t worry if you don’t have an iPhone 15 Pro series! Using your current binocular device, you can still “shoot” your own stereoscopic memories. Whether you’re shooting binocular fisheye videos or binocular left-right eye videos, you can encode them into a spatial format recognizable by Apple Vision Pro. Here, we recommend two tools for everyone to use:

1. Halocline, an almost “all-in-one” website where you can convert almost all video formats. New users can get 1 minute of free time upon website registration. The advantage is that you can process videos without Apple devices, but the downside is the price of $9.97 per month after the free quota is used up (though there is a 50% discount for students).
2. Spatialify, developed by Wu Tian, can directly process left-right eye videos on iPhone / iPad / Mac (even Apple Vision Pro). The advantage is a one-time purchase for lifetime use at $2.99 (18 Yuan), but the downside is that it doesn’t support the fisheye image conversion process.

The basic function of these two tools is to convert existing Side By Side Format (abbreviated as SBS Format) videos into the MV-HEVC format supported by Apple Vision Pro. So, if you’ve already accumulated a considerable volume of SBS Format videos, you can also convert these directly into MV-HEVC format for viewing on Apple Vision Pro.

How to Watch Spatial Videos Currently

Of course, some might ask, “Now that I’ve shot spatial videos, I can’t watch them!” Indeed, if you really want to get a sneak peek at your spatial videos, you can transcode them into left-right eye videos, which can then be viewed in some mainstream MR headsets, like Quest or PICO. (However, due to different resolutions and decoding algorithms, the effect will differ from watching on Apple Vision Pro).

Here, we’d like to thank E是天翼的翼’s hard work, who, with the help of the XR-Stereoscopic-Viewer project combined with Spatialify mentioned earlier, allows us to directly experience our own shot spatial videos without Apple Vision Pro.

Here, we take PICO as an example for the overall process:

1. Search for “空间西洋镜” in the app store and download it to your PICO.
2. Open Spatialify and select “Batch Export from Spatial Video”.
3. In the export settings, select “Full Width Left-Right Format”.
4. Wait for the export to finish and transfer it to your computer.

Next, simply transfer the video to the /Pictures/3dMdedia/ directory on PICO (Mac users can use Android File Transfer to transfer files).

Then open the app, and you can view the spatial video in PICO:

If you have a Quest, the only difference is that you need to use the APK to install XR-Stereoscopic-Viewer directly, and the other steps are the same as with PICO.

One More Thing

Using cameras to record life transforms the transient into the eternal, allowing us to cross the boundaries of time and reconnect with past emotions and experiences. Using spatial videos creates a more three-dimensional, vivid, and eternal link. Currently, some friends on X have used spatial videos to record and share their lives.

Additionally, after nearly two months of shooting, editing, and viewing experiences, we’ve summarized some tips on how to better shoot spatial stereoscopic videos, which we will share with you soon. Stay tuned!

Contributors to This Article

Link	Image
Onee
Zion

Medium Footer

If you think this article is helpful for you, you are welcome to communicate with us via Email.