Apple Vision Pro - The best and most expensive head display, redefining the next generation of computing

At 2 a.m. on June 6, Apple’s Vision Pro head-mounted display device, which has been developed for seven years, was officially unveiled at WWDC 2023. Cook said that Vison Pro pioneered a new class of computing devices that can integrate the digital world into the real world to achieve augmented reality (AR). This device is compatible with various software of iOS and iPadOS, can work, entertain, shoot spatial videos, and can interact with only hands, eyes and voice. Apple calls this new computing paradigm “spatial computing.” The only drawback is that the final price of Vision Pro is as high as US$3,499 - this price exceeds the US$3,000 previously reported by media and research institutions, and it will not be officially released until early next year. Apple’s stock price even plummeted as a result, and its market value fell by more than $70 billion in less than half an hour.

But it is undeniable that compared with all past VR/AR platforms, the emergence of Vision Pro has created a new era. From human-computer interaction, to hardware specifications, to operating systems, ecology, and data privacy, Apple has redefined the standards for head-mounted devices. Although limited by price and content supply, it will be difficult for Vision Pro to be popularized in the consumer market in a short period of time. But when Oculus Quest sales growth is sluggish and the old paradigm has begun to see its ceiling, the new standards brought by Vision Pro mean that the industry has the possibility to break the bottleneck.

01 New paradigm: hand-eye voice interaction + Eyesight

Vision Pro is a mixed reality device that integrates AR and VR. Users can play games, watch movies, work, and experience the functions of VR in a fully immersive way. They can also use the sensors on the surface of the headset to view people and objects in the outside world. Project into the virtual world to realize the AR function. Each generation of Apple devices has its own unique interaction innovation. For example, the Mac has a mouse, the iPod has a Click Wheel, and the iPhone has multi-touch. The interactive innovation of the Vision Pro lies in the development of a controller that completely eliminates the need for contact with controllers and additional hardware. mode of interaction.

There is no need for a handle for interaction, only the cooperation of eyes and hands, and the movement of the eyes to select the module you want to interact with, such as an App you want to open, and the operation can be completed with small movements of the hands.

You can also call Siri by voice to open and close the App.

Most iOS and iPadOS applications can be used in Vision Pro, such as Safari browser, Keynote, etc. Apple also said it will create a new special version of the App Store just for Vision Pro, which will be the main way to download new AR and VR software. In addition, in order not to isolate users from the people around them, Apple has also developed a feature called Eyesight, which is another basic interaction design of Vision Pro. When someone is nearby, the Vision Pro’s black shell can become transparent, allowing those around you to see the user’s eyes and make those around you aware of what the user is paying attention to.



For example, when viewing an app, people around you can know that you can see them.

But at full immersion, the mask becomes a blur and people around you know you can’t see them. Including when taking photos and videos with Vision Pro, this feature allows people to see you taking photos.

It is worth mentioning that Vision Pro is Apple’s first personal device equipped with a 3D camera, which can take panoramic photos and panoramic videos and play them on the device.

This device can also be used in office scenarios, such as opening multiple monitors to work in a virtual space, or having remote meetings with colleagues.

You can also watch movies at home or on the plane and enjoy a private theater. It is worth mentioning that there is a top knob on the top of the headset, similar to the side knobs of the Apple Watch, which can be used to adjust the size of the virtual space. In this way, even if you are in a small space, you can feel like you are in a vast world.

02 Vision Pro, what kind of headset sells for $3,499

For the first-generation spatial computing platform, Apple developed many “whimsical” functional designs. In order to realize these designs, Apple spent a lot of money on hardware equipment, which also kept the manufacturing cost and selling price of Vision Pro high.

In its overall appearance, the Vision Pro looks like a pair of ski goggles. The headset has a whole glass front panel and an aluminum alloy frame, which contains sensors, cameras, two 23-megapixel displays, and a cooling fan. Comes with external battery and new magnetic charger. The mask and band are cloth-lined and modular, and Apple says they can bend to fit a variety of face and head shapes. The threaded headband can be exchanged for different sizes and styles.

Due to the ergonomic design that fits the face, Vision Pro cannot be used directly with glasses. To solve this problem, Apple and Zeiss collaborated to develop custom lenses that can be magnetically attached to the lens.

There is a knob “Digital Crown” on the top for users to control the depth in VR or AR, similar to using the crown on AirPods Max to control the volume.

The external battery provides 2 hours of battery life and is connected via a “flexible braided cable” that can be placed in your pocket when in use.

In terms of display, Apple says its display will be clearer than ever. Vision Pro is equipped with 12 cameras and is equipped with a Micro OLED screen with 23 million pixels. The pixels allocated to each eye exceed that of a 4K TV, enabling viewing on a 100-inch screen, supporting 3D video viewing, and providing a cinematic video viewing experience.

Of course, there is also standard spatial audio. Vision Pro is equipped with the most advanced spatial audio system. There are a set of dual drive units on each side of the ear. The sound surrounds the three-dimensional sound. It also analyzes the situation in the space to match the best audio effect.

Dual-chip design: Vision Pro adopts a dual-chip design, including a Mac-level M2 chip and a real-time sensor processing chip R1. The R1 chip is mainly responsible for sensor signal transmission and processing. Officials say that the transmitted data can be reached in 12 milliseconds, which is faster than the blink of an eye and almost eliminates delay.



03 visionOS

Vision Pro’s new visionOS operating system will feel familiar yet unfamiliar, built on decades of engineering innovation in macOS, iOS, and iPadOS. Vision Pro is compatible with Microsoft apps, and iOS and iPadOS apps are available in both visionOS and Apple Vision Pro. But the difference is that these icons have come to space.

In the demo screen, the interface design inside the device looks like it is actually in the room where you are. Users can switch between multiple different interfaces in the space, place any screen in front of them, and adjust it freely within the viewing distance. The Vision Pro does not come with a controller, but is operated via voice, eye tracking, and gestures. Users can browse app icons by looking at them, tap to select, swipe to scroll, or issue voice commands. They can also enter text using a virtual keyboard.

Additionally, users can connect their Mac via Bluetooth and expand its display in the virtual space. In other words, in visionOS, there are no restrictions on the boundaries of the monitor. The space in front of you is the monitor. After mirroring the Mac with an external keyboard and mouse, you can even cut videos directly in the space. Users can also project 3D objects into real space, including pulling objects out of message threads and placing them into the real world.

The solution to the problem of user image during video is to make a 3D face in advance, called the Persona function. FaceTime on Vision Pro can display life-size characters during video calls, input the user’s 3D face data in advance, and produce a 3D modeled rendering of themselves. visionOS is the first operating system designed from the ground up for spatial computing, and visionOS adds many new features to support the low-latency requirements of spatial computing.

Such as a new real-time execution engine that can guarantee performance-critical workloads.

For example, the introduction of dynamic foveated rendering can accurately deliver the maximum image quality to every frame your eyes are looking at.

There’s also the first multi-application 3D engine, which allows different applications to run simultaneously within the same simulation.

04 Content Ecology

In addition to good hardware and a good operating system, in order to get more consumers to pay for “spatial computing”, Apple also needs to build a good enough content ecosystem.

At the meeting, Disney CEO Bob Iger made an appearance and said that the Disney+ service will be available on Vision Pro on the launch day. He also said that this is only part of the cooperation between the two parties, and more content will be shared in the next few months. Apple has made Vision Pro available to some developers before, and some amazing content has been developed so far.

An interactive 3D heart created by Complete HeartX.

JigSpace allows designers to visually review their drafts, such as F1 racing cars.

Business operators can use PTC’s Stages to review production lines before they are officially put into production.

The DJay app can be used to mix music.

SkyGuide can lead users into a private planetarium to gaze at the stars. Microsoft Office software and video conferencing software such as Zoom are also connected and can be used directly. Apple also announced a partnership with Unity to allow developers to bring 3D content to the Vision Pro platform.

Vision Pro will have an independent App Store where users can explore and download apps specifically built for visionOS.

05 Privacy and Security: Optic ID

Finally, Apple also talked about the privacy and security issues of the next generation computing platform.

In Vision Pro, user login and authentication will no longer use password login in the PC era or fingerprint and face recognition in the smartphone era, but use Optic ID.

Optic ID uses iris recognition for verification. Human eye iris recognition is unique and is a verification method with a higher security factor than faces and fingerprints. Optic can be used to download apps from Apple Pay or the Apple Store, and the data is encrypted and does not leave the device, and can only be accessed by the security verification program within the device. In addition, Apple has integrated a variety of privacy protection measures into Vision Pro. For example, the gaze point data of the human eye and the video data of the space in which it is located all involve personal privacy. These data will be isolated and processed separately in the background of the device. As the most significant hardware product in Tim Cook’s Apple era, Vision Pro set a new benchmark in terms of hardware and engineering. Apple’s ingenuity also plays a role at the software and interaction levels. But whether this device can become the platform that defines the next decade may ultimately depend on whether the content and ecosystem can be truly established.