Apple Vision Pro

"Vision Pro" redirects here. For the non-linear editing system, see D/Vision Pro.

Apple Vision Pro
Apple Vision Pro headset and its battery pack
DeveloperApple Inc.
ManufacturerLuxshare
TypeStandalone mixed reality headset
Release dateEarly 2024 in the United States
2024 in other countries, such as the United Kingdom, Canada, Japan, India, Brazil, Australia, France, Germany, Spain, Italy and South Korea
Introductory priceUS$3,499
Operating systemvisionOS (iOS-based[1])
CPUApple M2 and R1
DisplayInternal: ~23 MP total (equivalent to WUHD for each eye) dual micro-OLED (RGBB π subpixel geometry) up to 96 Hz refresh rate[2]
External: "EyeSight" curved OLED[3]
Graphics
SoundStereo speakers, 6 microphones
Input6DoF inside-out tracking, photogrammetry, and gesture recognition through 12 built-in cameras and lidar
Websiteapple.com/apple-vision-pro

Apple Vision Pro is an upcoming mixed-reality headset developed by Apple Inc. It was announced on June 5, 2023, at Apple's Worldwide Developers Conference, with availability scheduled for early 2024 in the United States and later that year internationally. It is Apple's first product in another major category since the Apple Watch in 2015.[5]

Apple advertises the Vision Pro as a "spatial computer", where digital media is integrated with the real world and physical inputs, such as motion gestures, eye tracking, and voice input, can be used to interact with the system. Primarily intended as a standalone device booting visionOS, a derivative of iOS designed to run its own extended reality apps, it can also be wirelessly connected to a Mac.

History

Development

In May 2015, Apple acquired the German augmented reality (AR) company Metaio, originally spun off from Volkswagen.[6] rumored to be originally motivated by Project Titan.[7] That year, Apple hired Mike Rockwell from Dolby Laboratories. Rockwell formed a team that included Metaio co-founder Peter Meier and Apple Watch manager Fletcher Rothkopf. The team, called the Technology Development Group, developed an AR demo in 2016 but faced opposition from then-chief design officer Jony Ive and his team. Augmented reality and virtual reality (VR) expert and former NASA specialist Jeff Norris was hired in April 2017.[8][9] Rockwell's team helped deliver ARKit in 2017 with iOS 11. Rockwell's team sought to create a headset and worked with Ive's team; the decision to reveal the wearer's eyes through a front-facing eye display went over well with the industrial design team.[10]

The headset's development experienced a period of uncertainty with the departure of Ive in 2019. His successor, Evans Hankey, left the company in 2023.[11] Senior engineering manager Geoff Stahl, who reports to Rockwell, led the development of its visionOS operating system,[9][12] after previously working on games and graphics technology at Apple.[13] Apple's extended reality headset is meant as a bridge to future lightweight AR glasses, which are not yet technically feasible.[14][15] In November 2017, Apple acquired Canadian XR company Vrvana for $30 million. The Vrvana Totem was able to overlay fully opaque, true-color animations on top of the real world rather than the ghost-like projections of other AR headsets, which cannot display the color black. It was able to do this while avoiding the often noticeable lag between the cameras capturing the outside world while simultaneously maintaining a 120-degree field of view at 90 Hz. The Totem also used IR illuminators and infrared cameras to perform spatial and hand tracking. [16][17]

According to leaker Wayne Ma, Apple was originally going to allow macOS software to be dragged from the display to the user's environment, but was scrapped early on due to the limitations of being based on iOS, and noted that the hand-tracking system was not precise enough to game. Workers also discussed collaborations with brands such as Nike for working out with the headset, while others investigated face cushions that were better suited for sweaty, high-intensity workouts, but was scrapped due to the battery pack and the fragile screen. A feature called "co-presence", a projection of a FaceTime user's full body was also scrapped due to unknown reasons.[18]

Unveiling and release

Information about a headset, then rumored to be called the Reality Pro,[19] began surfacing in 2022. In May 2022, Apple executives previewed the device, including CEO Tim Cook.[20] The company began recruiting directors to develop content for the headset in June. One such director, Jon Favreau, was enlisted to bring the dinosaurs on his Apple TV+ show Prehistoric Planet to life.[21] By April, Apple was attempting to attract developers to make software and services.[22] Apple filed over 5,000 patents for technologies which contributed to the development of Vision Pro.[23] The Apple Vision Pro was announced at the Worldwide Developers Conference on June 5, 2023, with availability beginning in early 2024 in the United States at a starting price of US$3,499 and later in 2024 in additional countries, including the United Kingdom, Canada, Japan, India, Brazil, Australia, France, Germany, Spain, Italy and South Korea, at varying starting prices.[24] An estimated million units will be shipped initially.[25]

On June 6, Apple acquired the AR headset startup Mira, whose technology is used at Super Nintendo World's Mario Kart ride. The company has a contract with the United States Air Force and Navy. Eleven of the company's employees were onboarded.[26]

Specifications

Hardware

Apple Vision Pro has a laminated glass display front, an aluminum frame covered by a flexible cushion, and an adjustable headband. The frame contains five sensors, six microphones and 12 cameras. Two micro-OLED displays with a total of 23 megapixels usually running at 90 FPS from a maximum of 96, each with a size of a postage stamp, will be seen by the user through the lens. The eyes are tracked by a system of LEDs and infrared cameras, which form the basis of the device's iris scanner named Optic ID (used for authentication, like the iPhone's Face ID). Custom optical inserts are supported for users with prescription glasses; these lenses will attach magnetically to the main lens and are developed in partnership with Carl Zeiss AG. The device's speaker is inside the headband and is placed directly over the user's ears. It can also virtualize surround sound.[2][27]

The Vision Pro uses the Apple M2 system on a chip. It is accompanied by a co-processor known as Apple R1, which is used for real-time sensor input processing. The device can be powered by an external power supply, a USB-C port on a Mac, or a battery pack rated for two hours of use.[28][29][30]

The user's face is scanned by the headset during setup to generate a persona—a realistic avatar used by OS features.[31] During a public demo at Worldwide Developers Conference (WWDC), hands-on participants faces were scanned by an iPhone with TrueDepth camera for fitting purposes, and ears were scanned to optimize the speakers, with Apple stating this process can be done with the user's own iPhone or at an Apple Store as part of the purchase process.[32]

A "digital crown" dial on the headset is used to control the amount of virtual background occupying the user's field of view, ranging from a mixed-reality view where apps and media appear to float in the user's real-world surroundings, to completely hiding the user's surroundings.[28][29] The Vision Pro has an outward-facing display called EyeSight, which displays the eyes of the user's CGI persona. Its eyes appear dimmed when in AR and obscured when in full immersion to indicate the user's environmental awareness. When someone else approaches or speaks, even if the user is fully immersed, EyeSight shows their persona's virtual eyes normally and makes the other person visible.[29][27]

Software

visionOS
DeveloperApple Inc.
OS familyUnix-like, based on Darwin (BSD), iOS, iPadOS
Working stateCurrent
Initial releaseEarly 2024
Marketing targetMixed reality headsets, Apple Vision Pro
PlatformsARMv8-A
Kernel typeHybrid (XNU)
LicenseProprietary software with open-source components
Official websitewww.apple.com/apple-vision-pro/
Support status
Beta

Apple Vision Pro runs visionOS (internally called xrOS during its development[33]), which is derived primarily from iOS core frameworks (including UIKit, SwiftUI, and ARKit), and XR-specific frameworks for foveated rendering and real-time interaction.[1][34]

The operating system uses a 3D user interface navigated via finger tracking, eye tracking, and speech recognition. For example, the user can click an element by looking at it and pinching two fingers together, move the element by moving their pinched fingers, and scroll by flicking their wrist. Apps are displayed in floating windows that can be arranged in 3D space. visionOS supports a virtual keyboard for text input, the Siri virtual assistant, and external Bluetooth peripherals including Magic Keyboard, Magic Trackpad, and gamepads.[29][35] Calls can be presented via windows of participants's personas.[32]

The Walt Disney Company announced plans to develop apps for visionOS, including a Disney+ app incorporating mixed-reality experiences.[36] Over 100 Apple Arcade games will be compatible with visionOS at launch.[35][37][38] The company is working with Unity Technologies to support the Unity engine on visionOS.[39][40] WebXR will be supported in Safari for visionOS.[41] visionOS will be backwards compatible with existing iOS/iPadOS apps, which will be rendered in windows within the user environment.[42]

Reception

Pre-release and unveiling

Ars Technica reported that the Vision Pro was "truly something I had never seen before," noting the intuitiveness of its user interface in a choreographed demo given by Apple, and praising a dinosaur tech demo for its immersiveness. Feeling that its displays were dim but "much better than other headsets I've used on this front, even if it still wasn't perfect," and that the personas looked "surreal" but conveyed body language better than a more stylized avatar (such as Animoji or Horizon Worlds).[32] Arguing that the Vision Pro was not a virtual reality (VR) platform, nor a competitor to Meta Platforms' Quest (formerly Oculus) product line, due to its positioning as "primarily an AR device that just happens to have a few VR features," and not as a mass market consumer product.[32]

The Verge similarly noted that Apple had deliberately avoided presenting the Vision Pro as a VR platform, such as not referring to the device as a "headset," describing it as an AR device and "spatial computer," and not showcasing fully-immersive experiences such as games and metaverse platforms in its presentation (only demonstrating non-VR games, displayed in windows and controlled using an external gamepad). He suggested that this positioning "leaves wiggle room for the likely future of this technology that looks nothing like a bulky VR headset: AR glasses."[43]

Bloomberg noted that loud gasps were heard among the crowd in response to the announcement of the Vision Pro price tag, followed by criticism of the cost by both consumers and online commentators. The three priciest components in the Vision Pro are its camera and sensor array, its dual Apple silicon chips, and the twin 4K micro-OLED virtual reality displays. Apple is reportedly working on a cheaper model that is scheduled for release in late 2025 or early 2026 as well as a second-generation model with a faster processor.[44]

The Vision Pro has been criticized due to its high cost, seen by some as too high to go mainstream,[45][46][47] its short battery lifespan,[48] appearing distracting to others,[48] and its lack of HDMI input[49][50] and haptic feedback.[48]

See also

References

  1. ^ a b Wiggers, Kyle (June 5, 2023). "visionOS is Apple's latest operating system". TechCrunch. Retrieved June 7, 2023.
  2. ^ a b Brown, Rory. "Apple Vision Pro: Full Specification – VRcompare". VRcompare. Archived from the original on June 7, 2023. Retrieved June 7, 2023.
  3. ^ "Apple's EyeSight Feature on Vision Pro Is Creepier Than It Needs to Be". CNET. Retrieved August 8, 2023.
  4. ^ Don't Believe the Hype: Apple's M2 GPU is No Game Changer, June 7, 2022
  5. ^ Wong, Aloysius (June 5, 2023). "Apple announces new VR headset Vision Pro, to launch next year". CBC News. Retrieved June 7, 2023.
  6. ^ Wakabayashi, Daisuke (May 28, 2015). "Apple Buys German Augmented-Reality Firm Metaio". The Wall Street Journal. Archived from the original on May 17, 2023. Retrieved June 5, 2023.
  7. ^ Heaney, David (May 20, 2022). "Report Details The Troubled Development Delaying Apple's AR/VR Headset". UploadVR. Archived from the original on June 26, 2023. Retrieved June 26, 2023.
  8. ^ Gurman, Mark (April 24, 2017). "Apple Hires NASA AR Guru to Help Run Its Own Efforts". Bloomberg News. Archived from the original on September 20, 2022. Retrieved June 5, 2023.
  9. ^ a b Gurman, Mark (December 1, 2022). "Apple Renames Mixed-Reality Software 'xrOS' in Sign Headset Is Approaching". Bloomberg News. Archived from the original on May 13, 2023. Retrieved May 27, 2023.
  10. ^ Ma, Wayne (May 20, 2022). "Behind the Apple Design Decisions That Bogged Down Its Mixed-Reality Headset". The Information. Archived from the original on May 9, 2023. Retrieved June 5, 2023.
  11. ^ Mickle, Tripp; Chen, Brian (March 26, 2023). "At Apple, Rare Dissent Over a New Product: Interactive Goggles". The New York Times. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  12. ^ Gurman, Mark (June 5, 2023). "Live: Apple Headset, iOS 17 and Other WWDC 2023 Updates". Bloomberg News. Retrieved June 6, 2023.
  13. ^ Evans, Jonny (May 21, 2018). "Will Apple play nice with others to make Siri smarter?". Computerworld. Archived from the original on May 28, 2023. Retrieved May 28, 2023.
  14. ^ Mickle, Tripp; Chen, Brian X. (March 26, 2023). "At Apple, Rare Dissent Over a New Product: Interactive Goggles". The New York Times. Retrieved June 12, 2023.
  15. ^ Lawler, Richard (January 18, 2023). "Apple reportedly shelved its plans to release AR glasses anytime soon". The Verge. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  16. ^ Kevin Carbotte (October 21, 2016). "Vrvana Totem Mixed Reality HMD Prototype: Beyond Expectations". Tom's Hardware. Retrieved June 12, 2023.
  17. ^ Lunden, Lucas Matney and Ingrid (November 21, 2017). "Apple acquired augmented reality headset startup Vrvana for $30M". TechCrunch. Retrieved June 12, 2023.
  18. ^ "Vision Pro: What features were killed and what could still be coming - 9to5Mac".
  19. ^ Hector, Hamish (June 5, 2023). "Don't get excited for an Apple Reality Pro price switcheroo at WWDC". TechRadar. Archived from the original on June 6, 2023. Retrieved June 5, 2023.
  20. ^ Gurman, Mark (May 19, 2022). "Apple Shows AR/VR Headset to Board in Sign of Progress on Key Project". Bloomberg News. Archived from the original on December 17, 2022. Retrieved June 5, 2023.
  21. ^ Mickle, Tripp; Chen, Brian (June 4, 2022). "Apple Starts Connecting the Dots for Its Next Big Thing". The New York Times. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  22. ^ Gurman, Mark (April 18, 2023). "Apple's AR/VR Headset to Feature Sports, Gaming, iPad Apps and Workouts". Bloomberg News. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  23. ^ Cross, Jason (June 5, 2023). "Apple Vision Pro: Apple's big bet on a brand-new platform starts at $3,499". Macworld. Archived from the original on June 6, 2023. Retrieved June 5, 2023.
  24. ^ Fowler, Geoffrey (June 5, 2023). "Apple unveils Vision Pro, its $3,499 augmented-reality headset". The Washington Post. Retrieved June 5, 2023.
  25. ^ McGee, Patrick; Bradshaw, Tim (March 11, 2023). "Tim Cook bets on Apple's mixed-reality headset to secure his legacy". Financial Times. Archived from the original on May 13, 2023. Retrieved June 5, 2023.
  26. ^ Schiffer, Zoe; Heath, Alex (June 6, 2023). "Apple has bought an AR headset startup called Mira". The Verge. Retrieved June 6, 2023.
  27. ^ a b "Apple Vision Pro". Apple. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  28. ^ a b Wiggers, Kyle (June 5, 2023). "Apple Vision Pro: Here's everything you need to know". TechCrunch. Retrieved June 7, 2023.
  29. ^ a b c d Goode, Lauren (June 5, 2023). "Apple's Mixed-Reality Headset, Vision Pro, Is Here". Wired. Retrieved June 5, 2023.
  30. ^ Goode, Lauren. "Hands on With Apple's Vision Pro: The Opposite of Disappearing". Wired. ISSN 1059-1028. Retrieved June 10, 2023.
  31. ^ Roth, Emma (June 5, 2023). "Apple's Vision Pro headset will turn you into a digital avatar when FaceTiming". The Verge. Retrieved June 8, 2023.
  32. ^ a b c d Axon, Samuel (June 6, 2023). "Hands-on with Apple Vision Pro: This is not a VR headset". Ars Technica. Retrieved June 7, 2023.
  33. ^ "Apple visionOS was called 'xrOS' before last-minute change, WWDC23 keynote and developer sessions confirm it". The Apple Post. June 9, 2023. Retrieved October 13, 2023.
  34. ^ "WWDC 2023 Biggest Reveals: Vision Pro Headset, iOS 17, MacBook Air and More". CNET. Retrieved June 7, 2023.
  35. ^ a b Peters, Jay (June 5, 2023). "Apple announces visionOS, the operating system for its Vision Pro headset". The Verge. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  36. ^ Weatherbed, Jess (June 5, 2023). "Disney Plus is coming to Apple's Vision Pro headset". The Verge. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  37. ^ Peters, Jay (June 5, 2023). "Apple announces visionOS, the operating system for its Vision Pro headset". The Verge. Retrieved June 7, 2023.
  38. ^ Parrish, Ash (June 5, 2023). "Apple's new VR headset will feature over 100 Apple Arcade games at launch". The Verge. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  39. ^ Novet, Jordan (June 5, 2023). "Unity stock jumps 17% on Apple Vision Pro partnership". CNBC. Retrieved June 7, 2023.
  40. ^ Warren, Tom (June 5, 2023). "Microsoft Word, Excel, and Teams are all coming to Apple's new Vision Pro headset". The Verge. Archived from the original on June 5, 2023. Retrieved June 5, 2023.
  41. ^ Lang, Ben (June 20, 2023). "Apple Vision Pro Will Support WebXR for Fully Immersive Experiences in the Browser". Road to VR. Retrieved June 24, 2023.
  42. ^ Roth, Emma (June 10, 2023). "The Vision Pro's biggest advantage isn't Apple's hardware". The Verge. Retrieved August 27, 2023.
  43. ^ Peters, Jay (June 7, 2023). "Apple made a VR headset, but it'll never admit it". The Verge. Retrieved June 7, 2023.
  44. ^ "How Apple Can Bring Down the Price of Apple Vision Headset From $3,500". Bloomberg.com. June 11, 2023. Retrieved June 13, 2023.
  45. ^ Daniel Piper (June 6, 2023). "Apple Vision Pro: The good, the bad and the ugly". Creative Bloq.
  46. ^ Syme, Pete. "The crowd at the unveiling of Apple's Vision Pro headset let out a collective groan when they found out how wildly expensive it was". Business Insider.
  47. ^ Plant, Logan (June 12, 2023). "Fans React to Apple Vision Pro Announcement With Memes, Shock Over Price Point". IGN. Archived from the original on June 12, 2023. Retrieved July 4, 2023.
  48. ^ a b c Jalan, Ayush (June 6, 2023). "Everything We Dislike About Apple's Vision Pro AR/VR Headset". MakeUseOf. Archived from the original on September 4, 2023. Retrieved September 4, 2023.
  49. ^ Cranz, Alex (June 14, 2023). "The Apple Vision Pro might be a TV, but it won't replace the TV". The Verge.
  50. ^ Knibbs, Kate. "Apple's Vision Pro Isn't the Future". Wired – via www.wired.com.

External links