Textbook in PDF format Extending traditional digital platforms to the new frontier of extended reality (XR) requires taking into account what best practices, new concepts, and conventions have been established and what learnings can be brought forward from case studies involving industry leaders. By looking at practical examples from the field of handheld AR breakthroughs, virtual reality (VR) success stories and experimental interaction concept of pioneering XR platforms, you'll see how it's possible to map out a framework of user experience (UX) guidelines to close in on opportunities and challenges that lay ahead. UX for XR defines, identifies, and analyzes UX practices for XR environments and reviews the techniques and tools for prototyping and designing XR user interactions. You'll approach the design for experiential state and spatial cognition, using established UX key performance indicators, while taking into account the social dynamics, emotional framework and wider industry context. UX design for handheld AR has to consider the fact that the environment is part of the design context. This fact makes it different from mobile and web design approaches, where the interaction is limited to a screen and the app’s UI. Another important aspect is that AR objects live in a 3D world. This means objects have to be designed as 3D geometry. Even if the object is a 2D plane, it is still placed within a 3D context and can often be viewed from different 3D viewing angles; as a consequence, its 3D context has to be considered. Specific solutions like environment guidance for users and off-screen object notification are often required to make sure the flow is not interrupted. The 3D content and the possible trigger animations need to be designed in a way that works in AR. To help with prototyping, an increasing variety of AR design apps have helped streamline the process that would otherwise be very technical if only using Unity or Unreal with ARCore and ARKit as a starting point. Adobe Aero, for example, enables UX design prototyping and testing for user interaction within an environment using imported 3D objects. The application allows the testing and fine-tuning of typical object properties, such as trigger types and trigger actions. This allows UX designers to evaluate 3D design choices that can then be adjusted according to the test results. Unfortunately, since the writing of this book, Adobe Aero has been discontinued. However, the examples in the following chapters that use Adobe Aero illustrate general AR workflow concepts, which can also be recreated using platforms such as 8thwall.com or zap.works. More complex web-based production and prototyping tools such as 8th Wall provide an even more complete prototyping solution using a visual editor for interaction design, while the prototyping app ShapesXR is a sophisticated tool for spatial ideation in XR, including a complete pipeline with Figma and Unity. At a basic level, AR prototyping tools provide a sandbox for planning out the core components, spatial layout, UI, and object components to test an idea or concept. Being able to sketch out ideas for onboarding, user flow, and UI interaction using these tools is a helpful shortcut in the development process. Ideation and prototyping play an important role in AR, as small changes in the 3D environment can have a dramatic impact. Fine-tuning and testing is required when the position or size of a placed AR object is important. Visual stimuli, such as animations, help to signal if an object has changed its properties, has become active, or is awaiting action. Voice and sound in general may help with navigation and assistance. User testing helps to evaluate if the AR design is delivering the expected outcome. Compared to UX/UI web and mobile design best practices, users may find it hard to rotate objects, readjust the view, or navigate active items. Therefore, a visual language that is able to assist with the application’s usability has to be part of the design solution. The additional responsibilities and possible pitfalls are manifold, when comparing UX design for digital AR products with UX approaches for mobile and web applications. This new edition has been fully updated with new cases studies and examples and explores new developments relating to Apple Vision Pro, the new paradigm of Mixed Reality HMDs and includes brand new chapters on spatial computing and AI applications to XR. You Will: Explore the challenges and opportunities of designing for XR See how spatial interaction is revolutionizing human computer interaction Examine sensory input and interaction beyond the screen Work with 3D Interaction Design and build a strong 3D UX Understand VR and augmented reality essentials for emotion-rich user experiences Apply UX research techniques for the XR space This is for: UX designers, consultants, and strategists; XR developers and media professionals. Contents: Preface About the Author About the Technical Reviewer Introduction The History and Future of XR The Rise of UX and How it Drives XR User Adoption UX and Experience Design: From Screen to 3D Space Pioneering Platforms and UX Learnings Spatial Intelligence and AI Oppurtunities New Design Frontiers in Spatial Computing Practical Approaches: UX and XR in the Real World Appendix A: Glossary Appendix B: Resources Index