Virtual reality replaces what a user sees with a rendered environment. Augmented reality leaves the real world in view and adds registered digital content to it. Mixed reality is the term generally used when the added content is meant to interact with the real environment rather than simply float in front of it.
01The hardware problem
A head-mounted display is a rendering system with unusually strict constraints. It draws a separate image per eye, at a high refresh rate, through optics that magnify a small panel to fill the field of view. Every one of those requirements multiplies the rendering cost of the same scene compared with a flat screen.
Tracking is what separates a display from a headset. Six-degree-of-freedom tracking follows both rotation and position of the head, so leaning and stepping change the view correctly. Modern systems mostly do this inside-out, using cameras on the device itself to observe the room, which removes the need for external sensors. Controllers, hand tracking and eye tracking extend the same idea to input.
The number that governs comfort is motion-to-photon latency: the delay between a head movement and the corresponding pixels appearing. When it is too high, or when the rendered motion disagrees with what the inner ear reports, the result is discomfort. This is why frame rate in VR is a comfort and safety requirement rather than a quality preference, and why techniques that reproject the last frame to match the newest head pose exist at all.
Augmented reality adds a further problem: registration. Content must stay locked to a real surface as the viewer moves, which requires the device to build and maintain a map of the environment and to place persistent anchors within it. Optical see-through displays face the additional physical constraint that they add light rather than subtract it, so genuinely dark or opaque content is difficult.
02Building content for it
Immersive content is built in a real-time engine and rendered per frame, which makes it much closer to game development than to video production. Assets carry hard performance budgets, and the usual pipeline reduces high-detail source geometry to optimised meshes with baked lighting and level-of-detail variants.
Interaction design has few settled conventions and several well-established hazards. Moving the camera without user input causes discomfort for many people. Text at small sizes is often unreadable through headset optics. Interfaces attached rigidly to the head are uncomfortable, and content placed too close to the eyes is difficult to focus on. Comfort options — seated modes, alternative locomotion, adjustable interpupillary settings, subtitles — are accessibility features, not extras.
Delivery targets differ more than they appear to. Standalone headsets run on mobile-class processors within a thermal and battery budget; tethered systems borrow a workstation's graphics capability; browser-based immersive sessions run inside the browser's own constraints. The same scene generally needs different budgets for each, and the practical planning question is which of those targets is actually being supported.