Every day, we interact with the world through screens. Whether we take photos with our smartphones, connect with friends on social media, set timers or use GPS maps, digital devices increasingly influence our experiences.
But what if you could digitally add objects, textures, and people to your surroundings? Augmented reality technology does just that, blurring the boundaries between the physical and virtual worlds. As this technology develops and more industries adopt new applications, understanding augmented reality becomes increasingly important.
What is Augmented Reality (AR)?
Augmented reality (AR) is a technology that blends virtual and physical elements into one integrated display using devices such as tablets or special goggles. AR devices overlay virtual elements onto an image of the physical world.
For example, you can point your smartphone at your living room and use the software to place different chairs in that room – you’re overlaying digital objects (the chairs) onto a real-world setting (the room).
The differences between VR, AR, and MR
Augmented reality is not the only technology that allows humans to interact with realistic digital objects and virtual environments. There are three main categories of this technology with important differences:
Virtual Reality (VR)
virtual reality VR dates back to at least the 1980s, but has become increasingly popular over the past decade due to advances in technology. Virtual reality creates a completely immersive experience for the user, usually through the use of virtual reality goggles. Once in a virtual reality world, the user is completely unaware of the physical world around them. VR replaces the user’s surroundings with entirely digital images.
Augmented Reality (AR)
Unlike virtual reality, augmented reality (AR) virtually adds digital objects to a real-world environment. Using input devices such as cameras, light sensors, depth sensors, and microphones, AR apps place virtual objects into the user’s environment, which are displayed on the phone or AR-enabled device. Augmented reality overlays digital images onto an image of a real-world setting, allowing users to see things that don’t exist in their actual environment.
AR has many uses, including in fashion, beauty, and interior décor apps. For example, a beauty brand can allow customers to digitally try on makeup products before purchasing. Through the device’s camera, the brand’s app can show customers which lipsticks, blushes, and eyeshadows will go well with their skin tone.
Mixed Reality
Mixed reality (MR), also known as AR 2.0, is a combination of augmented reality and virtual reality. Because MR is similar to augmented reality, it is easy to confuse the two. The terms “mixed reality” and “augmented reality” are sometimes used interchangeably.
However, there are some technical differences. The main difference between mixed reality and augmented reality is that mixed reality allows for interaction between virtual elements and the physical environment. In MR, digital objects can sense and react to changes in the environment. For example, augmented reality overlays static objects onto an image, while in mixed reality the filter may change as real-world objects move.
Types of Augmented Reality
There are a few main types of augmented reality, each with slightly different features.
Marker-based AR
Marker-based augmented reality relies on image recognition of a predetermined physical cue to trigger the AR element. This is usually a QR code, a photo, or a visual pattern that the AR software recognizes. As the name suggests, this type of AR relies on some kind of visual marker.
Markerless-based AR
Markerless AR doesn’t require specific markers for the AR software to recognize. It’s an advanced application of AR that requires more computing power. Markerless AR uses object detection to map physical space and adds computer-generated images of the user’s surroundings that are viewed through a phone screen, smart glasses, or other interface.
Projection-based AR
Unlike marker-based and markerless AR, projection-based AR does not require the user to have a mobile device. Instead, projection-based AR uses cameras and projectors to augment the environment by projecting digital images onto physical objects (such as walls or sculptures) or by inserting images into the user’s real-world surroundings.
This could be the appearance of a hologram, or a mapped projection that continually changes the texture of a wall. Projection-based AR allows multiple people to see the same augmented reality at the same time, without each person having to look at their device.
How does augmented reality work?
AR technology requires three main components to work:
1. Input
AR experiences require some input from the real world, typically provided by the mobile device’s camera. However, the experience can also be enhanced with more advanced input. Light sensors, depth sensors, microphones, and GPS can augment the camera’s visual input. Essentially, an input component is any technology that can collect real-time data from the environment.
2. Software
The most complex and invisible part of augmented reality is the task of a computer processing the input it receives. AR software processes real-time data and uses object recognition to process information about the depth, shape, and texture of the environment. It then uses processing power to accurately overlay virtual objects onto the environment. Many augmented reality apps leverage artificial intelligence to efficiently ingest and process data obtained from a variety of inputs.
3. Output
Once the software has processed the input data and determined where to overlay digital elements, the final image needs to be displayed to the user. This is where output devices come in. Output devices include head-mounted displays that place visual elements within the field of view. More common output devices include smartphones, tablets, and projectors that display digital objects in physical space.
For example, a popular example of augmented reality is the video game “Pokémon Go.” This mobile game uses the player’s smartphone camera as the input and a mobile app as the software to process the visual data. Finally, the phone’s screen acts as the output, so when the player looks at the phone screen, they see a digital image superimposed on the surrounding environment.
FAQs on how augmented reality works
What is the difference between VR and AR?
Virtual reality is a fully immersive experience that replaces the user’s environment with a digital one, usually through the use of VR goggles. Augmented reality is a technology that adds digital elements to the user’s real-world environment, creating a physical-digital hybrid. With augmented reality, users can see into their physical environment, as well as a layer of digital imagery added to their real-world experience.
Does AR use AI?
Sometimes. Augmented reality uses sensors to capture data from your real-world environment and then uses that data to precisely add digital content to that environment. Data can come from a variety of sources, including cameras, light sensors, depth sensors, GPS, and microphones. Some AR apps use artificial intelligence to process large amounts of input data more efficiently.
What are the main components of AR?
Augmented reality relies on three main components working together: input (camera, depth sensor, microphone, or other sensor), software to process the input and output (the final image displayed on a screen or projector), either in a dedicated AR device or on a typical modern smartphone.
