Before you start an augmented reality project, read this

What do Terminator, Iron Man, and Robocop all have in common? They all use some form of augmented reality (AR) in their day jobs. If you’ve ever seen a heads-up display in a vehicle or played the mobile game Pokémon Go, you’ve experienced AR.

But what exactly is AR and what can we do with it today?

What is AR?

AR blends digital environments or objects with reality and allows you to interact with the digital environment. This has a ton of applications in everything from industrial uses to education and entertainment. AR is a great way to provide a unique experience to a user and stand out among the crowd. It’s also a great way to drive engagement and get people talking about your brand, product or idea.

Imagine you’re shopping online for a new bookcase. You see one you like, but you’re not sure how it would look next to the rest of the furniture in the room. An AR-enabled experience within an online store would allow you to use your phone to digitally place the shelf in your room to see how it fits in.

Have you ever been to a museum and wanted to know more about a piece of art? Adding a scannable QR code next to the art work could launch a video describing more about the artwork.

You could take a virtual tour with your phone and an AR-enabled app can overlay a 3D reenactment at a specific location using GPS. These are just a few of the things you can build with AR.

AR can be used to display digital information and context within natural environments and make them more interactive.

AR was mostly out of reach for the average person until the last decade or so when mobile phones managed to put a small computer in everyone’s pocket. Now that both iOS and Android phones have built-in support for many AR features, AR is more accessible than ever before.

What kind of experience can you create? While there are many different techniques, we can break down most of the current AR experiences into a few types.

Types of AR

Since AR uses a camera to get information about the outside world, it’s most common to focus on building and optimizing a mobile device, although PCs and laptops with webcams or wearables can be targeted as well. The type of AR you use depends on the goal of the experience.

Marker-based

A marker-based AR experience requires a specific image or icon to be visible within the camera view to trigger the display of content. Often a combination of a QR code and a marker image is used in Web applications to make it easier both to get the user to the experience and to start the experience, rather than requiring them to type in a URL.

Play Video Marker-based Augmented Reality Sample

Once you view the marker image through the camera, it can overlay anything on top of the marker. You could print multiple markers on a printed recipe and then overlay a video on each to show different steps of the recipe.

You could place a marker on a product and display a 3D-animated model to show how easy it is to assemble.

Scan the QR code below with your iOS or Android device and point towards the logo to view an animated version.

Markerless

Markerless AR can refer to many different experiences. You may also hear the terms World Tracking, SLAM, or surface tracking referring to this type of experience.

A markerless experience uses a phone camera to map the local environment in order to place items and track them as the camera moves.

Some e-commerce stores use this to let you place virtual furniture in your home to see how it fits in with the rest of your furniture. While any 3D object can be placed in a camera view without AR, it’s the surface-tracking feature of AR, allowing the object to snap to the floor or flat surfaces easily, that makes the experience more immersive.

Markerless AR can also be used in combination with maker-based AR to set the starting location of virtual items in a scene.

Location-Based

Location-based AR uses the location of the device (its GPS coordinates) to display a virtual object at a certain location in the real world. Pokémon Go is a good example of this type of AR.

In Pokémon Go, GPS data is used in order to know where to show a Pokémon. If an optional AR+ mode is enabled, it can use SLAM-tracking to place the Pokémon on a surface within the environment.

A museum could use location-based AR to display videos or even add a historical figure in 3D talking about a display in front of you at certain physical locations, like virtual tour guides.

Face Tracking

Face tracking AR allows you to detect human face features and their location within a camera view. This allows you to place objects at certain areas found by the tracking.

Common examples are adding masks or sunglasses to your face in the camera. You could implement a virtual try-on, allowing your customers to see what different pairs of sunglasses would look like on their face without them having to leave the house.
Face tracking also allows people to display a virtual avatar in place of their real face when making videos or livestreaming.

Technology

Many AR experiences are built around mobile devices, using either mobile apps that must be downloaded from an app store or web apps that can be run directly on a website. There are advantages and disadvantages to consider when deciding whether to build an AR experience with a downloadable app or a web app.

A downloadable mobile app costs more to build and deploy than a web app and requires you to convince someone to download a new app to their phone. You also must support two separate apps, one for iOS and one for Android. This requires more time in development and testing as well as getting both apps accepted by their respective app stores.

The advantage of a mobile app though is that you can access the full native capabilities of both platforms. This can help you create more immersive and better performing experiences.

A web app can be published in one place and be accessed by both iOS and Android devices. Web apps cause less friction because they keep the user on your website or microsite without requiring them to download a new app but may not have the full capabilities or performance that a downloadable mobile app would have.

iOS and Android both have frameworks included (ARKit and ARCore), which are available to mobile apps currently and where the better performance and capabilities come in.

Android has started supporting a new standard called WebXR which allows web app capabilities and performance to become very close to mobile app performance. Currently iOS doesn’t yet support WebXR though it is on their roadmap. When WebXR is fully supported, expect to see more and more AR experiences on the web.

Which AR experience is right for you?

There are many different options when building an AR experience. Is it right for your project and what type of AR would provide the best engagement? If you want to stand out and create a unique campaign or experience, AR is a great way to get people talking about your brand and give them easier access to information about your product.

We love building all types of digital experiences including interactive websites and more. Interested in creating a digital experience of your own? Contact us.