After the HoloLens' impressive E3 2015 display, and the coming onslaught of virtual reality hardware questions, what lies ahead for traditional gaming, and should fans be concerned?

Microsoft's presentation of HoloLens during its E3 press conference sparked a storm of excitement within the minds of watching gamers. The undeniably impressive demonstration of playing Minecraft using the augmented reality headset transformed the survival/crafting game into something more akin to a god game like Populous or The Sims. When the demonstrator moved the entire game from the two-dimensional projection on the wall to a three-dimensional model rising up out of a table on stage with a simple voice command, the audible amazement of the crowd was well justified.

If that wasn't remarkable enough, the presenter then proceeded to manipulate the blocky world with simple gestures while his fellow demonstrator played from the typical first-person perspective in co-op. Using his all-seeing HoloLens vantage, the demonstrator was able to locate rich mines below the ground and fire lightning bolts at his partner’s TNT to open access to the buried treasure.

The demonstration was, of course, a heavily scripted affair that showcased the optimal HoloLens experience, and the degree to which it reflected what a normal player might enjoy is debatable; the controller-free voice commands in particular seemed a little too good to be true. Given the shaky reliability of Microsoft's current Kinect voice recognition, the likelihood of it functioning so smoothly – especially with non-US accents - seems dubious.
Nevertheless, the demonstration generated a great deal of enthusiasm, along with ample speculation: what could this technology mean for the future of gaming? In ten years, will we be interfacing with all our games this way? Headsets, hand gestures and conversations with computers? Could HoloLens spell the death of traditional, physical-screen gaming?

Unequivocal answers remain elusive until the technology reaches the general public - assuming it eventually does – and our experience with the Halo HoloLens tech demo revealed just how far away that a release date is. But in the meantime there are more than enough tantalising morsels of possibility to chew on. Let us feast.

What exactly is Augmented Reality and why should you care?

First and foremost, a clarification is necessary: augmented reality (AR) and virtual reality (VR) are two separate concepts. Though they utilise similar technologies and share some elements of design, the type of experience each offers is notably different. VR focuses on a wholly transportive experience, whisking the user away to a digital world entirely distinct from the physical one they typically inhabit. It is all about fooling the body and mind into interpreting the visible reality as the real one – and it can be horrifying. Because of the total substitution of one world for another, VR can be as close to or as far from traditional reality as desired - so long as it's consistent. Alien worlds and fantasy realms are well within reach.

AR, on the other hand, is a bit less binary in its implementation. It too can construct a completely new digital world superimposed over the natural one, but that's not its sole imperative, nor its primary one. As the name suggests, AR is about augmenting the world that's already there, enhancing it with the kinds of special effects and futuristic functionality we've been ogling at in Minority Report and The Matrix - and nearly every video game developed since the advent of three dimensions.

Graphical representations of raw data, breadcrumb trails offering contextual guidance to selected destinations, associative data collation reminding you where and when you met the stranger greeting you on the street without having to hesitate while you plumb the depths of your memory. AR exists in parallel with the reality we physically embody improving our world rather than replacing it.

A good example of an existing implementation of AR that, though nowhere near as powerful or polished as HoloLens, still highlights the value of merged realities, is Word Lens. Word Lens is an app that takes the camera feed from a phone and scans for and replaces any text it detects with a translation into the target language selected by the user. It does this all in real-time, integrating the translated text into the video feed as if it was meant to be there, mimicking the font and framing with impressive accuracy. It's far from perfect, with lag and visual artifacts common by-products, but when it works, the effect is quite astounding. As mind-blowing as VR can be, it lacks the wonder of seeing the real world transform before your very eyes.

Looking Into The Crystal Ball

Now, with semantics out of the way, let's start with the most obvious advantage of augmented reality for gaming: heightened immersion. Currently, there are numerous obstacles that need to be overcome to be fully absorbed into a game's world. Ignoring the quality of its presentation and storytelling, factors like screen size, ambient light and noise, peripheral vision, and the visible movement of your hands controlling the keyboard or gamepad all vie for attention, constantly threatening to disrupt the immersion the game is working so hard to engineer.

These impediments are hardly insurmountable, but they still require appreciable effort to tune them out. And that's when it's even possible; I don't know about you, but I've never been particularly good at filtering out the murmur of conversation. I find immersing in a game while playing in a public place nigh impossible.

But what if you could banish all those obstacles to the ether, out of sight and out of mind without the need for sustained effort on your part? With a single command, an AR headset could black out everything in your vision apart from the TV screen, or even construct its own virtual display within the infinity of digital space, immune to light glare or physical obstruction. The incessant barking of the neighbour's dog and the grinding whine of the washing machine would be replaced by sweet, unadulterated game audio.

For all intents and purposes, nothing would exist outside of the game. It would be a perfectly pure gaming experience, with no mental energy wasted on filtering out distractions. And all as easy as a single, transient command; no complaints from parents or roommates as to why the curtains are drawn in the middle of the day and duct tape is covering every window and door seam. The instant the need for returning to the physical world becomes apparent, it's as simple as a flick of a switch. No muss, no fuss. The superlative gaming experience; with AR, you'll never be able to blame distraction for a lousy kill:death ratio again.

Another prominent allure of AR for gaming pertains to its portability. Most games currently require a screen of some sort, a significant physical limitation that is burdened by an inverse relationship between accessibility and visual clarity: bigger, clearer screens take up more space, while smaller screens can only display so much at once before it all becomes an indistinguishable mess. This inescapable fact likely contributed significantly to the basement-dwelling, darkness-loving stereotype that still hangs over gaming to this day; the optimal environment for immersion has always been an empty, isolated space with minimawl distractions.

It is also a major factor in the specialisation of mobile and portable gaming, where the big, bombastic home-console experiences are rarely found. On tiny screens with limited control schemes, those games simply don't have the same impact they do in front of a crisp, high-resolution TV or monitor.

But what if you had instant access to an infinitely adjustable screen everywhere you went, without the struggle of lugging a TV or portable projector around with you? What if you could use the world itself as a screen, playing Pac-Man on the side of seventy storey skyscraper? Or hide the world entirely behind a temporary canvas, your own private game room accessible wherever you are at the press of a button? No more settling for a simple tile-sliding puzzler on your lunch break when you're in the mood for a blood-pumping, chaos-filled dogfight in the endless depths of space, with no expenses spared in the explosions department. Simply don your AR headset and watch the world bow to your will, defying the limitations of reality to deliver the unbridled experience you desire, anywhere, anytime.

How the HoloLens could be a dream.

Ever since the early portable gaming systems like Nintendo's Game & Watch devices, gamers have dreamed of a time when they would truly be able to game whenever, wherever and however they like. Streaming services like PlayStation Now or Remote Play are attempting to address the pixel-pushing problem, but up until now, screen size has always been a mandatory sacrifice. With AR, though, that could be all about to change. Your ideal gaming environment could come with you wherever you go, accessible at the touch of a button.

Say goodbye to the crowded confines of the peak hour train and hello to a peaceful personal cinema room, replete with enormous screen and surround sound. Oh, and don't worry about missing your stop; if you can't set up a GPS alert on your phone, just keep a window open in the augmented cinema to track the passing stops. Unlike virtual reality, where it's a choice between two discrete worlds, the beauty of AR is in the versatility of its experience; you can tailor it to be as transparent or as fully immersive as you like.

Most people would undoubtedly be reluctant to leave their bodies unattended in a public place while they plunge wholesale into a simulated world. For this reason, VR will likely always be a home-bound experience. AR, though, is not fettered by the all-or-nothing investment model. Its awareness of the world beneath its illusions is one of its greatest strengths.

For current gamers who might be concerned that an AR revolution could take from them the traditional style of gaming they hold sacred, such worries are understandable, but also unnecessary. Gaming as we now know it will not be replaced by AR, simply augmented by it. There will still be a market for the standard gaming experience, just as there is still a market now for physical board games, pen-and-paper RPGs and arcade emulation. There is room enough for more than one way to play, and it is this freedom of choice that makes the medium so compelling.

Change can indeed be daunting, especially when it seems to threaten the very thing you hold so dear. But change can also be refreshing. Without denying traditional gamers the ability to enjoy games the way they're used to, AR could enhance their experience with optional, unobtrusive visual aids. A world map layered over their coffee table, to save them from opening a separate menu to orient themselves towards their next objective. Health bars and magic meters adorning the walls beside them, clearing the immersion-breaking clutter from their main vision while still being available for reference when necessary. Auto-pause functionality for when the player's attention is drawn away from the screen, ensuring they never miss a crucial prompt due to their devious dog knocking over the potted plant in the corner of the room.

These features may be less revolutionary than a fully augmented experience, but it is clear that AR could be just as beneficial to the traditional means of play as it could to the new realm of augmented gaming.

The capacity for controller-free gaming with AR holds much promise. Using your own body to affect objects within the game world would not only be a novel means of finer control, it could completely change the way games are both played and developed. Combined with haptic feedback gloves that simulate the weight and texture of in-game objects, physical gestures could become the primary means of interfacing with a game. Not for every game, perhaps, but certainly for many. Being able to manipulate items and tools in ways infeasible with a traditional controller would allow for an exponential increase in possible play styles.

Prying open gaps in fences, groping for hidden switches on the musty walls of a buried tomb, adjusting the grip on a 3W for that perfect drive; the natural control scheme would not only expand the player's arsenal of viable options, but would also significantly lower the barrier of entry for gamers unaccustomed to the standard joysticks-and-buttons template of physical controllers. Navigating a virtual space with gestures directly comparable to the actions performed is a whole lot more intuitive than controller manipulation, just as swinging a Wiimote to simulate bowling was much easier to understand than flicking sticks and timing button presses.

The same people who struggle to comprehend the arcane mastery necessary to guide Mario through a horde of frenetic Goombas while collecting every coin and headbutting every question mark block would have far more luck with an interface that mirrored 1:1 the actions they performed in real life. AR could open up the world of gaming to an entirely new segment of the community, much in the way mobile gaming is currently doing. The unfavourable stereotypes that stem from those who lack personal experience with the medium would be further expunged. Anything that expands the reach of gaming and enables us to share our hobby with more people is something we should champion as hard as we can.

What about general life gamification?

Taking a slightly different tact, it is worth considering the implications of AR in an area that has attracted its fair share of scorn in recent times: gamification? The application of game-like mechanics to activities other than gaming is not inherently bad, though it can be a little jarring when an achievement pops during the latest episode of your favourite TV show informing you that you've received 10 points for watching ten hours of TV in the last two days. The potential for incentivising deleterious behaviour is just as present as the promotion of healthy habits. But that's the same with any tool: the merit is in its use, not in the tool itself. The success of fitness trackers and weight-loss programs is evidence of the positive impact gamification can have on society.

AR provides the means to take gamification to the next level. Rather than having to manually enter data like meals eaten or exercise performed into a website or an app, an AR device could keep track of it all automatically. Scanning barcodes and counting calories would be a cinch. And what about grocery shopping? Not normally the most exciting of activities, but with discounted ingredients spawning tantalising recipe suggestions that cook themselves virtually before your very eyes, as well as a satisfying fanfare every time you shave another dollar off your average grocery bill, the experience would become both financially rewarding and maybe even entertaining - or at least, less boring.

Further opportunity for multilayered satisfaction abounds. Have to clean the house, but dreading the dullness of mopping and vacuuming? Why not turn it into an FPS, a First-Person Sweeper? Dust and grime transform into tiny gremlins that can only be sent back to the demonic dimension from which they came with the power of the BFB - Big Friggin' Broom. Throw a countdown timer up and see what percentage of the evil critters you can banish back to the nether before the dustpocalypse hits. I don't know about you, but I'd sure be a lot more likely to take care of chores if they all involved minigames like that.

Augmented Reality isn't a new concept. Ever since science-fiction novels like William Gibson's Neuromancer and Neal Stephenson's Snow Crash, the idea of a digital world superimposed on top of the physical one has been omnipresent in the minds of the children of the computer age. Star Trek's Holodeck epitomises the fantasy: access to the infinite possibility of the virtual realm without the need for a clunky computer interface or some other obstacle between the body and the manifestations of imagination.

Like all fantasies, though, the bridge to reality has not been easy to build. The technology necessary to bring the binary into the organic has been slow coming. There have been many attempts, with recent endeavours like Google Glass or the dinky AR cards included with the Nintendo 3DS proving to be more of a gimmick than a legitimate merging of the two discrete domains. But with HoloLens, it might finally be possible that true convergence is just around the corner. The headset might be bulky, and the field of view might be limited, but the experience at its core could deliver on the promise of those fantastical stories like no other technology ever has.

Though it is likely that HoloLens, along with most other large-scale AR initiatives, will focus primarily on productivity and commercial utility rather than gaming simply because of the broader market and the greater potential impact, there is still much for gamers to be excited for. Whether it be enhancing existing experiences or crafting entirely new ones, I for one am eager to welcome our augmented future.

Now, when can I expect to finally get my hoverboard?

Writer:
Matt Sayer

New comment