Augmented Reality
A Brief Overview of Augmented Reality
Augmented reality uses technology to add an additional layer (or layers) of information to real life. “The basic idea of augmented reality is to superimpose graphics, audio and other sensory enhancements over a real-world environment in real time” (Nandu91, 2012, p.1)The most common way to augment reality is through visual displays. Smartphone users can download applications that show additional information such as restaurant locations and reviews when a location is identified. Surgeons can use headgear connected to a computer interface to superimpose MRI imaging data onto a patient’s body to pinpoint a tumour (Educause Learning Initiative, 2005). The US military are also using augmented reality with a headset that can overlay blueprints directly onto a soldier’s field of vision (Bonsor, n.d.).
Other senses can also benefit from augmented reality. As well as local information in the video display, smartphone users can access audio tours on a location. Gadgets are already available that can interact with websites to release scents ("I-aroma smell dispenser," n.d.). The sense of touch can be augmented through the use of haptic interfaces such as cybergloves ( Sterling, 2010). Even a person’s sense of balance can be augmented to give the impression of being under water or in outer space (Ruhl, 2011).
Augmentation of multiple senses is another area of exploration. At the University of Toyko, a headset was created that sprays different scents as a user eats a plain biscuit, superimposed with a picture of an appropriate flavoured biscuit. This combination of visual and olfactory feedback fools the user’s sense of taste so they believe they are in fact eating a flavoured biscuit (Horsey, 2010).
Each of the examples above uses a different sort of hardware (smartphone, cyberglove, headset etc.) to project data onto a real object or space the user is interacting with. Each also uses some form of software to provide meaningful information. Both hardware and software developers are developing products to bring augmented reality into mainstream usage and some of the front-runners are detailed next.
Augmented Reality Providers and Solutions
Augmented reality providers currently have three types of offerings: hardware, software, and expertise. Some focus on just one piece of the puzzle but the most talked about offerings come from those that can integrate all three into a complete package.Hardware solutions
Long known for its software expertise and ability to appeal to a mass audience, Google is looking to change the face of mobile computing by using a pair of high-tech glasses as an interface (rather than a smartphone or an iPad). The future promise is that the glasses will be able to project augmented reality elements into the visual display. The company is promising to release ‘explorer edition’ glasses in early 2013 but these units are little more than a video capture device (Rivington, 2012). However, when Google do manage to produce a wearable always-connected form of mobile computing, their track record in software applications such as Google Maps and Google Search should ensure they can create content rich augmented reality applications that will enthral the masses. Sullivan (2012) imagines such applications would include hands-free gaming, access to the view from friends glasses, language translation, first-aid instruction, travel information, and nutrition facts.Software solutions
Phone maker Nokia is making the most of its hardware expertise in smart phone technology by releasing the City Lens application software designed to work with its Nokia Lumia phones. City Lens is an augmented reality browser that uses the GPS and compass embedded in its phones to highlight local places of interest and gather more information about them (Nokia City Lens Team, 2012).There are many similar types of application and the travel industry has embraced the technology. Wikitude SDK offers a development platform where augmented reality can be added into existing applications. The way the information is displayed (via augmented reality) is convenient but this information is also readily available online so is unlikely to drive uptake.
One exciting avenue of exploration is facial recognition software that will allow users to access the name and social media sites of people (be they friends of strangers) their phone is pointed at. Whilst this raises many privacy issues, facial recognition may be the “killer app” that makes augmented reality a must-have for the masses (Perez, 2009).
Expertise
Combining the right sort of hardware with the right sort of software to create a unique user experience requires a specific sort of expertise. Companies such as Total Immersion aim to connect companies with customers through the use of augmented reality applications. The focus is on the advertising and media markets and the ways augmented reality can be used to engage an audience.Augmented reality is a marketer’s dream – a chance to really affect people in the course of their everyday lives. “The possibilities are truly endless. Engaging with animation and AR will lend itself to instant opt-in for contests, redirects to e-commerce, exploratory views of products, or even crossword puzzles without using a pen” (Rorick, 2012, para 14). That helpful information that augmented reality reveals may really be just another way to sell more stuff.
Issues with Augmented Reality
Technical Issues with Augmented Reality
Kruijff, Swan II, and Feiner (n.d.) provide an intensive list of perceptual issues in Augmented Reality related to the environment, capturing, augmentation, display, and individual user differences. More specifically, they are detailed as follows:- Environmental issues are caused by the interaction between the environment and the augmentations. This includes problems with visibility related to the screen, such as size, reflections, and brightness, or colour and texture, as well as patterns that interfere. Depth ordering issues are the inability to correctly match the overlaid information to the real world, scene distortions, object relationships, augmentation identification, and surface perception.
- Capturing issues relate to visual elements between the real world view matched with the layered digital imaging, “digitizing the environment in video see-through systems, and optical and illumination problems in both video see-through and optical see-through systems” (Kruijff, Swan II, and Feiner, n.d., p. 2.).
- Augmentation issues refer to the layout, design, and registration of augmentations referring to the “tracking accuracy dependant on the environment and distance of the objects being viewed: lower accuracy tracking may be acceptable for objects far away in large scale environments where offsets are less noticeable, while accurate augmentation of nearby objects is harder” (Kruijff, Swan II, and Feiner, n.d., p. 4.).
- Display device issues are the technical issues associated with the display device such as the camera quality, and pixel density.
- User issues are related to the user’s perception of the experience. For instance, some people would find it difficult to distinguish between real, augmented reality, and virtual reality.
Legal issues
- Easy access to information within proximal distance - Spatial Law is the set of legal issues associated with geospatial technology and the collection, use and transfer of location and other types of spatial data. Issues that spatial law addresses are: Privacy, Intellectual Property Rights, Data Quality/Liability, and National Security particularly in how and what is accessed when a person is within close proximity.
- Copyrights, trademarks, and patents - Gatto (n.d.) focusses on the patenting issues of integrating the location awareness/real-time data sensors with Augmented Reality application; contextual awareness, etc. This has been an on-going issue throughout internet usage also. The need for legal considerations must be addressed when voice activated search and respond applications as seen in this Project Glass video resemble other existing programs, such as Siri, or implement maps and global positioning system data so freely.
- Spam - when marketing opportunities arise, so does spam, and illusory advertising to trick people into buying things. But along with this trickery comes the reaping of data you share publicly over the internet with attractive offers hooking you into their schemes.
Ethical Issues
- Profiling - A person may walk around in the physical world as an unknown person, but in augmented reality they have a digital profile identified via facial recognition, and geo-location capabilities. This digital profile details personal information that either that person has made available, or others have provided online. The idea that others are able to make a profile of you online, other than voluntarily nominated by oneself is not only an ethical issue, but a social issue too.
- Physical safety - Imagine crossing the road and being hit by a car while in Augmented Reality. Extremely distracting the applications and then the music. Not all people can listen to music and cross the street safely, and not everyone can see a moving target coming towards them while reading a message through Augmented Reality. Imagine those trying to drive and operate one of these, or operating dangerous machinery.
Social Issues
- Profiling - With regard to the digital profiling, imagine walking into a meeting and being ostracized by a group of people because you have self-identified yourself with a religious/political association that they oppose.
- Intrusive – As identified in this Project Glass video i.e. Paul was located as 402ft from Strand Books; I’m not sure if all people would be comfortable with someone knowing their whereabouts all the time.
- An unhealthy technological dependence resulting in a lack of human social contact is at play (Dahiya, 2010). Currently, we can see the dependence on computers, internet, and other mobile digital devices in schools, and educational settings. It is plausible that we will be technologically awake connected to an online, or virtual world all the time. There is a simple solution to this, which is to take the equipment off, and engage in real conversation, real sports, and real life interaction. Batteries are known to fail, power is known to go down, and systems are known to crash. On the other hand human beings are known to talk, walk, eat, write, and function too.
Company Specifics
Contribution of Augmented Reality to existing Tourism strategies
Digital technologies have widely assisted museums, and art galleries to display collections, historical simulations, preservation, exploration, and much more. The accessibility and portability of the devices that controls Augmented Reality, means we can take it outside while trekking in a bush, walking down the streets of Paris, or skydiving across Auckland City. Augmented Reality’s contributions towards the tourism industry are favourable, and well documented as highlighted below.Banholzer (2010) reports on Emerging business models and commercialization opportunities of Augmented Reality technologies in the tourism industry. He contends that Augmented Reality has supported the tourism trade in the following ways:
- It makes traveling easier and facilitates a deeper, more informed type of travel.
- It has the potential to allow a tourist to truly connect with a destination.
- It reduces language barriers allowing translation more effective and faster.
- It allows users to venture into unknown/unseen places.
- Future plans see Augmented Reality offering personalized, mood, and location based information.
In addition, Fritz, Susperregui, and Linaza (2005) point out a number of ways that Augmented Reality can help assist existing tourism trades through the introduction of Augmented Reality tourist binoculars. These binoculars are called PRISMA, where the real scene is enriched by personalized multimedia information to enhance tourism experiences. Users can discover more about objects they see, obtaining more information through clicking on the screen, where a pop up menu appears offering choices, showing the proper augmented content. Information is entirely multimedia, including 2D pictures (maps, old photographs), texts, movie clips, or 3D models of existing and non-existing tourist spots.
Other identified methods in which Augmented Reality has already, and can assist tourism is as follows (Fritz, Susperregui, & Linaza, 2005):
- Augmented walks – Tourists are put in their real surroundings and can view the real physical setting and 3D reconstructions of monuments as it was in historical times. Screens receive the real scene through the camera, and then layer the 3D models over the top. As tourists walk through the real scene they are able to blend the two scenes together.
- Ename 974 - Similar to augmented walks, a Timeframe technology is used to generate 3D simulations, real scenes, and reconstructions of monuments of archaeological locations and presents these environments to different profiles of users. This is viewed in a kiosk to protect the tourist and system.
- Archeoguide – An IST European project is currently working on an Augmented Reality experience where tourists are able to walk through a cultural environment viewing the real world, while visualising 3D reconstructions of monuments with a multimedia virtual guide assisting them.
- Similar to the PRISMA, several national parks in America are using Augmented Reality stations to observe archaeological sites on faraway cliffs and hard to reach locations. It is a telescopic device that overlays virtual displays, animations, and other archaeological information.
Strengths to Introducing Augmented Reality
Currently in tourism anywhere in the world, people are able to book flights to any destination via the internet. The internet also provides information for the traveller about the places they are likely to visit and gives options on things to see and do.Take Wellington.com for example, on their website they have everything from accommodation, events, sights and activities, bars and restaurants, shopping, visitor information and conventions. You are able to book accommodation from their website. If you want to travel into Wellington from another part of the country you can make an airline booking via Air New Zealand from this website. They also have maps and guides and you can build an itinerary for your stay in Wellington. Further down at the bottom of the page they have broken down main headings into sub-headings to make it easier for the tourist who wants to visit Wellington, on the type of accommodation they want, the type of event they may be interested in and much, much more, including breaking down the shopping from general to women’s wear, menswear, footwear, jewellery and accessories, kids’ stuff, health and beauty, gifts and home ware and services. This helps the tourist to Wellington to get an in-depth analysis about the type of shopping available and many other things also.
As with anyone who is in business tourism is also on Facebook, have a twitter account and you can sign up to their newsletters to receive information about that particular place on a regular basis. What Wellington.Com has also done is linked up to Trip Advisor, which also links up to Facebook and people who have travelled to Wellington before have made comments about the city. This enables would be travellers to see what other people thought about Wellington before embarking on their journey. As they say the best form of advertising is word of mouth.
Putting all of this into perspective with Project Glass from Google and augmented reality would mean that visitors or locals to Wellington or any other destination in the world can use the glasses to locate places using the GPS system on board. They would be able to take photos of what they see without having to take out a camera or their cell phones. Currently there is an app for an IPhone using augmented reality, pointing technology, GPS and interactive maps. Scottish Tourism launched this app in December 2011. This app is based on Landmark Press’s Welcome to Scotland website.
Landmark Press publishes information for Scotland’s tourism industry, including 2.2 million annual ‘Welcome to Scotland’ and ‘Welcome to Edinburgh’ guides in 6 different languages. They also supply ‘Welcome’ bedroom folders to virtually all the hotels and other ‘accommodations’ in Scotland. Drawing on that pool of over 9,500 hotels, guest houses, B&Bs, holiday cottages, self-catering apartments, hostels and caravan and camping parks, the app enables visitors to search for and book accommodation by destination.
Project Director, Alan Bird, says the app also has Special Offers from many of Scotland’s top attractions, activity operators, accommodation providers and restaurants, meaning that users will be able recoup the £2.99 purchase price many times over.
And that’s not the only way they can save money, he says:
“One of the Welcome to Scotland app’s most valuable and unusual features is that all of its content and main map are preloaded. This means visitors can access the information wherever they are with no internet connection being required. As a Scottish travel app, this makes a huge difference because many of our most beautiful rural locations suffer from poor broadband and mobile data coverage. It also enables visitors from abroad to use the app on the go without risking expensive roaming charges.” (Alistair McKenzie, December 5, 2011)
For any tourism operator anywhere in the world the glasses would be a new and innovative way of providing guided tours of anything. Imagine walking up to the Empire State Building in New York with these glasses and know the history without stepping into the lobby. Riding the elevator up to the 86th floor which is the lookout point of the building and by wearing these glasses you would be able to see other landmarks of New York and know their history, and know which businesses currently occupy space in them. How about walking into the Sistine Chapel and looking up at the ceiling that Michelangelo painted and learning how he painted it, how long it took and what it means without having a tour guide explaining all of this to you. You would be able to save this information and bring it up when you later got back home to show your family and friends.
These are only possible if the tourism companies where using the Cloud to store this information on and it was retrievable via cloud to the glasses. This would be the next step that all major tourism companies and operators would need to take to enable these glasses to work.
Risks to introducing Augmented Reality
The down side of using the Project Glass in a tourist aspect would be that users would get plugins and advertisements from local businesses nearby.“The portability and wearable as glass is the key aspect of Google project Glass that compelling the industry to consider it as a booster for the tourism across the globe.” (PolyEyes Staff, 9 April 2012)
The other dangers to these glasses are the health and safety of the person wearing them, as Jani Seneviratine from Optimum7.com points out below: (J Seneviratne, April 5, 2012)
- How fragile are they and will be they susceptible to breakages? They could possible drop of your face as glasses can become loose.
- How easily can they be lost? What if you put them down somewhere when having a conversation with someone and don’t remember where you left them?
- You can easily be more distracted while wearing these glasses as your vision is focussed on what you see in the glasses. Today we hear of people being run over by buses and cars because they are listening to their mp3 players and not paying attention to what is happening around them. The same could be said for the Glasses.
- Currently they have not made anything for those people who wear prescription glasses. How would these fit over the top of normal glasses that people wear every day?
- How long is the battery life on these glasses and where is the battery on the device? The battery on a Smartphone constantly needs to be charged, will the battery on Google Glasses need to be charged as often?
- Will advertiser cramp our vision while we are walking down the road?
- Privacy issues are also of a concern, especially if you open up someone’s profile.
As for wearing them with prescription glasses, Google Glasses can be worn with them. In the future Google is looking at making augmented reality contact lenses for those who wear contacts and even for those who don’t wear contacts.
If driving, these glasses from a safety point of view probably shouldn’t be worn, again because they would distract the driver from their driving and make them crash.
The user needs to be aware of what is being opened up in front of their eyes, because there may be things that pop up that the user does not need to know about.
For a business, instead of using Google Glasses, a better way would be to create an app for both the Apple and Android markets and have augmented reality available on Smartphones. This way the user is aware of their surroundings more than they would be if they wore the glasses.
Augmented reality is a great thing to have especially when travelling as you can capture rare moments you wouldn’t normally. You would be able to have better access to information about public transport systems, the weather, accommodation and flight details for your destination rather having to rely on using the internet and having to go to an internet café or information centre to get more information.
You wouldn’t need to carry maps with you because it has GPS services as part this technology, so you would be able to get to a place without having to read maps. After being out all day, you would be able to locate a place to have dinner in the evenings and wouldn’t have to look around for a place to eat.
Final Recommendations
Feasibility of adopting Augmented Reality
Augmented Reality is set to revolutionize the mobile user experience as did gesture and touch (multi-modal interaction) in mobile phones. This will redefine the mobile user experience for the next generation making mobile search invisible and reduce search effort for users. Augmented Reality has a long history of usability research, analysis and experimentation and therefore has a solid history as an interface technique. (Hopkins, 2010)Augmented reality is far more wildly futuristic than the iPad. It is a set of technologies that are coming together right about now. One day we’ll all be walking around with Augmented Reality glasses on, or wearing Augmented Reality contacts. (Abrash, 2012).
Computer World (“Computer World,” 2012) says that according to ABI Research, the future of augmented reality will be in interactive print media, mobile shopping and children’s education.
The ABI study also predicts that the future in augmented reality will not be in Smart Phones but in eyewear based interfaces, with Google's Project Glass being a prime example. The future of the technology is going to depend on the development of a light weight and lasting source of battery power that will keep the device powered and also the ability to have constant wireless connectivity to broadband internet.
Companies providing software platforms for the development of Augmented Reality applications stand to reap large rewards from the evolution of Augmented Reality according to the ABI Study.
An article by Mobile Commerce Press (“Mobile Commerce Press,” 2012) says that Augmented Reality to date has mainly been used to bring gimmicky and novelty applications to users, however with the advancement of the technology the time is coming when users will be exposed to more practical applications of Augmented Reality – and these applications will not be exclusive to Smart Phones.
There is the problem that Augmented Reality may simply provide too much information. There is also the risk that on top of this, users will also be bombarded with unwanted information such as advertising. For example, knowing where McDonald’s or Starbucks is in Paris or Rome might not interest users as much as “off the beaten track information” that you might seek out in travel experiences. (Hopkins, 2010). No doubt, developers will also release ad blocking applications to help keep unwanted advertising at bay, as with internet browsing software available now.
Augmented Reality is a relatively new concept and some users may have difficulty accepting the new form of browsing. Older generations who are not tech-savvy will have trouble keeping up with the newer technology.
GPS, which is an essential element for Augmented Reality by providers such as Project Glass, has its limitations, in that it is only accurate to within 9 metres and doesn't work as well indoors. An indoor positioning system (IPS), a network of devices used to wirelessly locate objects or people inside a building, can be used alongside GPS to solve this issue. Instead of using satellites, an IPS relies on nearby anchors making use of various optical, radio or even acoustic technologies. Google’s approach tracks you via WiFi — it knows where the WiFi hotspots are in a given building, and through signal strength triangulation it can roughly work out where you are. (Anthony, 2012)
The mobile internet connection that is needed to download the layered information may have issues with speed and congestion making the Augmented Reality experience less seamless. Faster mobile broadband connection speeds such as 3G, and the new 4G currently being rolled out, overcome these issues and Augmented Reality will become more of a reality as technology improves.
Hardware performance can also be an inhibitor. Augmented Reality relies on a mobile computer that will need to be small and easy to wear. As technology improves, more powerful mobile platforms are entering the market place. This can be seen in the latest Smart Phones currently being released. This new hardware will increase the performance of projecting the layered images.