What is Augmented Reality?

The term augmented reality (AR) was first coined in 1990 by former Boeing researcher Tom Caudell, who used it to describe ways in which digital information could be overlaid in real time with the visual information we are used to seeing in the real world. (Heads-up displays in aircraft were an early outgrowth of the technology.) While the capability to deliver that sort of augmented reality experience has been around for decades, it has up to recently always required a very expensive customized system, or special equipment. Advances in mobile devices as well as in the different technologies that combine the real world with virtual information have led to augmented reality applications that are as near to hand as any other application on a laptop or a smart phone.

Emerging augmented reality tools to date have begun to overlay marketing, amusement, or location-based information via heads-up displays or real-time video, and new applications continue to appear as the technology becomes more popular. As they have, augmented reality is now poised to enter the mainstream in the consumer sector. Learning applications, such as the ability to overlay information over a video image of an historical site, or an artifact in a museum are not far behind.

INSTRUCTIONS: Enter your responses to the questions below. This is most easily done by moving your cursor to the end of the last item and pressing RETURN to create a new bullet point. Please include URLs whenever you can (full URLs will automatically be turned into hyperlinks; please type them out rather than using the linking tools in the toolbar).

Please "sign" your contributions by marking with the code of 4 tildes (~) in a row so that we can follow up with you if we need additional information or leads to examples- this produces a signature when the page is updated, like this: - alan alan Jan 27, 2010

(1) How might this technology be relevant to the museums you know best?

  • I can conceive of augmented reality, coupled with location-based services, cloud computing, learning objects, collaborative environments, and haptic icon devices being the way one experiences the museum environment as a holistic experience along with the "real" collections on exhibit.- david.dean david.dean May 1, 2010
  • Augmented reality can impose “layers” of information on what the visitor sees and hears while onsite. This could be done using the visitor’s own smart device, or a loaner. There are many social networking applications (Yelp, layar) that do this now, and these rely on the visitor looking at the world through the small screen on the device. In some situations a device isn’t even needed http://www.youtube.com/watch?v=JSB6TN0jZZ0 , just a series of symbols carried by the visitor and recognized by the “smart exhibit” with it’s own display system. Esquire Magazine's AR edition http://www.youtube.com/watch?v=wp2z36kKn0s&feature=channel The relationship could also be turned around. Visitors could collect AR symbols by taking a photo of an exhibit, then using the museums web site show the website the symbol for additional interpretive value. - mike.kelly mike.kelly May 1, 2010
  • I think the key here is using smartdevices to enhance interpretation by offering more, and more diverse, levels of interpretation; allowing for visitors to customize the types of additional information they would like to receive; and allowing visitors to extend their experience/relationship with the museum through linked personalized websites. - elizabeth.babcock elizabeth.babcock May 2, 2010 - ninmah ninmah May 5, 2010
  • I think augmented reality could be a powerful tool for historic sites, living history centers, and other museums that have inside/outside or place/larger environment foci--it can be most useful within a museum, but how about between the museum and other locations? - marsha.semmel marsha.semmel May 2, 2010
  • Initial implementations of augmented reality along with location- and direction-aware applications on handhelds hint at some exciting possibilities for museums. Imagine--on site AR could be used as a wayfinding tool to create a fun, dynamic, personal virtual guide helping a visitor craft the most engaging visit. Educational AR applications could lead scavenger hunts that help a visitor learn about the relationships between artworks. I also think this type of application will be important to museums as our culture becomes to expect information to be dynamically available in a manner that is very personal to each individual and his/her handheld. I already expect my iPhone to be my Tinkerbell by providing the pixie dust of relevant just-in-time information at any given moment. - liz.neely liz.neely May 2, 2010
  • For art museums, interpretation of objects beyond an object label usually takes place in a digital environment (whether audio tours, kiosks, online, or mobile devices). The ability with AR to call up information on demand while having a minimal footprint on the physical space certainly has potential for art museums. - allegra.burnette allegra.burnette May 4, 2010
  • As Chris McLaren from Tristan Interactive pointed out to me, audio is the earliest, the most pervasive, and still the most successful form of augmented reality in that it allows museums to layer the visitor's visual experience with audio content. The current generation of AR allows us to add text, images, and other media to that enhanced experience and is, to my mind, one of the most exciting areas of potential development in the mobile sphere in particular. Like social media, AR has the ability to radically reconfigure the visitor experience both on-site and beyond. Coupled with location-based services, AR will be an important tool in taking museums' collections and content beyond the institution's walls - seeding the world with serendipitous encounters that mobile device users can both seek out and stumble across wherever they are. - nancy.proctor nancy.proctor May 4, 2010nancy.proctor

(2) What themes are missing from the above description that you think are important?

  • The term "augmented-reality" is confusing to many people--and since the number of types of devices that can deliver this type of experience is proliferating, it is important to provide more concrete examples in the description of this trend. This section should address some of the operational challenges facing museums as they seek to implement this technology. Not all visitors will want to experience the museum in this way. So one question this raises is ROI--museums need to understand their visitor base and allocate resources accordingly, realizing however, that the audience for this type of experience is growing. - elizabeth.babcock elizabeth.babcock May 2, 2010

(3) What do you see as the potential impact of this technology on education and interpretation in museums?

  • My vision of augmented reality's use in museums begins as a visitor enters the facility, where he or she is issued a pair of glasses. Upon wearing the glasses, "heads up" information begins to flow into the visitor's experience. Information about current exhibitions, one's location within the building/site and in relationship to galleries, suggestions about touring approaches, questions to be explored, additional information or activities offered and a whole "cloud" of possibilities and options would be delivered to the visitor's senses. Perhaps coupled with haptic icons via a hand-held device, or even the visitor's own cell phone, choices could be offered to the visitor allowing him/her to interact with the museum/environment/staff to enhance and personalize the whole experience. Such augmented reality would allow for overlays of diagrams, information, video, etc., to be placed with and/or over exhibits themselves, so that, upon demand (through the hand-held choosing device) the visitor can explore, at his/her own pace and level of interest, the subject or related materials. Also, such a user interface would allow the visitor to potentially join a "group" of other visitors, regardless of their location within the building (or world for that matter), in discussing and exploring concepts inspired by the exhibitions they are viewing. Age-appropriate interpretation could be delivered to the glasses/hand-held combo that would allow children, teenagers, and adults to have different but relevant experiences.- david.dean david.dean May 1, 2010
  • The impact of such an immersive experience would make museum visits more personalized and flexible, and answer most of the leisure choice criteria described by Marilyn Hood (i.e., allowing social interaction, learning something new, participating actively, being comfortable, finding the experience to be worthwhile, and having the challenge of new experiences. Hood, Marilyn G. (1983) "Staying Away: Why People Choose Not to Visit Museums," Museum News, April: pp.50-7.). It would create and confirm the relevance of the materials offered for the visitor, based upon his or her own interests, educational level, and personal worldview. In short, by creating this kind of immersive environment/experience, the visitor would be able to create his or her own experience and thus match their own satisfaction expectations.- david.dean david.dean May 1, 2010
  • I can't profess to have much knowledge of the area but I'm excited about AR in a way that I wasn't about virtual reality, which had its own moment in the spotlight of popular awareness a few years back. I had a belated realisation that, although VR sounds the more extreme and radical, by replacing reality wholesale, it's actually less so (of course they're both broad bands upon a spectrum). It can never be part of mainstream daily life like AR will rapidly become to because it doesn't have that connection to what we're doing and where we are. AR's challenges are different to VR's: although it doesn't require a whole new reality to be created, the need to physically and psychologically locate the experience where the user is poses a fresh set of technical and design challenges. In terms of making it happen, though, there is a set of technologies around that can bootstrap/act as scaffolding, if only by doing something as straightforward as geolocating images and leaving an app like Layar and a smart-phone do the rest. Image recognition technology set to give us a another tool in the AR kit for museums.
    Hmm, just lost loads of my edit.
    One reason why AR looks set to take off in museums and elsewhere is the existence of suitable devices. AR needs to be where the user is, and that means mobile, and with ever increasing numbers of people carrying their own suitable device in the form of a phone this makes it much more possible. Of course the other great thing about phones, though, is that they're inherently personal. Having a single user, they contain, or can be made to contain, lots of the information that would be necessary to do what David suggests and personalise the experience. As personal networked devices, though, they're also social and may well be hooked into existing networks like Facebook and Twitter, which can provide the spine of a shared experience, again bootstrapping the aims of a museum that wanted to enhance the "sharedness" of a visit. Koven just introduced me to a superb service built on top of Twitter in just this way, SwiftFM, which uses Twitter merely as the means for socialising another application, a model that could be readily followed te create an AR app that let people visti as a shoal and share a more profound experience.- jeremy.ottevanger jeremy.ottevanger May 1, 2010
  • I agree with David and Jeremy about the flexibility of the AR interpretive potential as it may allow social networking based learning or it could just as well lend itself to a private personal experience. One thing that this technology suggests may be possible is a version of “Clippy the Paperclip” personal interpretive guide, (but perhaps more tastefully done). If I could choose my virtual guide to be the persona of Cardinal Scipione Borghese before entering the Galleria di Villa Borghese, imagine the different experience and insights I might have than if I chose the persona of Bernini! - mike.kelly mike.kelly May 1, 2010
  • The key advantage is the option for the visitor/patron to control his/her experience and the depth of their experience. - elizabeth.babcock elizabeth.babcock May 2, 2010
  • I am interested in the way that AR can enable a museum visitor to link the experiences in/with a single institution with the broader community, such as in creating customized "history trails" around a location/neighborhood; in a past life, I did this with women's history (pre-Google!) creating trails that linked women past and present with historic sites, outdoor sites, roads and highways, artifacts, and other museums and libraries.- marsha.semmel marsha.semmel May 2, 2010
  • I agree with the above, and to reiterate my comment to question 1: AR + LBS = incredible new tools for taking museums and their content to audiences wherever they are, increasing our ability to make museums' collections and work relevant to broader audiences and more completely integrated into their everyday lives. - nancy.proctor nancy.proctor May 4, 2010nancy.proctor - ninmah ninmah May 5, 2010

(4) Do you have or know of a project working in this area?

I know someone at Qualcomm who is doing this with museums in Balboa Park in San Diego.
And I am familiar with Chris Deede's work at Harvard, not with museums per se, but with schools and, I believe, some historic sites. I know of someone who works with the Virginia Dept of Education who is doing AR with some cultural organizations in conjunction with school curricula. - marsha.semmel marsha.semmel May 2, 2010
Some related tools http://www.upsidelearning.com/blog/index.php/2010/04/30/tools-for-developing-augmented-reality-applications/ - scott.sayre scott.sayre May 3, 2010

I know as a museum we are looking into ways of using AR and a recent example was used during a temporary exhibition, but as far as I could tell it was a simple interpretation being used as a marketing tool. There is however much activity involving our photography dept looking at representing 3D versions of our objects on mobile devices and online and there are several international projects looking at the technologies employed. So although there is not much to see at the moment I think it will not be long before we see AR creaping into museum interpretation both online and on mobile, both in the museum and away from it. If marketing and commercial get involved then we could be in for all sorts of reasons for its use. - eric.bates eric.bates May 4, 2010

Please share information about related projects in our Horizon Project sharing form.]