The dark side of mobile Augmented Reality

Augmented Reality, is the real-time mixing of computer generated and real-world information.  We experience it, each time we see computer generated information during live broadcasts of news and sporting events.

As smartphones become more popular, the use of mobile Augmented Reality (AR) applications has increased.  Out on the town and looking for a restaurant?  Pull up Google Maps on your phone and it shows your current GPS location.  If you’re unfamiliar with the area, it will show an aerial view so you can find your bearings.  It also points out all the restaurants near you.  And while walking to the restaurant, Google shows you  street level views of intersections along the way, with spoken directions telling you which way to turn.

Virtual information can be geotagged to physical locations.

Other AR applications like Layar and Wikitude World Browser will “overlay” whatever your phone’s camera sees, with information content collected from various Internet sources.  If you walk down the street pointing your camera in different directions, you can see information collected from Google, Flickr, Panoramio, Twitter, Yahoo, YouTube and other sources; that have been geotagged to the spot or direction the camera is pointed at.

“Geotags” are nothing more than GPS coordinates that associate pictures, videos, documents, notes, tweets, etc. with a physical location.  Geotagging is similar to what happens when a picture is date-stamped by a camera.

Many AR applications rely heavily on user communities to provide content that others might find useful or interesting.  In the Google Maps example, you can read reviews from other people for each restaurant; and write your own, if you desire.  Reviews and ratings can also be found for other businesses and points of interest (POI) that appear in Google Maps.  With Google Maps, people can even create and share maps with each other; complete with notes and comments on each POI.

But beyond the promise of making our lives better, there lurks a “dark side” to Augmented Reality.  One that is not being publicly discussed, but should be.  Because it affects us on so many personal, civil, criminal, and national security levels.

When Google Earth was introduced in 2004, many governments expressed (and continue to express) alarm over the inclusion of “sight sensitive” locations and objects in the Google Earth database.  When something is “sight sensitive,” it means the mere act of letting your enemy see it, will make you more vulnerable.  With the introduction of Google Street View in 2007, individuals and privacy advocates were alarmed at the details that could be seen in Street View images.

Google has the ability to easily control, restrict and censor information in Google Earth and Street View; because the information is static and does not change very often.  But with AR applications such as Wikitude and Layar, content is constantly being delivered from several independent sources that cannot be easily controlled, by a single authority.  Additionally, the Internet can also contain private and anonymous LightRod AR servers; each delivering their own specialized AR content.

The ability to freely associate virtual information with physical locations, and then anonymously share it with others, creates opportunities for AR to be used in unintended, illegal and sinister ways.  Imagine the possibilities, if “free speech” included the right for anyone to plaster buildings with posters and signs, containing content of their own choosing.  Now imagine if everyone except you knew what the signs and posters said.  Or if you were unaware that such material was stuck to the side of your home or business.  That is the threat posed by mobile AR applications.  And it is not going to get better, anytime soon.

Private citizens might have geotagged information already attached to their homes; by their own children, spiteful neighbors, cyber-bullies, paparazzi, stalkers, criminal offenders, and vigilantes.  Many businesses don’t know what kinds of geotagged information are attached to them.  Gangs, vice and other criminal elements, now have another tool at their disposal.  And the Department of Homeland Security and Department of Defense have one more source of intelligence to worry about.

Please donate and support Information Assurance Threat Research



The risk at the personal level, is extremely high.  Many smartphones can automatically geotag pictures taken with the phone’s camera.  And that capability is usually turned on by default.  I conducted an experiment, in which I rode through several residential neighborhoods.  Using Wikitude and Layar, while pointing the phone’s camera out the car window, I was amazed at the number of geotagged Flickr images I found.  A large percentage of these pictures would be considered “sexting” material, or even pornographic.  Obviously, those who uploaded the pictures didn’t knowingly understand the images were being geotagged by their phone.  And because of the geotags, what was intended as anonymous fun is now an invitation for sexual predators.

The risk for businesses, is medium.  It is just another customer feedback and business intelligence tool.  But it does require monitoring.  Otherwise what you don’t know, can hurt you.  As a marketing tool, the opportunities are huge.  Companies such as Best Buy, Starbucks and Walmart have already geotagged their store locations in various AR spaces.

Map showing DroidSprayed locations.

Gangs can use AR to mark their turf.

For law enforcement agencies, the risk is medium.  As crime abatement programs succeed in the real-world, gangs will begin geotagging their turf with AR applications like DroidSpray and Virtual Graffiti.  The availability of subsidized and prepaid smartphones will increase the speed at which this occurs.  In the not too distant future, services similar to Craigslist might begin geotagging personal ads.  Finding an available prostitute may be as easy as driving through random neighborhoods with a smartphone.  Priority should be given to the development of “AR aware” forensic methods and tools.

Additionally, the judicial system needs to start thinking about the legal admissibility of Augmented Reality evidence.  Is there a difference between AR evidence discovered by an officer and a private citizen?  What about the authenticity of AR evidence?  And to what extent is geotagged information free speech or a violation of someone’s privacy?

Geotagging of strategic POI.

And for government intelligence and defense agencies, the risk is extremely high.  The numerous AR spaces (Wikitude, Google Maps, Layar, etc.) must be monitored to ensure that terrorists have not geotagged potential targets. Agencies must take into consideration the existence of geotagged information, and its relationship to other forms of intelligence.  And very soon, “AR reconnaissance” may become part of military operations.

Like the Internet, AR is a technology that cannot be ignored; because it has the ability to affect every aspect of our daily lives.  Even if you never use a smartphone or AR application, the very existence of AR information can still affect you.

Please donate and support Information Assurance Threat Research



Spread the word!