How Does AR Work?

diagramThere are many types of hardware AR, but all the software works in two stages—tracking and reconstructing.[i] Tracking is how AR devices recognize what they are looking at and the reconstruction phase brings up a secondary tier of information. First, the camera tracks a type of trigger point, or rather an optical indicator or fiducial marker, a fixed basis of comparison.[ii]  Similar to a bar code or a QR code, the reader scans the image to bring up more information that is electronically imbedded. With AR, the trigger image can be any image and is not restricted to a bar code or to a QR code—it can be buildings, objects, people, or any optical marker. This tracking, sensing and recognizing of images can be done in one of two ways—feature based or model based.[iii] In a feature-based tracking system, the camera looks for connections between a feature or outline of a two-dimensional (2-D) image, such as a photograph, and its relationship to the three-dimensional (3-D) world. The camera imports data that it senses, then a computer uses a complex algorithm to let the camera “read” a photograph. Then it “looks” for the subject of the photograph in the natural 3-D world recognizing the real 3-D image as the trigger image to move into the reconstruction phase.[iv]  In model-based tracking, a model is made in a 3-D computer modeling program such as CAD[v], or Google SketchUp[vi] that is then used for tracking the 3-D trigger image.

clothes

Once it detects the target and its edges, it then enters the reconstruction phase to return the coordinates allowing the computer to pull up a digital overlay.[vii] For example it will soon be possible to track one’s own movements to try on clothes digitally. One might find clothes they like online. They would then turn on their webcam, look into the screen and see themselves wearing the clothes and how the clothes might fit without ever needing to go to the store to try on the merchandise.[viii]

[i] Wagner D, Schmalstieg D handheld augmented reality displays. Graz University of Technology, Austria

[ii] Julie Carmigniani ; Borko Furht ; Marco Anisetti ; Paolo Ceravolo ; Ernesto Damiani ; Misa Ivkovic , “Augmented Reality Technologies, Systems and Applications,” Multimedia Tools and Applications, 51, no. 1 (2011): 341-377,

[iii] Arvanitis TN, Petrou A, Knight JF, Savas S, Sotiriou S, Gargalakos M, Gialouri E (2009) Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal and Ubiquitous Computing 13(3):243–250

[iv] Ibid.

[vii] Julie Carmigniani ; Borko Furht ; Marco Anisetti ; Paolo Ceravolo ; Ernesto Damiani ; Misa Ivkovic , “Augmented Reality Technologies, Systems and Applications,” Multimedia Tools and Applications, 51, no. 1 (2011): 342,

[viii] Firsth, Niall. 2013. “Jaron Lanier.” New Scientist 218 no. 2922:21 Science Reference Center EBSCOhost (accessed October 22, 2013).