Inside AR Conference

We built this city on A and R

written by Jack Dashwood in Miscellaneous on 01. Nov '11

Augmented City, 
augmented reality, 
augmented reality software, 
free software, 
industrial augmented reality, 
Mobile Augmented Reality, 
mobile sdk, 
press release 



(Apologies for the reference, but it had to be done)

While everyone here in the lovely city of San Francisco was preparing Halloween costumes, metaio was at the International Symposium on Mixed and Augmented Reality presenting research that will not only enable an actual Augmented City, but will lead to more natural, more subtle augmentations of the physical world.



metaio has made substantial improvements in mobile optical tracking and recognition, having implemented both 3D object recognition and Gravity-Aligned feature descriptors (GAFD). Through complex algorithms and sensor-fusion, we are able to align the field of vision of the smartphone with gravity. That means no matter where or how the user is positioned, his or her smartphone will always understand “up” and “down” as constants instead of relative positions. Looking at the photo above, you can see that natural feature tracking fell short in trying to describe surfaces buildings- all the windows look the same and there is little variability or contrast in the overall facade. Before GAFD, identifying or isolating only one or two of those windows would have been nearly impossible- defining gravity as a constant however allows the tracking to easily identify each individual window.  Check out the video below for a visual demonstration- you’ll notice that the tracking is faster, smoother, more robust, and above all more natural. Note also that we can align individual elements of a given 3D model with gravity, allowing 3D models and animations to behave more realistically when placed in the real world.



metaio also debuted 3D object recognition and tracking for the first time on a mobile device running on a consumer application: junaio. Not a specifically engineered research platform or prototype, but an iOS and Android application free to download and more importantly, free to develop. And in its first appearance, 3D tracking on junaio won the ISMAR Tracking Competition.

3D object tracking is the obvious next step for mobile AR- though printed 2D images provide a wealth of material for launching AR experiences, being able to track actual real-world objects will open up a whole world of opportunities for placing useful, engaging, and interactive content in the physical world. Combined with GAFD, objects like buildings, automobile engines, complex industrial prototypes, groceries, billboards, and public transportation vehicles become the canvases for the digital world.


3D Reconstruction

In order to have real-time, natural and subtle augmentations of the real world it is imperative that we able to understand and digitally reconstruct our immediate environments. For instance, a realistic digital or virtual piece of furniture placed into a physical room needs to behave as though it is real. You would expect an object in front of it to obscure it, and vice versa. Until recently, the placing of virtual objects into the real world in real-time has been restricted to the foreground of the user’s perspective. metaio however presented at ISMAR research that show the instantaneous online, real-time learning of 3D environments in order to realistically display or remove virtual objects from the physical world. This kind of technology will eventually allow prospective homeowners to furnish an entire house or apartment the first time they step across the threshold, or allow for the construction of truly immersive, realistic AR gaming environments.  Watch the video below to see it in action- note how the camera actually learns its surroundings in real-time, and within seconds adjusts to make a given virtual object or animation appear realistically in reference to surrounding real world objects.



If you build it…

metaio is well aware that in order to see stellar use cases, engaging content, and disruptive interfaces that this kind of technology needs to be in the hands of both creators and users. That’s why all of this research will soon be available through junaio (now, assuredly, the world’s most advanced AR browser) and the advanced metaio Mobile SDK (which if you haven’t heard has a free version coming soon).  We here at metaio believe that Augmented Reality as a whole is far too vast, multi-faceted and diverse to be unified under a sole “Killer App”. That’s why we’re giving people the tools to investigate, experiment, and explore all aspects of this wonderful technology to expand on all potential applications of AR.

For more information, see the press release, as well as some of the articles written about our research in the past few days:

Singularity Hub, “The Latest Updates to Augmented Reality: 3D and Gravity”

TechCrunch, “Metaio Adds Gravity To Their Augmented Reality Platform”


Jack Dashwood
Jack knew that he was interested in Augmented Reality ever since he learned how to read at age 25. Fast forward and Jack now works at the center of the universe when it comes to AR: the offices of Metaio. How did he wind up here? We don’t know, but we have yet to find a legitimate excuse to get rid of him. In the meantime we let him work on the blog and run the Metaio US PR operations. When not talking and writing about AR technology, he can usually be found augmenting his reality by more traditional means down the local pub.

Related posts


  1. Great post Trak! Entertaining and liked the video references! :-)

  2. Bernard François

    And all we need to do now to get these awesome experiences on-the-go is to wait until smartphones featuring a depth camera are released…

    …or until someone implements 3D object recognition based on moving images and perspective distortion. That would be awesome!

    In the mean time, here at PreviewLabs – a company specialized in rapid game prototyping – we’ve been playing around with Augmented Reality as well; check out our blog post featuring an augmented reality darts board video, running on a Google Nexus One Android device.

  3. Pingback: junaio 3.1, junaio Plugin now rocking on Android |

  4. Pingback: insideAR 2011: Peter Meier Opening Day Keynote « Augmented Reality Blog

  5. Pingback: My most important Twitter Messages #11 - Flash, Programming, Interaction | der hess

  6. Pingback: For the Weekend: Connecting the Dots with junaio | junaio Blog

Leave a Reply

Your email address will not be published. Required fields are marked *