Layar is a young company. I’ve joined the company as CTO only 4 months ago. So the people in the world of Augmented Reality are entirely new to me and everyone at ISMAR was a new face. It’s funny to think that after 3 days of ISMAR I already feel part of the club! It’s a small world, so running around the Marriott hotel with a Layar badge quickly got me acquainted with almost everyone at ISMAR. Everyone knew Layar, many knew me by name and wanted to talk with me, showing how much the new startups of 2009 like Layar have captured the minds of people and given a new impulse to Augmented Reality as a whole. And many presentations started by mentioning Wikitude and Layar as the companies that created a paradigm shift in AR for 2009, moving it in the eyes of the general public away from the marker-based 3D football players that you could see last year.
The Monday workshop organized by Christine Perey and Robert Rice had a good turnout. The major upcoming or established AR companies of this year were present: Wikitude, Metaio, ACrossair and Nokia presented their plans or visions.
- Peter Meier of Metaio announced a mobile SDK which allows app developers to integrate an AR view in their game. The API will take POIs in a XML format and will also support showing highscores in an AR view. Most promising is the announcement of 3D markerless tracking. The first release of Junaio, an iPhone app letting users place 3D content in the real world and share it with their friends, will not have this technology, but Metaio promised to add it soon.
- Markus Tripp of Mobilizy announced the upcoming API allowing developers to host their own POI content, similar to the Layar approach. With that, they will support ARML, an extension to KML, to describe the POIs. It will be possible to use Wikitude to enter the URL to fetch the POIs, bookmark it inside of Wikitude or search for new content providers (search all ARML sites on the web).
- David Murphy of Nokia presented the possibilities offered by Symbian for creating your own AR apps. It supports OpenGL but no 3D hardware acceleration, has a good JSON parser (a coincidence David Murphy mentioned this as Layar uses JSON?) and supports position and compass well. Maybe more interesting is the fact that it is increasingly becoming possible to use python for development on Nokia devices, making the learning curve on Symbian much less steep.
- Chetan Damani of Acrossair also announced they will be releasing an open development platform for their browser in November. They will support 3 types of AR apps: Simple XML files for uploading content to the Acrossair platform and viewing in the Acrossair browser, an advanced API for hosting the content on your own server (the Layar model) and full integration for app developers who want the Acrossair view in their own application.
We then had some good discussions on the future of our apps but really weren’t able to agree on a 2012 roadmap for this industry: It’s just too early now and we’re all investigating our own strategies and roadmaps right now.
It was great fun to sit in one room with Markus, Peter and Chetan and discuss possibilities for standardizing some of the stuff we offer. Of course content developers would love to be able to offer their content in a single way across the various browser platforms, like on the web (well, except for some small browser incompatibilities). How can we get there and not have the same crucial flaws the early web browsers had, e.g. with CSS, forcing content developers to customize for each and every browser? The easiest part is the metadata of the POIs where an XML-based structure like KML, used by Wikitude with ARML seems to make sense. But AR browsers are more than a list of POIs. We need a request structure (XMPP, HTTP?), objects in 2d and 3d (.obj? collada? VRML?), user interaction, image recognition and feature recognition to describe the AR world. And all of these would need standardization in order to guarantee interoperability. So whilst acknowledging we will need to standardize, we had to admit that the timing isn’t right yet and standardization would slow down our innovation drift rather than help us at this stage.
What I take away on this first day is that the competition for Layar will be tough, we’re all creating open platforms for AR content and we’re all adding 3D to our browser. On the other hand the focus of each browser is different: graphics and design, user experience, gaming aspects, user-generated content and a solid developer community are various aspects where each of us has its own strengths and weaknesses. Our company strategies are probably very different.
The evening spent with Tish Shute and later Peter Meier and part of his Metaio team was memorable, everybody is extremely open at sharing ideas and driving innovation forward.
My focus the next day was mainly on tracking and tracing. The presentations by Oxford University and TU Graz made my day, giving clearly the state of the art regarding tracking and tracing right now:
- Marker-based tracing has reached amazing stability on mobile devices, anyone would be able to use it and not get frustrated by objects jumping around or losing the marker. This is important, because even on a PC, I’ve often experience that using it myself led to a jumpy experience with the nice 3D object disappearing every so often due to my movements of the marker.
- Feature-based image recognition is getting there, although memory limitations on mobile devices will require a limited set of known images to recognize from. Still, we as browser makers will need to think of good user interfaces since the recognition rate is lower than 100% for not too simple images, think more like 80%.
- Markerless tracking on mobile phones was showcased with 2 quite spectacular demonstrations:
- One from Dieter Schmalstieg of the TU Graz, where the software was able to recognize the outlines of the area using the buildings around the user. Mapping the shape of the area (street block) to GIS data in order to accurately determine the location. Luckily Graz offers these typical European old-town layouts where none of the blocks has the same shape. One wonders how this would be done in a US city, where blocks are the same in the entire city.
- the other demo was by George Klein who managed to get PTAM to run on a mobile and track the objects on a desk. Then letting a 3D model of a flower bed grow out of the cover of a book, a video of it you might have seen on YouTube. Neat, but if one user does it on his phone, how will another device know exactly which book was used to ‘plant’ the flowers so someone else can relive the same scene?
- Also here, user interfaces to help users master this technology will be crucial, as acquiring data points for tracking and recognize features takes time and requires moving the camera around.
Later that evening over thai dinner, I had good discussions about future business models in AR with Tish, Dylan Philips and Joe Ludwig (‘one bus stop away’ layer for Seattle
). Can we make business out of the current mobile AR apps that use the ‘magic lens’ or will we have to wait for the glasses and more wearable devices to come on the market to see AR reach the masses? Will consumers feel more comfortable with AR looking through a pair of glasses or will they eventually adopt the ‘hold your phone in front of you’ pose? And what’s the advantage of finding things around you looking through the ‘magic lens’ rather than seeing them on a map? I think Layar has already proven that business can be done now, with a number of our developers making money out of Layar. Also the AR view certainly appeals to many users as being more intuitive than the traditional map view.
Some final take aways for me from ISMAR 09:
- Even though it sometimes looked like Wikitude and Layar were the AR heroes of today, the stuff that makes our hearts beat faster still comes from the great research by scientists at universities like TU Graz, Oxford, Georgia Tech and Canterbury (NZ). They are laying the foundations for AR in the future, not Layar with its simple ‘not even real AR’ browser :-)
- Using current AR browsers indoors is usually disappointing, lacking the positioning of GPS. But this is where image recognition and feature recognition will be easier to apply, with well known floor plans and objects. Think for example of musea, paintings and statues.
- Marker-based tracking (be it with textures or code-based markers) still offers a lot of value when brought into the mobile world. It’s so much easier to use than with PC/webcam setup. István Barakonyi from Imagination had very cool demos of this technology: The book on the little worm that eats his way through all the fruit and becomes a beautiful butterfly that we all read when we were children got new life with AR.