If you’ve downloaded Layar v6.2 for Android or iOS and have tried out some of the vision layers, you’ll probably have noticed that tracking was improved tremendously in this version. I’d like to shed a bit of light on what we’ve done technically to get this far.

So what is tracking in the first place? It’s the ability of the device to recognize and follow the object that is being viewed through the camera and determine its position in space. That way, if it’s augmented in a vision layer, the device knows where to draw the augmented content.

Making augments stick

Until v6.1, we were showing the live camera feed in the AR view with the augmented content on top of it. To do that, we’d take the camera frame from the live feed, do lots of calculations to determine the position of the tracked object and then draw the augmented content. That takes some processing time, so that in the end, the drawn content would always be lagging behind the camera images that you saw. 

We’ve changed that in v6.2: We’re now showing the camera frame that belongs to the calculation for the augmented content, so that the content now sticks much better to the object. You can see that by moving the object smoothly: There is a slight delay between your movement and the image you see, but the content really sticks to the object you move. Don’t move too fast, this will blur the image and the app will lose track of it.

Zooming into the augments

There’s another big improvement in 6.2: When a layer developer uploads a reference image (the image of the object that she wants to augment), we create a fingerprint of that image that is used to recognize and track the features of that object. Obviously you need a certain number of features inside the camera view to calculate a good estimate of the position of the object. If the user holds the camera close to the object, zooming into the object, there are less features that can be tracked and at some point we’d lose track of the object.

We’re now adding features dynamically as you zoom in, allowing the app to track an object much longer. This is really nice when the augments on a page are quite small and you need to get closer to read them. 

A cool example is the Eppo comic book, where you can view the original draft drawings on top of many of the pages. Well, you can now also zoom in really close, as shown in the screenshots below. Don’t zoom in too fast though, the app still needs some time to process the images.

Taming the ARM processors

A lot of the code we write for Layar is platform-independent. The entire vision algorithms are written in C++ and integrated into iOS using Objective-C++ and into Android using the JNI. This way we can write once and use the same code on both platforms. This has allowed us since Layar v6.0 to bring simultaneous updates for iOS and Android.

For Layar v6.2 we’ve decided to move one level down and write the processor intensive code directly in assembler using ARM NEON. The cool thing is that up to recently all iOS and Android devices run on ARM processors. Not all of them support NEON (like NVIDIA’s Tegra chips), so not all of them benefit of our new optimized code.

In our reference implementation, we’ve been able to decrease the processing time of certain critical operations by a factor of 8: For example, preparing a frame for display takes 2ms instead of 15ms when using the ARM NEON implementation. Obviously we don’t achieve this same speed enhancement in Layar itself, where there’s a lot more going on, but writing NEON-optimized code is proving a huge improvement.

Thanks to this optimization we were able to add support for streaming video as AR objects without significant loss in performance. We’ve decided to only support dual-core ARM processors, where different tasks like rendering video and analyzing the camera image to calculate the pose estimation can be distributed to different cores.

Now Intel has joined the club with its x86-based Atom processors. That’s a game changer and obviously our NEON-optimized code won’t run there. But we’ve been pleasantly surprised at the performance of the x86 chip. Even without optimization it runs our C++ code at higher speed than similar ARM chips.

We can conclude that with the arrival of Intel in the smartphone market, a new era of speed improvements has started which is great news for processor-hungry apps like Layar.

Dirk Groten, CTO

All credits to Michael Hofmann, Lawrence Lee, Ronald van der Lingen, Anatoliy Samara and Andrey Koval for these achievements.

Today in the App Store and Android Play Store you’ll find our latest update: Layar 6.2!

You may not notice any changes right away, but there is one significant addition. In this version of Layar, we’re introducing what we’re calling AR Video.

Essentially, now items around you that are normally augmented with Layar Vision content (magazines, newspapers, pamphlets, brochures and other print media) can now come alive with video right on top of them!

Imagine reading the newspaper on your way to work and watching as the box scores on the sports page turn into video highlights to catch you up on the game! With Layar 6.2 and AR Video, this is now possible.

Developers making Layar Vision content can now easily add videos to these experiences quickly and easily. Check out the video below to see a quick demo of the power of AR Video!

Click here to view the Dutch press release.

“Er is post!” is a quarterly magazine from the Dutch national postal service PostNL that is delivered to every household in The Netherlands - over 6 million homes in total. In the latest issue, the pages have been enhanced with Layar, allowing every home in Holland to interact with print media in a whole new way.

After installing the Layar app, readers can interact with pages in the magazine marked with the Layar logo. By viewing these pages through the Layar app, digital buttons will automatically appear with links to websites, Facebook likes, YouTube videos, app downloads and other actions.

Making traditional post advertising interactive is a nice bridge between the physical and online world, says PostNL Commercial Director Ger Jacobs. “With first-hand experience, we hope PostNL can better service its enterprise customers in the future by providing guidance in the use of old and new media.”

“’Er is post!’ is a great magazine with which to experience how paper can be made interactive,” says Layar CEO Raimo van der Klein. “We have worked closely together with PostNL to make it a wonderful edition. I am very curious to see how the readers will react to this innovative way of reading a magazine.”

Check out the video below to see for yourself and give it a try with the sample pages after the jump. If you live in The Netherlands, keep an eye on your mail for “Er is post!”

Read more »

In 2011, author, Martijn Arets, visited 20 inspiring companies around Europe to learn how to build a better brand. The stories and lessons he learned became the book Brand Expedition. 

What Martijn set out to do was to visit these companies and understand them better. This became reality when he quit his job and went on the road to interview CEO’s, founders and brand managers. He captured his findings on video and shared his experiences by blogging and tweeting while on the road. At each stop, he requested that each CEO ask the next CEO a question on video. To engage more with his growing audience, Martijn asked them to help pick out which company to visit next.  

While Arets set out to capture his expedition in a digital format, he found that a book was the best way to share the stories and lessons. The book he wrote based on his experiences was a great success in Holland, and he set about crowdsourcing an international edition.

To improve upon that success and tie his online content to the book, Arets and Layar teamed up. For the release of the international edition, Arets used Layar to enrich the book with video interviews of the CEO’s pan-company interviews. Each page enhanced with video content has the View with Layar logo, indicating to readers when the special videos are available.  

At Layar, we think it’s a cool book and a great way to use our platform. The videos fit nicely with the printed content of the book. It gives the book experience something extra, which keeps the reader more enaged and in tune with the “how to build a brand” message and lessons.

We are very curious how the readers will like it.

Watch the video below to see how it works.  You can buy the book here to experience for yourself.

Speakers Academy, a leading management agency that represents professional speakers across the globe, has embedded Layar into its biannual magazine and speaker catalogue.

In the new Spring 2012 issue, interviews and profiles of featured speakers are accompanied by “View with Layar” logos on the pages of the magazine. Scanning these pages displays bonus videos, giving readers a look at the speakers in action.

“It’s very hard to explain in the text what they can expect,” said Speakers Academy commercial director Rene Warmerdam of the difficulties of showcasing speakers in the print medium. “Now, when using Layar, [they] immediately get access toward the video that explains it all. It really, really helps.”

The magazine also features a full page Layar ad in the front which explains how readers can scan the pages to view additional content. The “View with Layar” logo is our way of signaling to readers that they should scan pages with Layar. You can learn all about the logo here, including links to download the logo.

We visited Speakers Academy headquarters in Rotterdam recently to talk with Rene Warmerdam. Check out the video below to see the details of how the magazine has adopted Layar to engage its print readership.