Is all that Sparkles gold?
A look at the near-future developments for Augmented Reality announced at F8 2019
Matt Greenhalgh, Technical Director
In 2018, Stink Studios was chosen as one of a handful of partners to build Instagram AR filters using Facebook’s Spark AR platform as part of its private beta program. We created pioneering AR filters on the platform, partnering with Disney to help Mickey and Minnie Mouse celebrate their 90th anniversary and with German influencers Lisa and Lena to drive engagement across 17 million followers.
As Instagram Beta matures, we were invited to this year’s F8 conference to hear what developments were in store for the platform, as well as for creators, agencies and brands.
To understand the potential of the Spark AR platform, it’s important to understand the context around augmented reality technologies in 2019. AR has long been seen as a sister technology to VR, and for many, it looked set to fall foul of the teething troubles that beset that platform — expensive hardware leading to low consumer uptake, resulting in indifference from brands unwilling to wait for the long tail to yield returns. One already significant difference, though, is that AR has solved its hardware distribution issue. Anyone with a modern smartphone already has everything they need to view AR content, and accessing it can require as little as a single tap of the screen.
Of course, AR has suffered its own challenges. The late-noughties webcam incarnation was a clunky experience, requiring java plugins, ugly tracking markers and providing a single-digit framerate for many experiences. With the advent of powerful GPUs in smartphones, there’s been a land grab for the second wave of powerful, stable AR experiences that could be accessed directly through a phone. Apple brought a world of markerless tracking with their own ARKit, and Google quickly followed suit with ARCore, both of which require dedicated app installs for users to access content. WebAR is seeing developments in the form of the open-source AR.js, but at the time of writing is still tied to marker-based tracking. Google’s WebARonArCore browser-based solution also shows a lot of promise, but remains a developer-only experience for the time being. That leaves creators and brands clamouring for easy to develop, high-quality AR, accessible to anyone, with minimal access requirements.
Enter Spark AR.
Stink Studios has always been technology agnostic: we bring to bear the best and most appropriate tools available to create amazing experiences for any context. Today, Spark AR represents one of the best tools available to create AR content. Its IDE — Spark AR Studio — though still a comparative newcomer, has seen impressive development in the year and a half we’ve been using it. New feature releases come thick and fast, and are updated on a regular basis. In addition to a developer-led JavaScript authoring environment, it also supports a more motion-graphics-artist-friendly node graph based approach, which should be familiar to anyone who’s used visual shader editors or material graphs. It also offers modern features like PBR material, lighting and rendering support as well as progressive tools like Signed Distance Fields to minimise reliance on costly external texture assets.
To date, Spark AR has excelled at creating the kind of fun face adornments that will be familiar to any user of Snapchat lenses. But Facebook’s platform scale represents a more enticing proposition to brands. Can it offer the kind of strides in development potential that will see brands join the AR revolution wholesale? We’re here at F8 to find out.
Day 1
Privacy was very much the headline of Mark Zuckerberg’s keynote. He seemed serious about it, and many of his subsequent product announcements affirmed this positioning for Facebook future strategic direction. The AR news, however, took a slightly different tack.
To date, AR filters have been mostly about sticking funny things on peoples’ heads. As such, they’ve avoided some of the privacy concerns expressed around other products. But as the Spark AR platform matures towards full release, it will come under closer scrutiny.
So what news on the platform maturing from F8?
First of all, a big milestone: Spark AR filters have been viewed 1 billion times in the last year. The detail of how this metric is calculated isn’t entirely clear, but it’s a hard number to ignore if you’re considering audience exposure.
Spark AR will also be in open public beta for any developer to create content from this summer. This is part of Facebook’s democratised vision for content creation. Spark AR has been designed as an easy to use tool with a shallow learning curve. That’s great news for creators, and great news for brands who now have more choice in their developer partners.
Another significant bit of news, for developers at least, is that the Spark AR platform can now be developed on Windows, using a new built-from-the-ground-up codebase with files shared across Mac and Windows.
But what struck me as the biggest bit of news was that Instagram (and the Spark AR filters delivered therein) will soon offer in-app shopping fulfilment. This feels like it could be huge. Influencers promoting brand products can prompt click-to-buy directly within Instagram with no need to go to a web store or rely on Amazon to provide the shopping component of the experience. Purchase is offered at the point of inspiration.
There were a number of other features announced for developers to get excited about. Some are available now and some are hinted at as part of future releases: programmable render pipelines, multiple tracking markers, a robust and full-featured audio processing platform and reusable ‘Block’ components to improve team-working during development, and code re-use, which should all equate to making the tool more powerful and easier to use.
Day 2
As tradition dictates, the Day 2 keynote was a more technical and more subdued affair. The theme was very much centred on how Facebook is leveraging machine learning to improve and automate the process of eliminating undesirable content across their platforms. There was a real sense of both the challenge they face and the lengths they go to to combat the much-publicised shortcomings of the platforms’ ability to stay ahead of ‘bad actors’. The extent to which automated machine learning now contributes to that task and achieves successful results was surprising. The examples cited included shutting down the sales of illicit goods, dealing with how profiles of the deceased are managed with surviving friends and family, and inclusive AI that seeks to remove algorithmic bias before it makes feature releases across the product range.
The Spark AR news wasn’t quite as plentiful as the previous day, but there were some interesting sessions.
First up was a whistle-stop tour of optimisation techniques specifically for effects targeting the Instagram platform. We’ve been fortunate enough to take advantage of these already in our beta work for the platform, but it was a useful reminder of the value of keeping your experience lean.
Of more interest to brands were the next two sessions. The first looked at how Facebook sees the relationship between AR and physical locations progressing. The four big themes were Retail, Education, Museums and Utility (the latter positing a world where a till receipt could be a tracking marker that launches a post-sales experience). The final future-facing slides gave us a sense of Facebook’s bigger ambition for AR. They see it having persistence, such that it can retain information about users’ prior visits to a location or be influenced by them over time. And they closed out with the most audacious claim — that, ultimately, ‘users will expect AR’. This is a bold but credible vision of how ubiquitous AR could become.
The final session focussed on shopping and AR, an understandably popular session that saw the classroom packed to bursting. The big news here was the introduction of ‘Commerce Blocks’ to Spark AR Studio. These will allow in-app purchasing directly from filters, and lends itself to the to the try on/try out use cases that AR is so well suited to. They also demoed improved lighting, make-up effects and scene semantic understanding features that should dramatically improve virtual product realism and integration.
So that was F8 2019, or at least the AR content side of it. There are a hundred blog posts that could be written to cover the breadth of information shared and displayed. It’s been a dizzying experience being here. Exciting and enlightening in equal measure. Facebook have their detractors and have had a difficult recent history but the sense of vision and purpose is something to behold. Speaking to the individuals who actually contribute to this enormous technology platform reveals a consistently smart group of people who are incredibly motivated to solve the problems they face.
We’re equally excited by the potential of AR and are already testing out the new features introduced in the beta of Spark AR Studio just released at F8. If you’re interested in having Stink Studios develop Spark AR content for your brand, you can email us as sparkar@stinkstudios.com
