The Uncanny Electrickery of Automatic Content Recognition

I’ve given up trying to fool my IntoNow iPad app into telling me that I’m watching something other than what’s on my TV.  After a fruitless weekend of surfing various live, recorded, YouTube, and time-shifted shows and being informed by IntoNow within a few seconds what I was watching, I admit defeat.  This Automated Content Recognition technology (ACR – great, another acronym!) is wizardry; the kind of witchcraft usually reserved for Monty Python Holy Grail sketches.

But of course, it’s nothing new – Gracenote has been correctly identifying (mostly) the CDs I burn for years.  What has changed are the opportunities ACR brings to video content owners and advertisers in terms of second screen companion apps.  ACR was one of the buzz words at the NAB show this year.  In this blog post, I briefly explain what ACR is, how it works in practice, and what it means in terms of managing metadata and revenue streams for broadcasters.

ACR: About Contextual Relevance

ACR is a way of listening to what viewers are watching sitting in their living rooms and automatically identifying the live feed, TV show, or movie and presenting them with stuff on a second screen that is contextually relevant.  The ACR technology uses audio fingerprinting and voice recognition to correctly identify what is being watched within 4-10 seconds.  It can be embedded within an app installed on a handheld device so that when content is recognized it synchronizes the app with what is being watched on TV.  Once the content has been recognized, the app grabs additional data, social network feeds, images, videos, games, or advertising from a central database over IP and presents it back to the viewer.

ACR makes socializing TV content simple.  If the companion app knows what you are watching, it can show you who else is watching the show and the comments they’ve made via Twitter or Facebook.  Metadata related to the content may be at the show level – the companion app knows you’re watching The Big Bang Theory Season 6 premiere and displays the synopsis, cast, etc for information.  Or it could be at timecode or clip level – Sheldon walks in wearing a new T-Shirt and the companion app asks the viewer if they’d like to buy it from Amazon.com.

ACR: Achieve Certain Revenue?

The opportunities that ACR technology opens up for content owners, service providers, and advertisers are only just starting to be realized.  It’s a crowded and confusing market with players like Audible Magic, Civolution, Shazam, Gracenote, Soundmouse, Into Now, and UK’s ZeeBox building out their solutions and winning clients.  Audible Magic has partnered with several major content database owners, including Digitalsmiths and Rovi, to allow their ACR technology to be linked with rich metadata for second screen apps.  Civolution offer ACR as a way to fingerprint content across multiple devices; this can be down to fingerprinting a particular timecode.  Some vendors are ingesting content sent by their partners to fingerprint the audio signature, others are ingesting live off air using cable set top boxes and employing voice recognition to identify live broadcasts.

Delivering contextual data to a viewer via a second screen companion app is a tantalizing opportunity for new revenue streams.  Here are a just a few:

• Increase the viewers of linear channels through offering loyalty rewards for watching a season premier live or watching the finale of The Voice
• Click-through revenue by displaying a link to a sponsors website at the same time as the viewers are watching the show
• Additional subscription for sports enthusiasts, either for additional match data or for live feeds of alternative camera angles (e.g. Tour de France) served via the companion app
• T-Commerce opportunities – viewers can buy Sheldon’s latest T-Shirt whilst watching The Big Bang Theory
• Coupons – in a recent campaign in partnership with IntoNow, Pepsi offered a coupon to viewers during the SuperBowl (see Gartner)

To take full advantage of some of these engagement opportunities, for the content owners and service providers, ACR means the operational challenge of capturing contextual metadata within the content.  The effort of tagging timecodes or clips for a video needs to be weighed against the revenue potential.

A fascinating technology that has the potential to significantly impact the way people watch and engage with TV, how revenue is generated, and the content metadata supply chain for content owners.

Recommended Further Reading / Watching:

What is Automatic Content Recognition:  http://www.advertisingperspectives.com/adblog/media-technology/what-is-automatic-content-recognition/

An excellent article by John Gilles, EVP, sales & marketing, Coincident – http://www.screenplaysmag.com/2012/02/02/acr-new-techs-vie-to-meet-sync-challenge-in-device-itv/