Automatic content recognition

From Wikipedia, the free encyclopedia

Template:Short description

Page Module:Message box/ambox.css has no content.Page Template:Multiple issues/styles.css has no content.

Automatic content recognition (ACR) is a technology used to identify content played on a media device or presented within a media file. Devices with ACR can allow for the collection of content consumption information automatically at the screen or speaker level itself, without any user-based input or search efforts. This information may be collected for purposes such as personalized advertising, content recommendations, or sale to companies that aggregate customer data.[1][2]

How it works

To start the process, a short media clip (audio, video, or both) is selected from within a media file or captured as displayed on a device such as a smart TV. Using techniques such as fingerprinting and watermarking, the selected content is compared by the ACR software with a database of known recorded works.[2] If the fingerprint of the media clip finds a match, the ACR software returns the corresponding metadata regarding the media as well as other associated or recommended content back to the client application for display to the user, or for collection by the device manufacturer or a company that collects user data.[1]

Fingerprints and watermarking

Two leading methodologies for audio-based ACR are acoustic fingerprinting and watermarking. Similarly, video fingerprinting is used to facilitate ACR for visual media.

Acoustic fingerprinting generates unique fingerprints from the audio content itself. Fingerprinting techniques are agnostic to content format, codec, bit rate and compression techniques.[3] This makes employment of acoustic fingerprinting possible across various networks and channels[<span title="Script error: No such module "decodeEncode".">clarification needed] and is widely used for interactive TV, second screen application, and content monitoring sectors.[4][5] Popular apps like Shazam, YouTube, Facebook,[6] TheTake, WeChat and Weibo reportedly use audio fingerprinting methodology to recognize content played from a TV to trigger additional features like votes, lotteries, topics or purchases.[citation needed]

In contrast to fingerprinting, digital watermarking require the inclusion of digital "tags"[<span title="Script error: No such module "decodeEncode".">further explanation needed] embedded within the digital content stream prior to distribution. For example, a broadcast encoder might insert a watermark every few seconds that could be used to identify the broadcast channel, program ID, and time stamp. This watermark is normally inaudible or invisible to the users, but is detectable by display devices like phones or tablets which can read the watermarks to identify the content it is playing.[5] Watermarking technology is also utilized in the media protection field to help identify where illegal copies originate.[7]

History

In 2011, ACR technology was applied to TV content by the Shazam service, which captured the attention of the television industry. Shazam was previously a music recognition service which recognized music from sound recordings. By utilizing its own fingerprint technology to identify live channels and videos, Shazam extended their business to television programming. Also in 2011, Samba TV (at the time known as Flingo[8]) introduced its patented video ACR technology, which uses video fingerprinting to identify on-screen content and power cross-screen interactive TV apps on Smart TVs.[9] In 2012, satellite communications provider DIRECTV partnered with TV loyalty vendor Viggle to provide an interactive viewing experience on the second screen.

In 2013, LG partnered with Cognitive Networks (later purchased by Vizio and renamed Inscape), an ACR vendor, to provide ACR driven interaction.[10] In 2015, ACR technology spread to even more applications and smart TVs. Social applications and TV manufacturers like Facebook, Twitter, Google, WeChat, Weibo, LG, Samsung, and Vizio TV have used ACR technology either developed by themselves or integrated by third-party ACR providers.[citation needed] In 2016, additional applications and mobile OS embedded with automatic content recognition services were available including Peach, Omusic and Mi OS.[11][12][13]

Applications

  • Advertising and customer data collection: Data collected on the media consumption habits of customers can be very valuable to device manufacturers, advertisers, and data aggregation companies. ACR technology helps these companies survey the interests of customers and collect data so that they can be more precisely targeted with personalized marketing and advertising campaigns. It was reported in Nov 2021 that smart television manufacturer Vizio profited more from the sale of their customers' data than from the televisions they sold.[14][15]
  • Audience measurement: Real-time audience measurement metrics are now achievable by applying ACR technology into smart TVs, set top boxes and mobile devices such as smart phones and tablets. This measurement data is essential to quantify audience consumption to set advertising pricing policies.
  • Content identification: ACR technology helps audiences retrieve information about the content they watched or listened to.[16] The identified video and music content can be linked to internet content providers for on-demand viewing, third parties for additional background information, or complementary media.
  • Content enhancement: Because devices can be "aware" of content being watched or listened to, second screen devices can feed users complementary content beyond what is presented on the primary viewing screen. ACR technology can not only identify the content, but also it can identify the precise location within the content and present additional information to users. ACR can also enable a variety of interactive features such as polls, coupons, lottery or purchase of goods based on timestamp.[17]

Privacy concerns

Organizations ranging from consumer rights advocates Electronic Frontier Foundation to tech web sites such as PCMag have expressed serious objections to the collection of user viewing consumption habits by their devices on privacy grounds.[18][19]

Research

Conducted tests primarily in UK and USA during 2024 by several Universities of UK, USA and Spain with specific TV models from LG and Samsung show that these devices create a consistent and constant network traffic, during tests the devices from LG sending digital fingerprints every 15 seconds and from Samsung every minute to certain network domains. It was found that ACR may not capture frames of third-party content like Netflix due to copyright issues and complicated terms with competitive aggregators, which restrict ACR for their own methods. ACR working when TV used as HDMI-only display, besides registration in TV manufacturer services, works differently in UK and USA due to different regulation by law. ACR traffic can be stopped by an opt-out mechanism.[20]

See also

References

Page Template:Reflist/styles.css has no content.

  1. ^ a b Page Module:Citation/CS1/styles.css has no content."Automated content recognition creating content aware ecosystems" (PDF). Civolution. Archived from the original (PDF) on 23 September 2015. Retrieved 24 June 2015.
  2. ^ a b Page Module:Citation/CS1/styles.css has no content."ACR(Automatic Content Recognition)". ACRCloud Docs. Archived from the original on 28 February 2017. Retrieved 27 February 2017.
  3. ^ Page Module:Citation/CS1/styles.css has no content.Six, Joren; Leman, Marc (2014-10-27). "Panako: a scalable acoustic fingerprinting system handling time-scale and pitch modification". Universiteit Gent. Retrieved 27 February 2017.
  4. ^ Page Module:Citation/CS1/styles.css has no content.Main, Sami (2017-09-19). "Nielsen Is Bringing Real-Time Interactive Ads to Smart TVs to Keep Streaming Audiences Engaged". Adweek. Retrieved 2018-01-11.
  5. ^ a b Page Module:Citation/CS1/styles.css has no content.Brink, Kyle. "A Primer on Automated Content". Viggle. Archived from the original on 2015-06-24. Retrieved 22 June 2015.
  6. ^ Page Module:Citation/CS1/styles.css has no content."Facebook Automatic Content Recognition". Starcom MediaVest Group. SMG. 2014-06-03. Archived from the original on 6 July 2015. Retrieved 6 July 2015.
  7. ^ Page Module:Citation/CS1/styles.css has no content.Solana, Anna (2017-08-17). "How these hidden video watermarks can help spot piracy, doctored images". ZDNet. Retrieved 2018-01-11.
  8. ^ Page Module:Citation/CS1/styles.css has no content.Baumgartner, Jeff (2013-09-24). "Flingo Rebrands as Samba TV". Multichannel News. Retrieved 2021-10-05.
  9. ^ Page Module:Citation/CS1/styles.css has no content.Swedlow, Tracy (July 7, 2011). "Interactive TV News Round-Up (II): Flingo, Hulu, ITU". ITVT. Archived from the original on 2011-07-09.
  10. ^ Page Module:Citation/CS1/styles.css has no content.Gorman, Michael (29 August 2013). "LG partners with Cognitive Networks to make Smart TVs smarter and more interactive". engadget. Retrieved 23 August 2016.
  11. ^ Page Module:Citation/CS1/styles.css has no content."ACRCloud Powers Song Recognition For Hottest New Social Network, Peach". Music Industry News Network. 2016-01-04. Archived from the original on 8 March 2016. Retrieved 3 March 2016.
  12. ^ Page Module:Citation/CS1/styles.css has no content.Victoria, Ho (16 February 2016). "Xiaomi will help you name that song you can't stop humming". Mashable. Retrieved 3 March 2016.
  13. ^ Page Module:Citation/CS1/styles.css has no content."ACRCloud Powers The Launch Of Taiwan's First Music/Humming Recognition Service For Omusic". Music Industry News Network. 2016-01-11. Archived from the original on 8 March 2016. Retrieved 3 March 2016.
  14. ^ Page Module:Citation/CS1/styles.css has no content.Dunn, Thom (2021-11-18). "TV manufacturer Vizio makes more money selling data than TVs". Boing Boing. Retrieved 2021-11-22.
  15. ^ Page Module:Citation/CS1/styles.css has no content."Press Release: VIZIO HOLDING CORP. Reports Q3 2021 Financial Results". Vizio Holding News. 2021-11-09. Retrieved 2025-04-03.
  16. ^ Page Module:Citation/CS1/styles.css has no content.Weiss, Tom (January 23, 2018). "Tom Weiss: Breaking the barriers to addressable advertising in Europe". Broadband TV News. Retrieved 30 August 2018.
  17. ^ Page Module:Citation/CS1/styles.css has no content.Wolf, Michael. "Three Ways Automatic Content Recognition Will Change TV". Forbes. Retrieved 20 June 2015.
  18. ^ Page Module:Citation/CS1/styles.css has no content.Derene, Glenn (2015-02-27). "Samsung, LG, and Vizio smart TVs are recording—and sharing data about—everything you watch Consumer Reports investigates the information brokers who want to turn your viewing habits into cash". Consumer Reports. Retrieved 27 February 2017.
  19. ^ Page Module:Citation/CS1/styles.css has no content.Whitney, Lance (2024-02-20). "How to Stop Smart TVs From Snooping on You". PCMAG. Retrieved 2021-11-22.
  20. ^ Page Module:Citation/CS1/styles.css has no content.Anselmi, Gianluca; Vekaria, Yash; D'Souza, Alexander; Callejo, Patricia; Anna Maria Mandalari; Shafiq, Zubair (2024). "Watching TV with the Second-Party: A First Look at Automatic Content Recognition Tracking in Smart TVS". arXiv:2409.06203v1 [cs.CR].
Retrieved from "http://cosmopedia.co/index.php?title=Automatic_content_recognition&oldid=14398668"