Reveal The Secrets of Your Videos

Senior Engineering Manager at IBM Watson Media, Budapest Lab (former Ustream)
Feb. 9, 2021

More Related Content

Slideshows for you(20)


Reveal The Secrets of Your Videos

  1. Zoltan Nemeth, Watson Media / Budapest Lab Reveal The Secrets of Your Videos
  2. IBM Budapest Lab • Formerly known as Ustream - acquired by IBM in 2016 • Full scale product development • Engineering • Devops • Product Management • UX / Design Studio • Data Science • Sales • Customer Relations
  3. Video or Black Box?
  4. is using cognitive-infused video products to help our media clients products to help out media clients leverage, monetize, and understand video to drive their core business. IBM Watson Media
  5. ©2018 IBM Cloud Video Watson Video Enrichment Analyze textual, audio, and visual data within multi- media content, and to build easily searchable metadata packages for every asset. • API first offering • Leveraging core Watson APIs • Haifa Research & Budapest Lab algorithms • Built on IBM Cloud Kubernetes Service
  6. ©2018 IBM Cloud Video Idea to Launch in 3 months • Newly formed team • Practices to enable highest velocity • Simple and unified software stack: Go, Docker, Kubernetes • Use what can be used from the Streaming codebase • ADRs • Successful launch at IBC
  7. ©2018 IBM Cloud Video Video Enrichment Workflow
  8. Customer Challenges Solved
  9. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Content Search & Discovery Adding automatically-generated concepts, entities, scenes, and additional keywords ensures the video shows up in more relevant searches
  10. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Live and VOD Captioning • Create highly-accurate captions, quickly • Decrease cost and manual intervention • Accuracy improves with every use
  11. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Trainable Client Specific Custom Models IBM doesn’t use clients’ training material to train core knowledge models so our clients can safely build their custom visual recognition and language models
  12. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Highlight Clipping Watson listens, watches, and learns a live sports feed to automatically identify and curate the most exciting moments into segments that can be used in online highlight packages to satisfy social demand
  13. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Up next Understanding the contents of the video and helping users to track and predict the performance of a logo inside of a set of video content Logo identification Detecting adult content recognition, violence detection, recognizing, detecting racial insensitivity, incorrect or fake material, missing captions, logos that don’t belong, section 508 compliance, ADA compliance, detecting trademark or licensing violations Compliance Integrating enhanced metadata with leading recommendation engines, like provide a solution that helps publishers better engage audiences, increase video views, and optimize yield on owned and operated sites Recommendation uplift
  14. Watson Media Case Studies
  15. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Provide automatically-generated high accuracy live captions CHALLENGE Optimize and automate the complex, error-prone process of closed captioning using the artificial intelligence capabilities of IBM Watson. SOLUTION Watson Captioning powers live and file-based closed captioning for Sinclair Broadcast Group’s 75+ stations. BUSINESS BENEFITS Automated process reduces labor costs and speeds up delivery time Training model and custom corpora produces accuracy rate of 92-96% - will continue to increase with time Hyper-localized data sets to generate more precise captions for local broadcast Major cost savings compared to legacy methods
  16. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Create the ultimate fan experience CHALLENGE Creating the FIFA World Cup Highlight Machine which is is Fox Sports’ first public use of AI and machine learning to more deeply engage fans with a sporting event SOLUTION For the Highlight Machine application, IBM Watson Video Enrichment, a programmatic metadata tool, analyzes the massive amount of historical and 2018 match footage, interviews, press conferences and other related video to create metadata that identifies what is happening in a scene at any given moment along with an associated timestamp BUSINESS BENEFITS Fans could create highlight reels drawing from more than 300 archived matches played between 1958 and 2014. After the tournament began, they were able to create highlight reels from just-completed games about 10 minutes after their completion
  17. ©2018 IBM Cloud Video ©2018 IBM Cloud Video AI-powered video intelligence CHALLENGE Integrating the Watson Video Enrichment cognitive video capabilities in Workday Video Intelligence feature. SOLUTION Workday 31 introduced a new feature, Video Intelligence, as a part of Workday Media Cloud. Video Intelligence is able to analyze the speech and visuals of each video the users load into Workday Media Cloud and surface rich metadata and deeper insights for users. BUSINESS BENEFITS Deeply enriching the value of every video asset uploaded by users by creating automated data sets that are thousands of times more detailed and searchable than is currently possible for large libraries of video.
  18. ©2018 IBM Cloud Video ©2018 IBM Cloud Video AI that enhances the fan experiences at Grammy Awards CHALLENGE Enterprise-grade AI takes a turn on the red carpet SOLUTION During the five-hour celebrity procession, Watson analyzed and categorized hours of video and close to 125,000 photographs, tagging them using facial recognition, facial emotion, position and color dominance. And for the music itself, it analyzed the emotional tone of GRAMMY-nominated song lyrics over the decades, creating a unique data visualization for fans on BUSINESS BENEFITS Enhancing the fan experiences at these events can help client’s business outperform the competition and connect with their customers
  19. 👏👏👏 Thank you! Questions?
  20. Slides that we can use at the booth but not on the stage Backup
  21. IBM Budapest Lab • Formerly known as Ustream - acquired by IBM in 2016 • Founded in 2007 • Startup before there was a startup scene in Hungary • Onboarded IBM as a lab
  22. Expertise in Video... and More Full scale product development capabilities in Budapest, connected with global peers • Engineering • Devops • Product Management • UX • Data Science • Sales • Customer Relations
  23. Our end-to-end video platform Streaming Manager
  24. Reach Every Device with Video • End to end video platform • Watson Cognitive capabilities on top • Fully homebrew, no black boxes • Main components: cloud transcoding, storage, CDN, player
  25. Why Do You Need An Enterprise Video Platform? How can your CEO address the entire company at the same time? Executive Communication How can you reach massive, remote audiences with high quality content? High Profile Events How can you make the biggest impact with your customers and prospects? Marketing Outreach Supplement a media strategy with live video, showcasing content or events Media and Entertainment
  26. Solving Customer Challenges in Video Cloud platform to create video from multiple sources with live transcoding Manage Complexity Multiple content delivery network connection for high quality video at scale Delivery Quality At Scale Complete end-to-end solution that meets the needs of internal or external audiences Maximize ROI Real-time analytics and individual viewer tracking with lead gen techniques Analytics Insights
  27. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Providing massive scale to reach over 380,000 employees with live and on-demand video CHALLENGE Enable any IBM employee across the world to easily communicate more effectively, both internally and externally, using streaming video to audiences of any size SOLUTION • Consolidating vendor presence to utilize IBM Cloud Video on a global scale by using a hybrid approach of cloud based infrastructure with virtual servers installed through on premise hardware RESULTS • Reduced number of video vendors from 6 to 1 • Transitioned 390 Qumu VNE (VideoNet Edge) installations to IBM Cloud Video’s ECDN (Enterprise Content Delivery Network) for local asset scale • Initial 50% annual cost savings • Projected 60% annual cost savings by year two and three
  28. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Turning a product launch into real time revenue with available pre-orders CHALLENGE Wants to showcase their new car to build excitement while learning more about the product’s potential prospects. SOLUTION • Live streamed content to a worldwide audience, opening to markets they previously didn’t sell in • Gathered lead details through in-player registration • Served content in high definition with adaptive bitrate support for those on slower connection speeds to leverage both quality and reach RESULTS • Audience of close to 1 million total with 350k peak concurrent viewers • Unprecedented results with $7.5 billion worth pre-orders within 24 hours of the live event • Average viewing time of 25 minutes
  29. ©2018 IBM Cloud Video ©2018 IBM Cloud Video This premier skincare company is creating innovative virtual experiences that engage and inform consultants world-wide CHALLENGE To reach, energize and inform consultants globally… anytime, anywhere SOLUTION • Streaming mobile compatible live and on-demand video to consultants • Done with SD-CDN technology for global scale with built-in QoS for an optimal viewing experience across devices • Utilize IBM Watson’s speech to text to generate captions for content RESULTS • Content viewed over 500,000 times across the United States, Canada and Australia • Broadcasted over 72 hours of live content and published 137 videos • Reached over 100,000 devices, with 70% of their audience on mobile • 60 minute average viewing time
  30. ©2018 IBM Cloud Video ©2018 IBM Cloud Video Connect employees across regions and across devices CHALLENGE Use video streaming to share information and connect teammates across multiple offices, continents and devices SOLUTION • Created 30 channels that included built-in player Q&A from functional areas such as Engineering and UX • Utilizes live transcoding to make content compatible across devices • Auto archives live streams as on demand assets for later use to avoid time zone conflicts RESULTS • Produced 395 broadcasts with 3,900 views by staff • Reached employees in 30 countries