Divya Jain at AI Frontiers : Video Summarization

•Download as PPTX, PDF•

1 like•1,927 views

As video content is becoming mainstream, video summarization is becoming a hot research topic in academia and industry. Video thumbnail generation and summarization has been worked on for years, but deep learning and reinforcement learning is changing the landscape and emerging as the winner for optimal frame selection. Recent advances in GANs are improving the quality, aesthetics and relevancy of the frames to represent the original videos. Come join this session to get an understanding of various challenges and emerging solutions around video summarization.

Technology

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Video Summarization
Divya Jain | Nov 2018

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Agenda
• Adobe Introduction
• Adobe Sensei (AI)
• Adobe Video Use cases
• Video Summarization
• Why
• What
• How

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
$7.3B
FY2017
Revenue
Adobe is one of the largest and
most diversified software companies in the world
.
35
Years of
revolutionizing industries
~4,300
Patents*
~18,600
Employees in
37 countries
* As of June 2018
3
Only Adobe gives people everything they need
to design and deliver exceptional digital experiences
Most Innovative
Companies in AI
Best Workplace
for Diversity

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Adobe Sensei
Creative Intelligence Experience
Intelligence
Content Intelligenc
e
Serve the creator, Respect the consumer

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Global Video Market is increasing and taking the center stage
Video Content Growth
Source: OMI Top Social Media Trends of 2018

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Video Processing
and
Understanding
Video
Summarization and
Segmentation
Bringing the Magic of AI to Videos
Video Stabilization
and
Quality

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo : Video Stabilization and Quality
https://research.adobe.com/project/subspace/

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo: Video Processing and Understanding
https://www.youtube.com/watch?v=BxhcqhPzQIU&feature=youtu.be
• Project Cloak – remove unwanted objects from a video
• Experimental tool from Adobe Research

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Video Summarization Use Cases
Efficient Viewing
Sport
Highlight
Length Limitation
Adaptation
Multilength Video
Ads
Ascertaining Interest
Thumbnail
Quick
Peek

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Video Summarization Categories
Summaries/Highlights
follow a specific
structure/template
Summaries/Highlights
are very diverse and
video-specific

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Video Summarization: Approach 1
Variational Auto-Encoder +
Generative Adversarial Network
Approach
Input
Video Frame and
Feature
Extractor
Frame
Features
Frame Selector
Subset of
Frames
Reconstruction
Minimize
Distance
Generate summary from the most representative frames from the video

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo
Original Video

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo
Summary Videos
30
Seconds
10
Seconds

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Video Summarization: Approach 2
• Use Reinforcement Learning
Source: https://arxiv.org/abs/1801.00054

© 2018 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo
Highlights, fouls, and injuries
only

© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Thanks!
For more information on Adobe AI follow @adobeSensei
For my views on various topics follow @divyajain1

Similar to Divya Jain at AI Frontiers : Video Summarization

Adobe Systems’ successful business transformation to a cloud-based business with a subscription model required significant shifts in infrastructure, mindsets and practices. Listening deeply to leaders, managers and employees, Adobe’s employee experience team abolished the annual performance review process for a light-weight performance management approach that provided more flexibility in a fast-paced environment. Adobe’s approach, called Check-in, removed complexity and bureaucracy that didn’t serve the business but focused on the “critical few” activities that supported business growth. This wasn’t just a process change but a different level of engagement from leaders, managers and employees with a greater sense of ownership and responsibility. Angela Szymusiak will share the underlying philosophies behind the design, how Adobe integrated Check-in into its people practices and the change management strategies used for a successful implementation

LESSONS LEARNED FROM ADOBE’S PERFORMANCE MANAGEMENT REVOLUTION

Human Capital Media

Adobe summit 5 myths of video marketing

Adobe Experience Cloud

AEM DataLayer IMMERSE 2017 Presentation by Dan Klco

Daniel Klco

Introducing AI Edit: Transforming Video Editing with Advanced AI Capabilities AI Edit is a groundbreaking editing platform that is revolutionizing the way videos are created. With its state-of-the-art features and intelligent automation, AI Edit is set to redefine the landscape of video editing for professionals and enthusiasts alike. Unleash Your Creativity with Easy Controls AI Edit brings a new level of simplicity to the editing process. It understands codewords like "c" for cut, "pal" for paste at last, and "m" for move, making editing effortless. Whether you prefer voice commands, keyboard controls, or a combination of both, AI Edit adapts to your style seamlessly. Revolutionary Storytelling Made Effortless Prepare to be amazed as AI Edit analyzes every aspect of your videos and audio, automatically organizing them into a cohesive and engaging story. Simply describe the desired sequence, such as "create Goku's entry, Vegeta's epic battle, and Frieza's dramatic entrance," and AI Edit will flawlessly arrange the clips and apply effects, allowing you to focus on the visual impact. Maximize Your Screen's Potential AI Edit provides an uncluttered workspace, maximizing your screen's real estate. Unnecessary buttons like cut and paste are intelligently hidden, allowing you to fully immerse yourself in the editing process. This streamlined interface enhances productivity and enables you to create extraordinary videos with ease. Experience Limitless Editing Capabilities With AI Edit, the possibilities are endless. Discover a wide range of advanced filters, cinematic effects, and artistic stylizations to enhance the visual appeal of your videos. Take advantage of intelligent audio editing options, such as noise reduction, audio leveling, and automatic suggestions for background music and sound effects. Seamlessly integrate smart transitions to create polished and captivating visuals. Unlock Your Video's Potential with Intelligent Features AI Edit empowers you to add professional-quality text, captions, subtitles, and titles to your videos using customizable templates. Correct color imbalances, enhance contrast, and adjust exposure effortlessly with automated color correction. Bring your footage to life with motion tracking, allowing you to incorporate graphics, text, and effects that follow the movement within your video seamlessly. Effortless Editing at Your Fingertips AI Edit simplifies complex tasks. Easily detect and replace green screens, stabilize shaky footage for a professional look, and create stunning time-lapse and slow-motion effects to capture attention and heighten the impact. Collaborate seamlessly with team members using AI Edit's collaboration features, and integrate with popular cloud storage platforms for easy access and sharing across devices. Invest in the Future of Video Editing Embrace AI Edit and witness the convergence of creativity and innovation. It's capabalities are beyond words that my 3000 word limit isn't enough

AI Editor Presentation.pptx

MohammedArif217750

Adobe I/O is a central hub for APIs, SDKs, and platform technology to all developers who want to integrate with and extend Adobe’s advanced technology. From APIs to Events, from Adobe Sensei to Serverless technology, Adobe I/O allows you to quickly integrate content and data, and create richer customer experiences. In this session, we will show you how to leverage Adobe I/O to build an event-driven, cross-cloud orchestration with ease.

Evolve18 | Carmen Sutter & Sarah Xu | Accelerate your Digital Experience with...

Evolve The Adobe Digital Marketing Community

Dalet Pulse at NAB 2019 - Adobe Creative Cloud for video and audio

Dalet

Reveal The Secrets of Your Videos

Zoltán Németh

ChatGPT 4-powered voiceovers have proven to be incredibly effective, with businesses reporting a substantial boost in brand perception and customer response. And Ai Video Suite has revolutionized and automated this process. ChatGPT 4 + AI Videos, Voiceovers = The Holy Grail of Video Marketing in 2023. Produce Engaging GPT 4 Powered Voiceovers at your fingertips: Step 1: Create: Use the built-in GPT 4 AI Vox creator to generate a voiceover script. You could also upload and edit an existing script! Step 2: Generate and Listen: Simply copy and paste the script into the Vox Generator. Select your preferred language, accent, tone, pitch, and volume. In a matter of minutes, your voiceover is ready to download. Optionally add background music from their built-in library or bring your own. Add this vox to your sales videos, Business ads, explainer and product promos and any other videos, audio books etc. Step 3: Profit: These voiceovers are yours to use in your own business or sell them to clients for big profits! Rinse and repeat! Activate Ai Video Suite Apps & ENJOY – Enjoy massive monthly savings, traffic, leads, sales, conversions & profits…. And Live The Of Your Dreams. Don’t Let The Competition Outshine You. Embrace Ai Video Suite and unlock a world of limitless possibilities for your business. Your success story begins now, and they’re here to guide you every step of the way. So why wait? Sign Up Today And Experience The Power Of AI-Driven Video Marketing &Content Creation.

AIVideo Suite Review .pdf

MahbubKhan46

In the rapidly evolving digital landscape, content creators, marketers, and businesses are consistently in search of inventive methods to captivate their audiences. Short-form videos have risen to prominence as a dominant format, with platforms like TikTok, Instagram Reels, and YouTube Shorts garnering widespread acclaim. Crafting short videos traditionally involves a time-intensive and intricate process, encompassing video shooting,

AI Bulk Shorts Review 2023-All Upsell Links + Coupon Code + Login App Software

harryjohn31

Current data shows that user generated content (UGC) is one of the most trusted sources of information, driving 29% higher conversion rates on websites and influencing 90% of purchase decisions. Learn how Adobe Experience Manager Livefyre enables you to discover, organize and publish a constant flow of trusted UGC onto owned properties to invigorate content strategies, engage and influence audiences and drive revenue.

Adobe Experience Manager Livefyre Overview

Adobe

Adobe Stock Integration community project

Magecom UK Limited

Adobe Session on Flash Online Conference #12

Flash Conference

See how Flash Professional CC lets you create assets and animations for virtually any use case, device, browser or platform. Too often, users confuse the Flash Professional application with the Flash runtimes and believe they can target only certain browsers and devices with their content. Not so! Adobe has been enhancing the creative toolset within Flash Professional while building support for custom platforms. Get current with the modern, creative capabilities of this amazing tool. Recording: http://max.adobe.com/sessions/max-online/#/video/4870

Adobe MAX 2015 - Giving Flash Professional Another Look

Joseph Labrecque

Did you know more than 50% of consumers prefer video content over other forms of content like emails, social, and blogs? This may not be a surprise but when we hear video marketing, we instantly think it will drain our budget and our resources. However, video can be made simple! Watch our on-demand webinar to see Marketo’s use case examples of inspirational and highly engaging videos that never involved any video production team. We’ll discuss: Key trends in video to get you inspired for 2020 How Marketo use video in new ways across digital, demand gen, content and email marketing to lift engagement and conversation rates Use case video examples from different stages of the customer journey

[Webinar] most engaging videos of 2019

Marketo

These are the slides that I presented at the 2010 Adobe MAX conference. Explore the role of prototyping in designing and developing compelling mobile applications. We'll examine the forms that prototypes can take (from paper prototypes to rich, interactive simulations), their advantages and costs, and how Adobe tools can be used to create them. Then we'll walk through creating a user interface for a mobile application using Adobe Illustrator, Photoshop, Flash Professional, Flash Catalyst, and Flash Builder. This presentation also covers best practices for UX, as well as developer-focused tips and tricks.

Creating Compelling Mobile User Experiences

Chris Griffith

During our LIVE event series (NAB 2020 Edition) two of Bitmovin's product experts took some time to define the five analytics metrics that matter most to ensure your OTT or Video Application succeed, why, and how. These metrics were Video Startup Time, Bitrate Heatmaps, Impressions v Total Hours Watched, Error Type (and %), and rebuffering percentage. Hear what they had to say in the full on demand recording here: https://go.bitmovin.com/techtalk-live-video-metrics-obsession?utm_source=slideshare

Bitmovin LIVE Tech Talks: 5 Analytics Metrics That Matter

Bitmovin Inc

Ai video generator from text

billypowelk

Undoubtedly, video content is more attractive than text. The demand for engaging visual content continues to soar quickly. As businesses and content creators strive to capture the attention of their audiences, tools that can seamlessly transform text into dynamic videos have become focus hots. So comes out OpenAI Sora. This text-to-video generative AI model looks incredibly impressive so far, introducing some huge potential across many industries.

More Besides Sora: Tools to Create Dynamic Videos from Textual Content

RachelWang856621

Graphical DSL with Sirius: how to simplify the creation of custom modeling tools

Etienne Juliot

Elastic @ Adobe: Making Search Smarter with Machine Learning at Scale

Elasticsearch

Similar to Divya Jain at AI Frontiers : Video Summarization (20)

LESSONS LEARNED FROM ADOBE’S PERFORMANCE MANAGEMENT REVOLUTION

Adobe summit 5 myths of video marketing

AEM DataLayer IMMERSE 2017 Presentation by Dan Klco

AI Editor Presentation.pptx

Evolve18 | Carmen Sutter & Sarah Xu | Accelerate your Digital Experience with...

Dalet Pulse at NAB 2019 - Adobe Creative Cloud for video and audio

Reveal The Secrets of Your Videos

AIVideo Suite Review .pdf

AI Bulk Shorts Review 2023-All Upsell Links + Coupon Code + Login App Software

Adobe Experience Manager Livefyre Overview

Adobe Stock Integration community project

Adobe Session on Flash Online Conference #12

Adobe MAX 2015 - Giving Flash Professional Another Look

[Webinar] most engaging videos of 2019

Creating Compelling Mobile User Experiences

Bitmovin LIVE Tech Talks: 5 Analytics Metrics That Matter

Ai video generator from text

More Besides Sora: Tools to Create Dynamic Videos from Textual Content

Graphical DSL with Sirius: how to simplify the creation of custom modeling tools

Elastic @ Adobe: Making Search Smarter with Machine Learning at Scale

More from AI Frontiers

Topic: How to use big data to enhance AI Outline: 1. Spark ETL Spark SQL Spark Streaming 2. Spark ML Spark ML pipeline Distributed model tuning Spark ML model and data lineage management 3. Spark XGboost XGboost introduction XGboost with Spark XGboost with GPU 4. Spark Deep Learning pipeline Transfer learning Build Spark ML pipeline with TensorFlow Model selection on distributed TF model

Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI

AI Frontiers

Topic：Heuristic Search and how does it apply to self-driving cars? (for Beginners) Outline: 1. Technologies behind self-driving vehicles - Motion Planning - Decision Making 2. Graph Search Algorithms - Depth-First Search - Breadth-First Search - A* Search 3. Incremental Heuristic Search Algorithms - Repeated A* Search - Adaptive A* - Generalized Fringe-Retrieving A*

Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...

AI Frontiers

In this tutorial I will introduce recent work in applying weak supervision and reinforcement learning to Questions Answering (QA) systems. Specifically we discuss the semantic parsing task for which natural language queries are converted to computation steps on knowledge graphs or data tables and produce the expected answers. State-of-the-art results can be achieved by novel memory structure for sequence models and improvements in reinforcement learning algorithms. Related code and experiment setup can be found at https://github.com/crazydonkey200/neural-symbolic-machines. Related paper: https://openreview.net/pdf?id=SyK00v5xx.

Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...

AI Frontiers

Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...

AI Frontiers

Instructor: Mat Leonard Outline 1. Text Processing Using Python + NLTK Cleaning Normalization Tokenization Part-of-speech Tagging Stemming and Lemmatization 2. Feature Extraction Bag of Words TF-IDF Word Embeddings Word2Vec GloVe 3. Topic Modeling Latent Variables Beta and Dirichlet Distributions Laten Dirichlet Allocation 4. NLP with Deep Learning Neural Networks Recurrent Neural Networks (RNNs) Word Embeddings Sentiment Analysis with RNNs

Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural Networks

AI Frontiers

Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...

AI Frontiers

Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...

AI Frontiers

In recent years, machine learning has undoubtedly been hugely successful in driving progress in AI applications. However, as we will explore in this talk, even state-of-the-art systems have "blind spots" which make them generalize poorly out of domain and render them vulnerable to adversarial examples. We then suggest that more unsupervised learning settings can encourage the development of more robust systems. We show positive results on two tasks: (i) text style and attribute transfer, the task of converting a sentence with one attribute (e.g., sentiment) to one with another; and (ii) solving SAT instances (classical problems requiring logical reasoning) using end-to-end neural networks.

Percy Liang at AI Frontiers : Pushing the Limits of Machine Learning

AI Frontiers

I will present several advances in deep learning from OpenAI. First, I will present OpenAI Five, a neural network that learned to play on par with some of the strongest professional Dota 2 teams in the world in an 18-hero version of the game. Next, I will present Dactyl, a human-like robot hand trained entirely in simulation with reinforcement learning that has achieved unprecedented dexterity on a physical robot. I will also present our results on unsupervised learning in language, that show that pre-training and finetuning can achieve a significant improvement over state of the art. Finally, I will present an overview of the historical progress in the field.

Ilya Sutskever at AI Frontiers : Progress towards the OpenAI mission

AI Frontiers

Mark Moore at AI Frontiers : Uber Elevate

AI Frontiers

The availability of affordable electronics components, powerful embedded microprocessors, and ubiquitous internet access and WiFi in the household has enabled a new generation of connected consumer robots. In 2015, iRobot launched the Roomba 980, introducing intelligent visual navigation to its successful line of vacuum cleaning robots. In 2018, iRobot launched the Roomba i7, equipped with the latest mapping and navigation technology that provides spatial information to the broader ecosystem of connected devices in the home. In this talk, I will describe the challenges and the potential of introducing consumer robots capable of developing spatial context by exploring the physical space of the home, and I will elaborate on the impact of AI in the future of robotics applications. Moreover, I will describe our vision of the Smart Home, an AI-powered home that maintains itself and magically just does the right thing in anticipation of occupant needs. This home will be built on an ecosystem of connected and coordinated robots, sensors, and devices that provides the occupants with a high quality of life by seamlessly responding to the needs of daily living – from comfort to convenience to security to efficiency.

Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...

AI Frontiers

Arnaud Thiercelin at AI Frontiers : AI in the Sky

AI Frontiers

As the data and models scale, it becomes necessary to have multiple processing units for both training and inference. SignSGD is a gradient compression algorithm that only transmits the sign of the stochastic gradients during distributed training. This algorithm uses 32 times less communication per iteration than distributed SGD. We show that signSGD obtains free lunch both in theory and practice: no loss in accuracy while yielding speedups. Pushing the current boundaries of deep learning also requires using multiple dimensions and modalities. These can be encoded into tensors, which are natural extensions of matrices. These functionalities are available in the Tensorly package with multiple backend interfaces for large-scale deep learning.

Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...

AI Frontiers

Wei Xu at AI Frontiers : Language Learning in an Interactive and Embodied Set...

AI Frontiers

The use of AI for voice search and image recognition is talked about often. Enterprises, however, have different challenges and requirements. In this talk, we will focus on talking about use cases in the enterprise and challenges in building out AI solutions. We will talk about how an Auto-machine learning software for videos and images called PowerAI Vision enables quick AI model training & deployment for various enterprise use cases.

Sumit Gupta at AI Frontiers : AI for Enterprise

AI Frontiers

Deep Reinforcement Learning (DRL) has made strong progress in many tasks, such as board games, robotics, navigation, neural architecture search, etc. I will present our recent open-sourced DRL frameworks to facilitate game research and development. Our framework is scalable so we can can reproduce AlphaGoZero and AlphaZero using 2000 GPUs, achieving super-human performance of Go AI that beats 4 top-30 professional players. We also show usability of our platform by training agents in real-time strategy games, and show interesting behaviors with a small amount of resource.

Yuandong Tian at AI Frontiers : Planning in Reinforcement Learning

AI Frontiers

The latest AI advances have the potential to massively improve our health and well being. However, most of the work is yet to be done. In this talk, we will explore the most important opportunities for AI in healthcare. For example, we will explore how AI can diagnose major life-threatening conditions even before those conditions emerge. We will talk about AI ability to recommend dramatically more effective and less harmful treatment plans based on AI understanding of patient's medical history and current conditions. Finally, we will talk about AI role in making our healthcare system effective and affordable for everyone.

Alex Ermolaev at AI Frontiers : Major Applications of AI in Healthcare

AI Frontiers

Games have been leveraging AI since the 1950s, when people built a rules-based AI engine that played tic-tac-toe. With technological advances over the years, AI has become increasingly popular and widely used in the gaming industry. The typical characteristics of games and game development makes them an ideal playground for practicing and implementing AI techniques, especially deep learning and reinforcement learning. Most games are well scoped; it is relatively easy to generate and use the data; and states/actions/rewards are relatively clear. In this talk, I will show a couple of use cases where ML/AI helps in-game development and enhances player experience. Examples include AI agents playing game and services that provide personalized experience to players.

Long Lin at AI Frontiers : AI in Gaming

AI Frontiers

Melissa Goldman at AI Frontiers : AI & Finance

AI Frontiers

Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...

AI Frontiers

More from AI Frontiers (20)