RedisConf18 - Video Experience Operational Insights in Real Time.

•Download as PPTX, PDF•

1 like•183 views

This document summarizes a video operational insights platform that provides real-time analytics on key metrics like quality of experience (QoE) and audience engagement for live video streams. The platform uses Redis to store streaming event data with HyperLogLog for unique counting. It partitions data across local and global Redis clusters and aggregates metrics near real-time to provide insights on millions of events with latency under 20 seconds.

Technology

Video Experience Operational
Insights in Real Time.
Aditya Vaidya - Sr Software Engineer

4
VDMS enables partner experiences
Uplynk : 400 channels 35,000 live events | Edgecast: 45 Tbps total capacity

8
Video Operational Insights Platform - 2015 v/s 2017

9
Redis
● In-memory db - High throughput & Low latencies
● Advanced data structures - SortedSet, HyperLogLog (probabilistic)

Operational Insights - Quality of Experience (QoE)
10
● Rebuffer Ratio - % of viewers time spent watching stalled playback
○ Metrics
● Rebuffering duration
● Playback duration
○ Dimensions
● Content Delivery Networks (Multi CDN environment)

Operational Insights - Audience Engagement
11
● Concurrent Views - Number of unique Concurrent Video Sessions
○ Metrics
● Unique Video Sessions
○ Dimensions
● Brands (Oath or Partner) e.g. yahoo sports, aol etc
● Devices e.g. Mobile, Desktop etc

12
● As close as possible to Real Time
● Near Real Time aggregations over millions of events
Near Real Time Challenges

1616
Partitioning
● Local Redis Cluster
○ video_session_id
● Global Redis Cluster
○ key -> DIMS:MEASURE

17
HyperLogLog - Unique Counting
● Approximate data sketch
● Count upto 2^64 with std. error of .81% in 12KB

18
Metrics - Concurrent Views
● Local Redis
○ PFADD key video_session_id (streaming)
○ DUMP key (once per interval)
● Remote Redis Cluster
○ RESTORE (once per interval)
○ PFMERGE keys_from_all_servers (once per interval)

20
Metrics - Rebuff, Starts, Bitrate etc
● Local Redis
○ INCRBY key (streaming)
○ GET key (once per key per interval)
● Remote Redis Cluster
○ INCRBY key (once per interval)
○ GET key (once per interval)

25
Results
● End-to-end latency < 20 secs
● Multi CDN failure detection during a global scale live event

SpringOne Platform 2017 Pulkit Chandra, Pivotal "One of the most important factors in a microservices architecture is that application logic is separate from the data store. This design choice makes it easier for the application to scale. Providing a caching solution inside Pivotal Cloud Foundry makes it easy for these microservices to store data which can be retrieved 100x times faster than with a regular database. Pivotal Cloud Cache not only provides such a cache but takes a “use case”-based approach which gets an application from 0 to production fast. This session will provide insights into how to use Pivotal Cloud Cache and its performance under load. We will demo a Spring Boot app which uses Spring Data Geode to talk to a Pivotal Cloud Cache cluster."

RedisDay London 2018 - Redis & BioCatch Catching the Bad Guys in Action with ...

Redis Labs

StorageArchitecturesForCloudVDI

Vinay Rao

Keeping your application’s latency SLAs no matter what

ScyllaDB

Businesses that once measured performance in seconds now measure it down to the millisecond and even the microsecond in order to provide optimal user experience. For a NoSQL database few things are more important than keeping latencies low and bounded. Yet some databases suffer latency spikes from such regular occurrences as Java Virtual Machine (JVM) “garbage collection,” context switches, database repair, cache flushes and so on. This makes long-tail latency very tricky to diagnose and fix, as it’s often a “whack-a-mole” exercise. In this session, we will cover: The systemic causes of latency spikes How to keep latencies bounded and predictable How to manage latency-inducing events How Scylla helps optimize for 99% latency of <1msec

Reliable, Scalable Kubernetes on AWS

Applatix

Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020

Redis Labs

JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...

Redis Labs

How Orwell built a geo-distributed Bank-as-a-Service with microservices

MariaDB plc

Orwell Group shares how they leveraged microservices, an event driven architecture and both master and reference data management methodologies to build a new banking system for high retail banking customers and corporate banks requiring cross border payments and cash flow management – and scaled it to handle customers with millions of clients. In particular they explain how they built a high availability, geo-distributed and consistent platform on top of MariaDB. The result was a secure and distributed platform with high cost efficiency, and the data accuracy and consistency needed to create high quality data pipelines from transactions to analytics and ensure regulatory compliance (e.g., GDPR).

Fast Tracking Dev Teams to Container Adoption

Nebulaworks

Introduction to CloudStack Storage Subsystem

buildacloud

Enterprise SaaS Persistence With AWS Databases

Vishwastam Shukla

Software-Defined Storage (SDS)

HTS Hosting

Don’t Leave Bare Metal Workloads Behind

NEXTtour

Ceph Management and Monitoring with Dashboard v2 - Lenz Grimmer

Ceph Community

StorPool Presents at Cloud Field Day 9

StorPool Storage

StorPool presents at Cloud Field Day - the leading technology event focused on the impact of cloud technologies on enterprise IT. During the event, the high-performance block storage specialist will showcase how its storage technology allows cloud builders to easily outperform cloud titans like AWS, Microsoft Azure and GCP. Performance is of major importance for modern applications and workloads. No matter if you run a private cloud or deliver public cloud services for customers, you need to ensure the excellent performance for the workloads running on the cloud. Often misunderstood, storage has a direct impact not only on the reliability of cloud services, but also on the performance of the entire cloud. https://storpool.com/news/storpool-presents-at-cloud-field-day-9

Container Native Storage

Data Source

Building a Real-Time Gaming Analytics Service with Apache Druid

Imply

At GameAnalytics we receive and process real time behavioural data from more than 100 million daily active users, helping thousands of game studios and developers understand user behaviour and improve their games. In this talk, you will learn how we managed to migrate our legacy backend system from using an in-house built streaming analytics service to Apache Druid, and the lessons learned along the way. By adopting Druid, we have been able to reduce development costs, increase reliability of our systems and implement new features that would have not been possible with our old stack. We will provide an overview of our approach to schema design, segments optimization, creation of our query layer, caching and datasources optimisation, which can help you better understand how you can successfully use Druid as a key component on your data processing and reporting infrastructure.

Mux loves Clickhouse. By Adam Brown, Mux founder

Altinity Ltd

What's hot

RedisDay London 2018 - How Redis Powers BBC Online's Biggest Pages

Redis Labs

Persistent Storage for Containerized Applications

Colleen Corrice

Red hat storage objects, containers and Beyond!

andreas kuncoro

RedisDay London 2018 - Stack Overflow's Next Steps in Redis

Redis Labs

Caching for Microservives - Introduction to Pivotal Cloud Cache

VMware Tanzu

RedisDay London 2018 - Redis & BioCatch Catching the Bad Guys in Action with ...

Redis Labs

StorageArchitecturesForCloudVDI

Vinay Rao

Keeping your application’s latency SLAs no matter what

ScyllaDB

Reliable, Scalable Kubernetes on AWS

Applatix

Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020

Redis Labs

JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...

Redis Labs

How Orwell built a geo-distributed Bank-as-a-Service with microservices

MariaDB plc

Fast Tracking Dev Teams to Container Adoption

Nebulaworks

Introduction to CloudStack Storage Subsystem

buildacloud

Enterprise SaaS Persistence With AWS Databases

Vishwastam Shukla

Software-Defined Storage (SDS)

HTS Hosting

Don’t Leave Bare Metal Workloads Behind

NEXTtour

Ceph Management and Monitoring with Dashboard v2 - Lenz Grimmer

Ceph Community

StorPool Presents at Cloud Field Day 9

StorPool Storage

Container Native Storage

Data Source

What's hot (20)

RedisDay London 2018 - How Redis Powers BBC Online's Biggest Pages

Persistent Storage for Containerized Applications

Red hat storage objects, containers and Beyond!

RedisDay London 2018 - Stack Overflow's Next Steps in Redis

Caching for Microservives - Introduction to Pivotal Cloud Cache

RedisDay London 2018 - Redis & BioCatch Catching the Bad Guys in Action with ...

StorageArchitecturesForCloudVDI

Keeping your application’s latency SLAs no matter what

Reliable, Scalable Kubernetes on AWS

Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020

JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...

How Orwell built a geo-distributed Bank-as-a-Service with microservices

Fast Tracking Dev Teams to Container Adoption

Introduction to CloudStack Storage Subsystem

Enterprise SaaS Persistence With AWS Databases

Software-Defined Storage (SDS)

Don’t Leave Bare Metal Workloads Behind

Ceph Management and Monitoring with Dashboard v2 - Lenz Grimmer

StorPool Presents at Cloud Field Day 9

Container Native Storage

Similar to RedisConf18 - Video Experience Operational Insights in Real Time.

Building a Real-Time Gaming Analytics Service with Apache Druid

Imply

Mux loves Clickhouse. By Adam Brown, Mux founder

Altinity Ltd

Netflix Playback Data Systems Team and Job Overview

Suudhan Rangarajan

Arm html5 presentationIan Renyard

A165 tools for java and javascript

Toby Corbin

WebRTC for Mobile

Shubh Agarwal

Kaltura Inspire Webinar: API Driven Video Platform - The Key to Scalability a...

Zohar Babin

Scalability and flexibility can make or break your online video success. When putting an online video plan in place for your organization, you want to be certain that you will be able to accommodate adoption and usage, increasing content quantity and quality, and any new features you may need. In Kaltura Inspire this webinar we discussed the fundamentals of API-driven online video solutions and how you can leverage this to gain scalability, flexibility, and control of end-user experience. How to avoid potentially slow, risky, and error prone manual deployments by using the Kaltura API to fully automate your cluster deployments and upgrades. See how you could plan and execute a video platform that better fits your needs and have more influence on how your users experience online video.

Managing the Customer Experience for Media – Web, Mobile, and Video

New Relic

You think you’re ready. You’ve doubled your cloud infrastructure. You’ve redesigned web pages and launched new mobile apps and OTT channels. Then that big event happens or a story goes viral. With all eyes on your website, what will your users see? Ads and great content? Or errors and buffering? From your CMS to the CDN, from the webpage to the media player, New Relic provides end-to-end monitoring and analytics that let you to connect your authors to your audience with bufferless video and audio performance. Learn the details on how the New Relic Digital Intelligence Platform can help you quickly identify, isolate, and fix problems—whether they reside in backend code, external services or scripts, a third-party mobile app, or an ad provider.

Choosing a Citrix Monitoring Strategy: Key Capabilities and Pitfalls to Avoid

eG Innovations

Citrix performance monitoring has been gaining a lot of attention and interest. In this presentation, we outline key requirements that any Citrix monitoring solution should support. We analyze the built-in Citrix monitoring tools: Citrix Director, Citrix NetScaler Insight and the Microsoft SCOM management packs and discuss their capabilities and limitations. The need to have at least three different consoles for monitoring a Citrix infrastructure makes monitoring and diagnosis very inefficient. We discuss how the eG Enterprise solution integrates with the built-in Citrix tools and provides 360 degree unified monitoring for a Citrix infrastructure with automated root-cause diagnosis.

Realtime classroom analytics powered by apache druid

Karthik Deivasigamani

At Noon – The Social Learning Platform, on a daily basis we process close to 100M audio, sketch samples from more than 80K students to help measure the voice & sketch quality of our online classrooms. This talk explores the need for real time analytics in EdTech, how we built a real time analytics platform on Apache Druid & Apache Flink to provide realtime feedback on classroom quality & engagement metrics. We will also share some of the lessons we learnt along the way.

How to consolidate Citrix Monitoring in a Single Pane of Glass

eG Innovations

A recent survey by eG Innovations and xenappblog found that 68% of organizations are using 2-5 different tools for monitoring and managing their Citrix infrastructure. Multiple monitoring tools make it expensive to operate and troubleshoot IT infrastructure issues. Furthermore, a lot of manual effort is required to diagnose and fix performance issues. Join Richard Faulkner, Enterprise Solutions Architect and CTP from Conversant Group, and John Worthington, Director of Customer Success at eG Innovations, and learn how you can get a single-pane-of-glass view of your Citrix infrastructure – from the client end to the virtual desktops/apps and even the backend applications. See how you can: --Monitor and get proactive alerts on the experience seen by Citrix users --Track the performance of every layer and every tier of your Citrix infrastructure: NetScalers, StoreFronts, Virtual apps and desktops, WEM, PVS, License servers, etc. --Troubleshoot in a single click and identify where the root-cause of a problem is: network, or storage, or virtualization, or the Citrix stack? --Get insights to right-size and optimize your Citrix deployment

Node.js BFFs - our way to the better/micro frontends

Eugene Fidelin

To be able to implement new features in as little time as possible, we migrated our monolithic Java-based frontends towards smaller Node.js BFFs (backend-for-frontend, BFF pattern - create separate backend services to be consumed by specific frontend applications). Now we are close to the next step in our journey and adopt Micro-frontends architecture to build our web apps as a composition of even smaller features which are owned by independent teams

Geode Meetup Apachecon

upthewaterspout

Bitmovin LIVE Tech Talks: Analytics for Workflow Automation (ft. Touchstream ...

Bitmovin Inc

As part of Bitmovin's NAB 2020 Virtual event series, we were joined by live video monitoring solutions provider Touchstream Media and had the chance to discuss how live-streaming organizations (such as Sports broadcasters) should automate analytics and data to best improve your video workflows. View our on-demand discussion featuring case studies from a few major sports broadcasters: https://go.bitmovin.com/techtalk-live-analytics-automation-touchstream?utm_source=slideshare

DRM Workflows: How to Provide Protected Content to Desktop, Mobile, TVs, & St...

Bitmovin Inc

DRM is a necessary evil for premium content providers, but with the recent streaming formats and standards, it gets rather easy to create DRM-protected streaming systems. MPEG-DASH with Common Encryption (CENC), together with HLS and FairPlay, is the state-of-the-art DRM approach to reach the majority of platforms today. This hands-on workshop focuses on the common approaches to content protection in 2017.

Sm west 2010-microsoft-workshopPrashant Ohal

Pivotal Greenplum Cloud Marketplaces - Greenplum Summit 2019

VMware Tanzu

Video Recommendation Engines as a Service

Kamil Sindi

ShowNTell: An easy-to-use tool for answering students’ questions with voice-o...

Anand Bhojan

The use of recording applications for teaching is increasingly popular, with tools such as Ink2Go and ShowMe being available on many platforms and at reasonable cost. However, most recording applications available are typically native applications and do not work within the web browser. In this work, we study the feasibility of implementing a recording solution in the web browser environment. The design details of ShowNTell can be found in the paper (http://www.comp.nus.edu.sg/~bhojan/papers/ShowNTell15.pdf) published in IEEE ASEE 45th Frontiers In Education 2015 Conference. Please use it for all the references of this ShowNTell project. Poster: http://www.comp.nus.edu.sg/~bhojan/papers/showNtell15.png Details and Demo @ www.sntboard.com NB:- There are new features such as real-time collaborative drawing and non-linear editing are added in the latest versions. Register at www.sntboard.com to get regular updates. "The greatest challenge of education in the digital world is to train the mind to focus with a strong feeling of purpose where the time shrinks and to re-focus at will" - Dr Bhojan Anand

Live Streaming from A-ZBrightcove

Recently uploaded

Designing Great Products: The Power of Design and Leadership by Chief Designe...

Product School

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

UiPath Test Automation using UiPath Test Suite series, part 4

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap. The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies. Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques What will you get from this session? 1. Insights into SAP testing best practices 2. Heatmap utilization for testing 3. Optimization of testing processes 4. Demo Topics covered: Execution from the test manager Orchestrator execution result Defect reporting SAP heatmap example with demo Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Product School

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

Product School

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

"Impact of front-end architecture on development cost", Viktor Turskyi

Fwdays

I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

Bits & Pixels using AI for Good.........

Alison B. Lowndes

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

Abida Shariff

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...

Thierry Lestable

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

Connector Corner: Automate dynamic content and events by pushing a button

DianaGray10

Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to: Create a campaign using Mailchimp with merge tags/fields Send an interactive Slack channel message (using buttons) Have the message received by managers and peers along with a test email for review But there’s more: In a second workflow supporting the same use case, you’ll see: Your campaign sent to target colleagues for approval If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team But—if the “Reject” button is pushed, colleagues will be alerted via Slack message Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors. And... Speakers: Akshay Agnihotri, Product Manager Charlie Greenberg, Host

Leading Change strategies and insights for effective change management pdf 1.pdf

OnBoard

Mission to Decommission: Importance of Decommissioning Products to Increase E...

Product School

DevOps and Testing slides at DASA Connect

Kari Kakkonen

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

Recently uploaded (20)