Web Scale Applications using NeflixOSS Cloud Platform. Infographics on IaaS, PaaS, SaaS. Commandments of developing a cloud based distributed application.
Architecting for the Cloud using NetflixOSS - Codemash WorkshopSudhir Tonse
Cloud development is inherently different than data center development. Understanding those differences, and architecting for them is critical to successful cloud solutions. In this workshop, we will both describe Netflix OSS platform components and show you how you can piece them together to build your own fault-tolerant REST services. These include: Hystrix, Ribbon, Eureka, and Archaius. In this hands-on lab, you will both learn the benefits of each of these services and use them in a sample application (in a test account). If you want to get things running in your own account, you may want to attend the afternoon session (Setting up your environment for the AWS cloud).
AWS Re:Invent - Optimizing Costs with AWSCoburn Watson
AWS Re:Invent 2012 presentation from Netflix which covers how to optimize cost and usage of your AWS resources. Areas of focus are Autoscaling EC2 instances, batch access of SQS, and improved S3 usage.
Netflix Cloud Platform Building BlocksSudhir Tonse
Architectural Building Blocks of the Netflix Cloud Platform and lessons learned while implementing the same.
Commandments of Web Scale Cloud Deployments
Join the product and cloud computing leaders of Netflix to discuss why and how the company moved to Amazon Web Services. From early experiments for media transcoding, to building the operational skills to optimize costs and the creation of the Simian Army, this session guides business leaders through real world examples of evaluating and adopting cloud computing.
ARC301 Intro to Chaos Monkey & the Simian Army - AWS re: Invent 2012Amazon Web Services
Join the product and cloud computing leaders of Netflix to discuss why and how the company moved to Amazon Web Services. From early experiments for media transcoding, to building the operational skills to optimize costs and the creation of the Simian Army, this session guides business leaders through real world examples of evaluating and adopting cloud computing.
AWS re:Invent 2016: From Resilience to Ubiquity - #NetflixEverywhere Global A...Amazon Web Services
Building and evolving a pervasive, global service requires a multi-disciplined approach that balances requirements with service availability, latency, data replication, compute capacity, and efficiency. In this session, we’ll follow the Netflix journey of failure, innovation, and ubiquity. We'll review the many facets of globalization and then delve deep into the architectural patterns that enable seamless, multi-region traffic management; reliable, fast data propagation; and efficient service infrastructure. The patterns presented will be broadly applicable to internet services with global aspirations.
Architecting for the Cloud using NetflixOSS - Codemash WorkshopSudhir Tonse
Cloud development is inherently different than data center development. Understanding those differences, and architecting for them is critical to successful cloud solutions. In this workshop, we will both describe Netflix OSS platform components and show you how you can piece them together to build your own fault-tolerant REST services. These include: Hystrix, Ribbon, Eureka, and Archaius. In this hands-on lab, you will both learn the benefits of each of these services and use them in a sample application (in a test account). If you want to get things running in your own account, you may want to attend the afternoon session (Setting up your environment for the AWS cloud).
AWS Re:Invent - Optimizing Costs with AWSCoburn Watson
AWS Re:Invent 2012 presentation from Netflix which covers how to optimize cost and usage of your AWS resources. Areas of focus are Autoscaling EC2 instances, batch access of SQS, and improved S3 usage.
Netflix Cloud Platform Building BlocksSudhir Tonse
Architectural Building Blocks of the Netflix Cloud Platform and lessons learned while implementing the same.
Commandments of Web Scale Cloud Deployments
Join the product and cloud computing leaders of Netflix to discuss why and how the company moved to Amazon Web Services. From early experiments for media transcoding, to building the operational skills to optimize costs and the creation of the Simian Army, this session guides business leaders through real world examples of evaluating and adopting cloud computing.
ARC301 Intro to Chaos Monkey & the Simian Army - AWS re: Invent 2012Amazon Web Services
Join the product and cloud computing leaders of Netflix to discuss why and how the company moved to Amazon Web Services. From early experiments for media transcoding, to building the operational skills to optimize costs and the creation of the Simian Army, this session guides business leaders through real world examples of evaluating and adopting cloud computing.
AWS re:Invent 2016: From Resilience to Ubiquity - #NetflixEverywhere Global A...Amazon Web Services
Building and evolving a pervasive, global service requires a multi-disciplined approach that balances requirements with service availability, latency, data replication, compute capacity, and efficiency. In this session, we’ll follow the Netflix journey of failure, innovation, and ubiquity. We'll review the many facets of globalization and then delve deep into the architectural patterns that enable seamless, multi-region traffic management; reliable, fast data propagation; and efficient service infrastructure. The patterns presented will be broadly applicable to internet services with global aspirations.
MicroServices at Netflix - challenges of scaleSudhir Tonse
MicroServices has caught on as the design pattern of choice for many companies at scale. While MicroServices and SOA in general have many positives compared to Monolithic apps, it does come with its own challenges - especially when running at scale. These slides were for a 15 min Meetup talk hosted at Cisco
Intro to Batch Processing on AWS - DevDay Los Angeles 2017Amazon Web Services
What to expect:
• Batch processing – overview and challenges
• Why run batch workloads in the cloud
• Overview of AWS batch solutions
• Deep dive look at AWS Batch and Amazon ECS
• Best practices
AWS re:Invent 2016: Another Day in the Life of a Netflix Engineer (DEV209)Amazon Web Services
Netflix is big. Really big. You just won't believe how vastly, hugely, mind-bogglingly big it is.
Netflix is a large, ever changing, ecosystem system serving million of customers across the globe through cloud-based systems and a globally distributed CDN. This entertaining romp through the tech stack serves as an introduction to how we think about and design systems, the Netflix approach to operational challenges, and how other organizations can apply our thought processes and technologies.
We’ll talk about:
The Bits - The technologies used to run a global streaming company
Making the Bits Bigger - Scaling at scale
Keeping an Eye Out - Billions of metrics
Break all the Things - Chaos in production is key
DevOps - How culture affects your velocity and uptime
Build a Serverless Web Application in One Day Workshop - DevDay Los Angeles 2017Amazon Web Services
What to Expect from the Session:
• Overview of Serverless Computing
• Introduction to AWS Services used in the Workshop
• Outline of the Workshop Scenario
• Preview of the Labs
AWS re:Invent 2016: Building a Platform for Collaborative Scientific Research...Amazon Web Services
This session discusses the architecture, formation, and usage of a collaborative HPC/big data scientific research and analysis environment on AWS. The pharmaceutical industry trend toward joint ventures and collaborations has created a need for new platforms in which to work together. We'll dive into architectural decisions for building collaborative systems. Examples include how such a platform allowed Human Longevity, Inc. to accelerate software deployment to production in a fast-paced research environment, and how Celgene uses AWS for research collaboration with outside universities and foundations.
Deep Dive with Amazon EC2 Container Service Hands-on WorkshopAmazon Web Services
This is an advanced workshop for Amazon ECS. In this workshop you will learn:
How to provision your Amazon ECS with CloudFormation
Amazon ECS with Windows Container
Amazon ECS CI/CD
Amazon ECS service autoscaling and host autoscaling design pattern and best practices
Amazon ECS log consolidation design patterns
Secure credential management with IAM and EC2 Parameter Store
Amazon ECS Events and design patterns
Service Discovery with fully-managed etcd3 cluster on Amazon ECS
Building HPC Clusters as Code in the (Almost) Infinite Cloud | AWS Public Sec...Amazon Web Services
Every day, the computing power of high-performance computing (HPC) clusters helps scientists make breakthroughs, such as proving the existence of gravitational waves and screening new compounds for new drugs. Yet building HPC clusters is out of reach for most organizations, due to the upfront hardware costs and ongoing operational expenses. Now the speed of innovation is only bound by your imagination, not your budget. Researchers can run one cluster for 10,000 hours or 10,000 clusters for one hour anytime, from anywhere, and both cost the same in the cloud. And with the availability of Public Data Sets in Amazon S3, petabyte scale data is instantly accessible in the cloud. Attend and learn how to build HPC clusters on the fly, leverage Amazon’s Spot market pricing to minimize the cost of HPC jobs, and scale HPC jobs on a small budget, using all the same tools you use today, and a few new ones too.
Building and Scaling a Containerized Microservice - DevDay Los Angeles 2017Amazon Web Services
From monolith to microservices, you'll learn to build and scale your first containerized microservice on AWS. We'll cover, microservices architecture, Amazon ECS, Task Placement and twelve-factor app with Amazon ECS.
Distributed Serverless Stack Tracing and Monitoring - DevDay Los Angeles 2017Amazon Web Services
What to Expect:
- AWS Lambda concepts
- Common Serverless use cases
- Monolithic vs. service oriented architecture
- Debugging applications with AWS Cloudwatch and AWS X-Ray
AWS re:Invent 2016: Running Batch Jobs on Amazon ECS (CON310)Amazon Web Services
Batch computing is a common way for developers, scientists and engineers to run a series of jobs on a large pool of shared compute resources, such as servers, virtual machines, and containers. Amazon ECS makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. In this session will show you how to run batch jobs using Amazon ECS and together with other AWS services, such as AWS Lambda and Amazon SQS. We will see how you can leverage Amazon EC2 Spot Instances to power your ECS cluster and easily scale your batch workloads. You'll hear from Mapbox on how they use ECS to power their entire batch processing architecture to collect and process over 100 million miles of sensor data per day that they use for powering their maps. Mapbox will also discuss how they optimize their batch processing framework on ECS using Spot Instances and demo their open source framework that will help you get up and running with ECS in minutes.
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...Amazon Web Services
This session is a technical journey through application migration and refactoring using containerized technologies. Flux 7 recently worked with Rent-a-Center to perform a Hybris migration from their datacenter to AWS and you can hear how they used Amazon ECS, the new Application Load Balancer, and Auto Scaling to meet the customers' business objectives.
Netflix's architecture for viewing data has evolved as streaming usage has grown. Each generation was designed for the next order of magnitude, and was informed by learnings from the previous. From SQL to NoSQL, from data center to cloud, from proprietary to open source, look inside to learn how this system has evolved. (from talk given at QConSF 2014)
AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS ...Amazon Web Services
Deep learning is an implementation of machine learning that uses neural networks to solve difficult and complex problems, such as computer vision, natural language processing, and recommendations. Due to the availability of deep learning libraries and frameworks, developers have the ability to enhance the capabilities of their applications and projects. In this workshop, you learn how to build and deploy a powerful deep learning framework called MXNet on containers. The portability and resource management benefit of containers means developers can focus less on infrastructure and more on building.
The labs start by demonstrating the automation capabilities of AWS CloudFormation to stand up core infrastructure; as an added bonus, you use Spot Fleet to leverage the cost benefits of using Spot Instances, especially for developer environments. Then, you walk through creating an MXNet container in Docker and deploying it with Amazon ECS. Finally, you walk through an image classification demo of MXNet to validate that everything is working as expected.
All you need to participate is a laptop and AWS account.
Machine Learning is increasingly being used by organisations to move from analysis to prediction. How AWS and open source technology can help you to perform both Deep Learning and Machine Learning
Introduction to the Netflix Open Source Software project, explains why Netflix is doing this, how all the parts fit together and what is planned to come next. Presented at the inaugural NetflixOSS Meetup February 6th 2013 at Netflix headquarters in Los Gatos.
Open Business Conference: Continuous Delivery At Netflix -- Powered by Open S...Dianne Marsh
How can we engineer velocity by weaving together tooling and culture with software
development to expose and elevate highly effective practices? This talk describes various
components of Netflix’s continuous delivery platform -- much of which is available in open
source. I’ll show how these pieces fit together and allow us to build scaffolding so that we’re comfortable with software developers making the decision to push the button for prod deployment -- and helps them to recover if necessary. As a result, we can run fast, trusting our tooling and our culture. I’ll also describe how we test our resiliency through simulating failure, unleashing the monkeys (Simian Army) on our production environment. Because if you’re afraid of cute little monkeys, imagine how afraid you’ll be of a production environment that offers those same risks but doesn’t give you an opportunity to test your response to those dangers.
Throughout this talk, I hope that you will challenge yourself to consider how your company can "shift the curve" through tooling and to achieve a high velocity environment without negatively impacting reliability.
How Netflix tests in production to augment more traditional testing methods. This talk covers the Simian Army (Chaos Monkey & friends, code coverage in production, and canary testing.
MicroServices at Netflix - challenges of scaleSudhir Tonse
MicroServices has caught on as the design pattern of choice for many companies at scale. While MicroServices and SOA in general have many positives compared to Monolithic apps, it does come with its own challenges - especially when running at scale. These slides were for a 15 min Meetup talk hosted at Cisco
Intro to Batch Processing on AWS - DevDay Los Angeles 2017Amazon Web Services
What to expect:
• Batch processing – overview and challenges
• Why run batch workloads in the cloud
• Overview of AWS batch solutions
• Deep dive look at AWS Batch and Amazon ECS
• Best practices
AWS re:Invent 2016: Another Day in the Life of a Netflix Engineer (DEV209)Amazon Web Services
Netflix is big. Really big. You just won't believe how vastly, hugely, mind-bogglingly big it is.
Netflix is a large, ever changing, ecosystem system serving million of customers across the globe through cloud-based systems and a globally distributed CDN. This entertaining romp through the tech stack serves as an introduction to how we think about and design systems, the Netflix approach to operational challenges, and how other organizations can apply our thought processes and technologies.
We’ll talk about:
The Bits - The technologies used to run a global streaming company
Making the Bits Bigger - Scaling at scale
Keeping an Eye Out - Billions of metrics
Break all the Things - Chaos in production is key
DevOps - How culture affects your velocity and uptime
Build a Serverless Web Application in One Day Workshop - DevDay Los Angeles 2017Amazon Web Services
What to Expect from the Session:
• Overview of Serverless Computing
• Introduction to AWS Services used in the Workshop
• Outline of the Workshop Scenario
• Preview of the Labs
AWS re:Invent 2016: Building a Platform for Collaborative Scientific Research...Amazon Web Services
This session discusses the architecture, formation, and usage of a collaborative HPC/big data scientific research and analysis environment on AWS. The pharmaceutical industry trend toward joint ventures and collaborations has created a need for new platforms in which to work together. We'll dive into architectural decisions for building collaborative systems. Examples include how such a platform allowed Human Longevity, Inc. to accelerate software deployment to production in a fast-paced research environment, and how Celgene uses AWS for research collaboration with outside universities and foundations.
Deep Dive with Amazon EC2 Container Service Hands-on WorkshopAmazon Web Services
This is an advanced workshop for Amazon ECS. In this workshop you will learn:
How to provision your Amazon ECS with CloudFormation
Amazon ECS with Windows Container
Amazon ECS CI/CD
Amazon ECS service autoscaling and host autoscaling design pattern and best practices
Amazon ECS log consolidation design patterns
Secure credential management with IAM and EC2 Parameter Store
Amazon ECS Events and design patterns
Service Discovery with fully-managed etcd3 cluster on Amazon ECS
Building HPC Clusters as Code in the (Almost) Infinite Cloud | AWS Public Sec...Amazon Web Services
Every day, the computing power of high-performance computing (HPC) clusters helps scientists make breakthroughs, such as proving the existence of gravitational waves and screening new compounds for new drugs. Yet building HPC clusters is out of reach for most organizations, due to the upfront hardware costs and ongoing operational expenses. Now the speed of innovation is only bound by your imagination, not your budget. Researchers can run one cluster for 10,000 hours or 10,000 clusters for one hour anytime, from anywhere, and both cost the same in the cloud. And with the availability of Public Data Sets in Amazon S3, petabyte scale data is instantly accessible in the cloud. Attend and learn how to build HPC clusters on the fly, leverage Amazon’s Spot market pricing to minimize the cost of HPC jobs, and scale HPC jobs on a small budget, using all the same tools you use today, and a few new ones too.
Building and Scaling a Containerized Microservice - DevDay Los Angeles 2017Amazon Web Services
From monolith to microservices, you'll learn to build and scale your first containerized microservice on AWS. We'll cover, microservices architecture, Amazon ECS, Task Placement and twelve-factor app with Amazon ECS.
Distributed Serverless Stack Tracing and Monitoring - DevDay Los Angeles 2017Amazon Web Services
What to Expect:
- AWS Lambda concepts
- Common Serverless use cases
- Monolithic vs. service oriented architecture
- Debugging applications with AWS Cloudwatch and AWS X-Ray
AWS re:Invent 2016: Running Batch Jobs on Amazon ECS (CON310)Amazon Web Services
Batch computing is a common way for developers, scientists and engineers to run a series of jobs on a large pool of shared compute resources, such as servers, virtual machines, and containers. Amazon ECS makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. In this session will show you how to run batch jobs using Amazon ECS and together with other AWS services, such as AWS Lambda and Amazon SQS. We will see how you can leverage Amazon EC2 Spot Instances to power your ECS cluster and easily scale your batch workloads. You'll hear from Mapbox on how they use ECS to power their entire batch processing architecture to collect and process over 100 million miles of sensor data per day that they use for powering their maps. Mapbox will also discuss how they optimize their batch processing framework on ECS using Spot Instances and demo their open source framework that will help you get up and running with ECS in minutes.
AWS re:Invent 2016: Get Technically Inspired by Container-Powered Migrations ...Amazon Web Services
This session is a technical journey through application migration and refactoring using containerized technologies. Flux 7 recently worked with Rent-a-Center to perform a Hybris migration from their datacenter to AWS and you can hear how they used Amazon ECS, the new Application Load Balancer, and Auto Scaling to meet the customers' business objectives.
Netflix's architecture for viewing data has evolved as streaming usage has grown. Each generation was designed for the next order of magnitude, and was informed by learnings from the previous. From SQL to NoSQL, from data center to cloud, from proprietary to open source, look inside to learn how this system has evolved. (from talk given at QConSF 2014)
AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS ...Amazon Web Services
Deep learning is an implementation of machine learning that uses neural networks to solve difficult and complex problems, such as computer vision, natural language processing, and recommendations. Due to the availability of deep learning libraries and frameworks, developers have the ability to enhance the capabilities of their applications and projects. In this workshop, you learn how to build and deploy a powerful deep learning framework called MXNet on containers. The portability and resource management benefit of containers means developers can focus less on infrastructure and more on building.
The labs start by demonstrating the automation capabilities of AWS CloudFormation to stand up core infrastructure; as an added bonus, you use Spot Fleet to leverage the cost benefits of using Spot Instances, especially for developer environments. Then, you walk through creating an MXNet container in Docker and deploying it with Amazon ECS. Finally, you walk through an image classification demo of MXNet to validate that everything is working as expected.
All you need to participate is a laptop and AWS account.
Machine Learning is increasingly being used by organisations to move from analysis to prediction. How AWS and open source technology can help you to perform both Deep Learning and Machine Learning
Introduction to the Netflix Open Source Software project, explains why Netflix is doing this, how all the parts fit together and what is planned to come next. Presented at the inaugural NetflixOSS Meetup February 6th 2013 at Netflix headquarters in Los Gatos.
Open Business Conference: Continuous Delivery At Netflix -- Powered by Open S...Dianne Marsh
How can we engineer velocity by weaving together tooling and culture with software
development to expose and elevate highly effective practices? This talk describes various
components of Netflix’s continuous delivery platform -- much of which is available in open
source. I’ll show how these pieces fit together and allow us to build scaffolding so that we’re comfortable with software developers making the decision to push the button for prod deployment -- and helps them to recover if necessary. As a result, we can run fast, trusting our tooling and our culture. I’ll also describe how we test our resiliency through simulating failure, unleashing the monkeys (Simian Army) on our production environment. Because if you’re afraid of cute little monkeys, imagine how afraid you’ll be of a production environment that offers those same risks but doesn’t give you an opportunity to test your response to those dangers.
Throughout this talk, I hope that you will challenge yourself to consider how your company can "shift the curve" through tooling and to achieve a high velocity environment without negatively impacting reliability.
How Netflix tests in production to augment more traditional testing methods. This talk covers the Simian Army (Chaos Monkey & friends, code coverage in production, and canary testing.
The Journey of Chaos Engineering Begins with a Single StepBruce Wong
PagerDuty Summit 2016
Presenters: Bruce Wong, James Burns
https://www.pagerduty.com/pagerduty-summit-2016/
Heard of Netflix' Chaos Engineering & the Simian Army? Google's legendary DiRT exercises? Hear about how Twilio is getting started on its journey with Chaos Engineering. This talk is the story of how Twilio got started with Chaos Engineering, lessons learned, and the impact to our engineering culture.
Embracing Failure - Fault Injection and Service Resilience at NetflixJosh Evans
A presentation given at AWS re:Invent on how Netflix induces failure to validate and harden production systems. Technologies discussed include the Simian Army (Chaos Monkey, Gorilla, Kong) and our next gen Failure Injection Test framework (FIT).
Presented at ISSA CISO Executive forum 2012
Comments/Questions: bill.burns@netflix.com
(3/8: Replaced Keynote for PDF version for compatibility)
An adjunct to Jason Chan's Practical Cloud Security preso: http://www.slideshare.net/jason_chan/practical-cloud-security
From resilient to antifragile - Chaos Engineering Primer DevSecConSergiu Bodiu
Can we inject failure scenarios into deployed systems to reduce platform risk? During this talk, demonstrations of the Simian Army, Chaos Lemur and Locust.io tools will be presented. We will go beyond reliability, stability and availability to help your platform operations team build a continuous process improvement program which will prepare your production systems for the unexpected.
92% of catastrophic system failures were the result of incorrect handling of nonfatal errors.
It is simply not possible to fully reproduce the entire architecture and run an end to end test.
Don't trust claims systems make about themselves & their dependencies. Verify by breaking.
How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...Amazon Web Services
You're on the verge of a new startup and you need to build a world-class, high-scale web application on AWS so it can handle millions of users. How do you build it quickly without having to reinvent and re-implement the best-practices of large successful Internet companies? NetflixOSS is your answer. In this session, we’ll cover how an emerging startup can leverage the different open source tools that Netflix has developed and uses every day in production, ranging from baking and deploying applications (Asgard, Aminator), to hardening resiliency to failures (Hystrix, Simian Army, Zuul), making them highly distributed and load balanced (Eureka, Ribbon, Archaius) and managing your AWS resources efficiently and effectively (Edda, Ice). You’ll learn how to get started using these tools, learn best practices from engineers who actually created them, so, like Netflix, you can too unleash the power of AWS and scale your application processes as you grow.
Slides from QConSF Nov 19th, 2011 focusing this time on describing the globally distributed and scaled industrial strength Java Platform as a Service that Netflix has built and run on top of AWS and Cassandra. Parts of that platform are being released as open source - Curator, Priam and Astyanax.
Same basic flow as the keynote, but with a lot more detail, and we had a lot more interactive discussion rather than a presentation format. See part 2 for some more specific detail and links to other presentations.
Netflix
has
built
and
deployed
a
scalable
global
Platorm
as
a
Service.
Key
components
of
the
Netflix
PaaS
are
being
released
as
Open
Source
projects
so
you
can
build
your
own
custom
PaaS
SV Forum Platform Architecture SIG - Netflix Open Source PlatformAdrian Cockcroft
Architecture overview of Netflix Cloud Architecture with a focus on the Open Source components that Netflix has put and is planning to release on http://netflix.github.com
Understand how to architect an infrastructure to handle going from zero to millions of users. From leveraging highly scalable AWS services to making smart decisions on building out your application, you'll learn a number of best practices for scaling your infrastructure in the cloud.
Using apache camel for microservices and integration then deploying and managing on Docker and Kubernetes. When we need to make changes to our app, we can use Fabric8 continuous delivery built on top of Kubernetes and OpenShift.
Slides of QCon London 2016 talk. How stream processing is used within the Uber's Marketplace system to solve a wide range problems, including but not limited to realtime indexing and querying of geospatial time series, aggregation and computing of streaming data, and extracting patterns from data streams. In addition, it will touch upon various TimeSeries analysis and predictions. The underlying systems utilize many open source technologies such as Apache Kafka, Samza and Spark streaming.
Pros and Cons of a MicroServices Architecture talk at AWS ReInventSudhir Tonse
Netflix morphed from a private datacenter based monolithic application into a cloud based Microservices architecture. This talk highlights the pros and cons of building software applications as suites of independently deployable services, as well as practical approaches for overcoming challenges - especially in the context of an elastic but ephemeral cloud ecosystem. What were the lessons learned while building and managing these services? What are the best practices and anti-patterns?
Big Data Pipeline and Analytics Platform Using NetflixOSS and Other Open Sour...Sudhir Tonse
Slides on the OSCON talk about the data platform used at Netflix for event collection, aggregation, and analysis. The platform helps Netflix process and analyze billions of events every day. Attendees will learn how to assemble their own large-scale data pipeline/analytics platform using open source software from NetflixOSS and others, such as Kafka, ElasticSearch, Druid from Metamarkets, and Hive.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Web Scale Applications using NeflixOSS Cloud Platform
1. Building Resilient, Web Scale Applications in the Cloud
IaaS, PaaS
- Sudhir Tonse @stonse
- Manager, Cloud Platform. Netflix LLC
2. Topics
• About Netflix and move to Cloud
• IaaS, PaaS …
• Architectural/Operational Insights
• How can you benefit?
– Various Open Source Offerings (& #cloudprize
$$$)
– Netflix PaaS Tech Blogs
– Build your own PaaS
3. Netflix
Netflix is the world’s leading Internet television
network with more than 33 million members
in 40 countries enjoying more than one billion
hours of TV shows and movies per month,
including original series.
4. Netflix Scale
• 100s of Mid-tier services and applications
• Billions of Requests per day
• ~70 Billion Events per day
• 10,000s of Ec2 Instances in use in multiple AWS
Regions/Zones
• Cassandra NoSQL database in a Global Ring
spanning regions: Terabytes of data
• At peak consumes ~1/3 of US Internet Bandwidth
5. What’s a Cloud?
• Cloud:
Cloud computing is the delivery of computing
and storage capacity [1] as a service [2] to a
heterogeneous community of end-recipients.
Images Courtesy: Wikipedia/Company logos
13. Commandments of a Resilient Web Scale Cloud Architecture
1. Thou shalt embrace stateless fine grained Micro Services (SOA)
2. Thou shalt not store important data on ephemeral instances
3. Thou shalt embrace a resilient homogenous architecture
4. Thou shalt embrace and understand the CAP Theorem
5. Thou shalt guard all external/RPC calls with a fault & latency
tolerance layer
6. Thou shalt embrace auto scaling; scale according to thy needs
7. Thou shalt keep a wary eye on thy cost
8. Thou shalt secure thy data and access points
9. Thou shalt instrument thy code
10. Thou shalt effectively monitor thy instances
11. Thou shalt deploy thy instances in multiple zones and regions for
availability and resiliency
12. Thou shalt be way of SPoF
13. Thou shalt always plan for failure
Note: These are my personal views and may or may not reflect the views of my employers, past or present.
It is not the intention of the author to offend anyone’s religious beliefs or thoughts : - @stonse
17. Build your own PaaS
Mix and match NetflixOSS Tools & Libraries
@adrianco
@rusmeshenberg
18. Platform Services
AWS Account
Asgard Console
Archaius Config
Multiple AWS Regions
Service
Cross region
Priam C* Eureka Registry
Explorers
Dashboards
Exhibitor ZK
3 AWS Zones
Application
Priam Evcache
Atlas Edda History Clusters
Cassandra Memcached
Monitoring Autoscale Groups
Persistent Storage Ephemeral Storage
Instances
Simian Army
Genie Hadoop
Services
@adrianco
19. Step 1
Choice A: Bare bones Web Service Template
$git clone https://github.com/Netflix/karyon.git
$cd karyon
$./gradlew clean build
$./gradlew :karyon-examples:hello-netflix-
oss:jettyRun
Choice B: Full blown Recipe (RSS Reader
Recipe)
$git clone
https://github.com/Netflix/recipes-rss.git
$cd recipes-rss
$./gradlew clean build
20. Step 2
• Customize your App using .war of previous
step as a template
– TIPS (optional)
• Use Ribbon for IPC calls
• Use Hystrix/RxJava for wrapping concurrent calls/IPC
calls (Latency/Fault Tolerance)
21. Step 3
• Bake an AMI (Amazon Machine Image) using
NetflixOSS Aminator
22. Step 4
• Deploy to EC2 using Asgard
Deploy & Manage AWS
Applications/Infrastructure
• TIP: Employ Simian Army to test Resiliency
23. How do you enter? $$
• Get a (free) github account
• Fork github.com/netflix/cloud-prize
• Send us your email address
• Describe and build your entry
Twitter #cloudprize
https://github.com/Netflix/Cloud-
Prize/blob/master/README.rdoc @adrianco
24. Thank You!
• Reach out to me at @stonse
– Yes, we are hiring!
Follow @NetflixOSS for more info
Credits/Attributions
• @adrianco
• @rusmeshenberg
• https://en.wikipedia.org/wiki/
File:Hyperviseur.png
• http://techblog.netflix.com
• http://www.slideshare.net/ne
tflix
• http://netflix.github.io