Rise of the Machines: PHP and IoT - ZendCon 2017Colin O'Dell
The Internet of Things (IoT) is fundamentally changing how we interact with the digital world. In this session we’ll explore the implementation of real examples which bridge the gap between the physical and digital world using PHP: asking Alexa for information within a PHP application; displaying API data on an Arduino-powered display; using PHP to control LEDs on a Raspberry Pi to monitor application uptime; and connecting IR sensors to Slack to see whether a conference room is in use.
Reactive Streams and the Wide World of GroovySteve Pember
The concept of Reactive Streams (aka Reactive Extensions, Reactive Functional Programming, or simply Rx) has become increasingly popular recently, and with good reason. The Reactive Streams specification provides a universal abstraction for asynchronously processing data received across multiple sources (e.g. database, user input, third-party services), and includes mechanisms for controlling the rate at which data is received. This makes it a powerful tool within a Microservice platform. And did we mention that the Groovy lang community is quite involved?
In this talk we’ll explore the various features and concepts of Reactive Streams. We’ll talk about some typical use cases for Rx and more importantly, how to implement them. We’ll focus primarily on RxGroovy and Ratpack, then provide example implementations that show you how to get started with this powerful technique.
OnAndroidConf 2013: Accelerating the Android Platform BuildDavid Rosen
Presented at the OnAndroidConf, October 22 2013, http://www.onandroidconf.com/sessions.html
Abstract:
Optimizing the Android build environment to perform at world-class level is a big challenge for many Android device and chipset makers today. Churning through thousands of platform builds per week requires laser-focus on high-performance infrastructure and tooling. If you’re looking at improving your overall engineering and developer productivity, the software build use case is an obvious area to prioritize.
This technical talk will focus on the following aspects of the Android platform build:
Common Android platform build challenges and opportunities with real-life production references
The various Android build use cases and their needs – full integration and release builds, developer incremental builds
Evolution of the Android build and codebase with trends and statistics
Detailed technical analysis of the Android platform build, highlighting opportunities for improvements
Proposed solutions and technical tricks to optimize an Android software build environment
Twitch Plays Pokémon: Twitch's Chat ArchitectureC4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2hmKFK1.
John Rizzo introduces Twitch's chat's architecture, telling how their engineers investigated and worked through the issues in what turned out to be a make-or-break situation for the company. Filmed at qconsf.com.
John Rizzo is a Senior Software Engineer at Twitch.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1nyhaC6.
Nathan Marz discusses building NoSQL-based data systems that are scalable and easy to reason about. Filmed at qconlondon.com.
Nathan Marz is the creator of many open source projects which are relied upon by over 50 companies around the world, including Cascalog and Storm. Nathan is also working on a book for Manning publications entitled "Big Data: principles and best practices of scalable realtime data systems". Nathan was previously the lead engineer at BackType before being acquired by Twitter in 2011.
Rise of the Machines: PHP and IoT - ZendCon 2017Colin O'Dell
The Internet of Things (IoT) is fundamentally changing how we interact with the digital world. In this session we’ll explore the implementation of real examples which bridge the gap between the physical and digital world using PHP: asking Alexa for information within a PHP application; displaying API data on an Arduino-powered display; using PHP to control LEDs on a Raspberry Pi to monitor application uptime; and connecting IR sensors to Slack to see whether a conference room is in use.
Reactive Streams and the Wide World of GroovySteve Pember
The concept of Reactive Streams (aka Reactive Extensions, Reactive Functional Programming, or simply Rx) has become increasingly popular recently, and with good reason. The Reactive Streams specification provides a universal abstraction for asynchronously processing data received across multiple sources (e.g. database, user input, third-party services), and includes mechanisms for controlling the rate at which data is received. This makes it a powerful tool within a Microservice platform. And did we mention that the Groovy lang community is quite involved?
In this talk we’ll explore the various features and concepts of Reactive Streams. We’ll talk about some typical use cases for Rx and more importantly, how to implement them. We’ll focus primarily on RxGroovy and Ratpack, then provide example implementations that show you how to get started with this powerful technique.
OnAndroidConf 2013: Accelerating the Android Platform BuildDavid Rosen
Presented at the OnAndroidConf, October 22 2013, http://www.onandroidconf.com/sessions.html
Abstract:
Optimizing the Android build environment to perform at world-class level is a big challenge for many Android device and chipset makers today. Churning through thousands of platform builds per week requires laser-focus on high-performance infrastructure and tooling. If you’re looking at improving your overall engineering and developer productivity, the software build use case is an obvious area to prioritize.
This technical talk will focus on the following aspects of the Android platform build:
Common Android platform build challenges and opportunities with real-life production references
The various Android build use cases and their needs – full integration and release builds, developer incremental builds
Evolution of the Android build and codebase with trends and statistics
Detailed technical analysis of the Android platform build, highlighting opportunities for improvements
Proposed solutions and technical tricks to optimize an Android software build environment
Twitch Plays Pokémon: Twitch's Chat ArchitectureC4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2hmKFK1.
John Rizzo introduces Twitch's chat's architecture, telling how their engineers investigated and worked through the issues in what turned out to be a make-or-break situation for the company. Filmed at qconsf.com.
John Rizzo is a Senior Software Engineer at Twitch.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1nyhaC6.
Nathan Marz discusses building NoSQL-based data systems that are scalable and easy to reason about. Filmed at qconlondon.com.
Nathan Marz is the creator of many open source projects which are relied upon by over 50 companies around the world, including Cascalog and Storm. Nathan is also working on a book for Manning publications entitled "Big Data: principles and best practices of scalable realtime data systems". Nathan was previously the lead engineer at BackType before being acquired by Twitter in 2011.
Interactive Data Analysis with Apache Flink @ Flink Meetup in BerlinTill Rohrmann
This talk shows how we can use Apache Flink and Apache Zeppelin to do interactive data analysis. The examples show the usage of FlinkML to solve a linear regression and classification problem.
An introduction to Reactive applications, Reactive Streams, and options for t...Steve Pember
The term “reactive” has lately become a buzzword, with a variety of definitions around the Web. When you hear “reactive,” what do you think of? Reactive Streams? The Reactive Manifesto? ReactJS? These terms may seem unrelated, but they share a common core concept.
Reactive applications and Reactive programming result in flexible, concise, performant code and are a superior alternative to the old, standard thread-based imperative programming model. The Reactive approach has gained popularity recently for one simple reason: we need alternative designs and architectures to meet today’s demands. However, it can be difficult to shift one’s mind to think in Reactive terms due to how accustomed we’ve become to the imperative style.
Stephen Pember explores the various definitions of Reactive and Reactive programming with the goal of providing techniques for building efficient, scalable applications. Steve dives into the key concepts of Reactive Streams and examines some sample implementations—including how ThirdChannel is currently using reactive libraries in production code. Steve looks at some of the open source options available in the JVM—including Reactor, RxJava, and Ratpack—giving you an idea of where to begin with the reactive ecosystem. If Reactive is new to you, this should be an excellent introduction.
Engineering Netflix Global Operations in the CloudJosh Evans
Delivered at re:Invent 2015.
Operating a massively scalable, constantly changing, distributed global service is a daunting task. We innovate at breakneck speed to attract new customers and stay ahead of the competition. This means more features, more experiments, more deployments, more engineers making changes in production environments, and ever-increasing complexity. Simultaneously improving service availability and accelerating rate of change seems impossible on the surface. At Netflix, operations engineering is both a technical and organizational construct designed to accomplish just that by integrating disciplines like continuous delivery, fault injection, regional traffic management, crisis response, best practice automation, and real-time analytics. In this talk, designed for technical leaders seeking a path to operational excellence, we'll explore these disciplines in depth and how they integrate and create competitive advantages.
In this presentation, learn how Agile Infrastructure for OpenStack enables you to quickly stand up a dynamic self-service cloud infrastructure so you can easily take advantage of the flexibility, scalability, and efficiency of OpenStack.
You'll gain a better understanding of how Agile Infrastructure:
* Extends the core values of cloud: scale, guaranteed performance, automation, high availability and efficiency
* Ensures you deploy OpenStack using a process that's repeatable and error free
* Allows you to run production and test/dev workloads on one storage platform
* Provides higher utilization, better performance and more operational efficiency than legacy solutions
Groovy Options for Reactive Applications - Greach 2015Steve Pember
Performance demands placed on the web applications we build have drastically increased over the past few years. The Reactive approach has gained popularity recently for one simple reason: we need alternative designs and architectures to meet today’s demands. Speed is everything.
Reactive applications and Reactive programming are an alternative to the standard thread-based imperative programming model that can result in flexible, concise code. However, it can be difficult to shift one’s mind to think in Reactive terms. It doesn’t help that the descriptions around the web can be contradictory and the library documentation can be obscure.
In this talk, we’ll explore the concepts of Reactive and Reactive Programming. We’ll demonstrate some of the useful Reactive functions and examine some practical implementations – including how we’re currently using Reactive libraries in production code. Most importantly, we’ll look at some of the open source options available to us in the Groovy community, including Reactor, RxJava, and the Java 8 stream API. If Reactive is new to you, this should be an excellent introduction.
Reactive programming and Hystrix fault tolerance by Max MyslyvtsevJavaDayUA
Reactive programming is a new paradigm that provides asynchronous event-based flow control. Java implementation is called rxJava and is being developed by Netfix. They have also released Hystrix — a non-functional layer that provides fault tolerance and latency features which also exposes reactive API.
Monitoring Cassandra: Don't Miss a Thing (Alain Rodriguez, The Last Pickle) |...DataStax
Monitoring is critical to successfully running Apache Cassandra in production. Creating a comprehensive and insightful set of dashboards requires a deep knowledge of Cassandra internals that can be intimidating. Everyone however can benefit from knowing where to start looking and why. So that the next time there is a problem you have the right metrics and knowing which dashboards to look at.
In this talk Alain Rodriguez, Consultant at The Last Pickle, will discuss what to monitor in Apache Cassandra, how, and why. He will present examples from commercial products such as DataDog, and open source systems like Grafana.
About the Speaker
Alain Rodriguez Consultant, The Last Pickle
Alain has been working with Apache Cassandra since version 0.8. He was the first Engineer at teads.tv which had grown to 400+ employees by the time he left. During his time at Teads Alain managed and scaled Cassandra clusters across multiple AWS Regions, fully on his own, taking care of the data modeling as well as the troubleshooting and tuning. Alain frequently contributes to the Apache Cassandra users mailing list.
Springone2gx 2015 Reactive Options for GroovySteve Pember
Reactive applications and Reactive programming are an alternative to the standard thread-based imperative programming model that can result in flexible, concise code. The Reactive approach has gained popularity recently for one simple reason: we need alternative designs and architectures to meet today’s demands. However, it can be difficult to shift one’s mind to think in Reactive terms. It doesn’t help that the descriptions around the web can be contradictory and the library documentation can be obscure. In this talk, we’ll explore the concepts of Reactive and Reactive Programming. We’ll demonstrate some of the useful Reactive functions and examine some practical implementations - including how we’re currently using Reactive libraries in production code. Most importantly, we’ll look at some of the open source options available to us in the Groovy community, including Reactor, RxJava, and Ratpack. If Reactive is new to you, this should be an excellent introduction.
Making Glance tasks work for you - OpenStack Summit May 2015 VancouverBrian Rosmaita
It's not widely known that the OpenStack Images API v2 contains an implementation of a "tasks" API that can be customized by operators to enable asynchronous processing of long-running operations. For example, a deployer might want to enable end users to upload their own custom images ... but only after such images have been approved by some thorough, computation-intensive validation process. The Glance tasks API provides a common interface across OpenStack installations, but allows the implementation of tasks to be customizable to a particular cloud environment. Join Brian Rosmaita, Compute Control Plane Product Manager at Rackspace to see how Glance tasks are being used at Rackspace and to learn how you can use Glance tasks in your OpenStack cloud.
Do you need Ops in your new startup? If not now, then when? And...what is Ops?
Learn how to scale ruby-based distributed software infrastructure in the cloud to serve 4,000 requests per second, handle 400 updates per second, and achieve 99.97% uptime – all while building the product at the speed of light.
Unimpressed? Now try doing the above altogether without the Ops team, while growing your traffic 100x in 6 months and deploying 5-6 times a day!
It could be a dream, but luckily it's a reality that could be yours.
Interactive Data Analysis with Apache Flink @ Flink Meetup in BerlinTill Rohrmann
This talk shows how we can use Apache Flink and Apache Zeppelin to do interactive data analysis. The examples show the usage of FlinkML to solve a linear regression and classification problem.
An introduction to Reactive applications, Reactive Streams, and options for t...Steve Pember
The term “reactive” has lately become a buzzword, with a variety of definitions around the Web. When you hear “reactive,” what do you think of? Reactive Streams? The Reactive Manifesto? ReactJS? These terms may seem unrelated, but they share a common core concept.
Reactive applications and Reactive programming result in flexible, concise, performant code and are a superior alternative to the old, standard thread-based imperative programming model. The Reactive approach has gained popularity recently for one simple reason: we need alternative designs and architectures to meet today’s demands. However, it can be difficult to shift one’s mind to think in Reactive terms due to how accustomed we’ve become to the imperative style.
Stephen Pember explores the various definitions of Reactive and Reactive programming with the goal of providing techniques for building efficient, scalable applications. Steve dives into the key concepts of Reactive Streams and examines some sample implementations—including how ThirdChannel is currently using reactive libraries in production code. Steve looks at some of the open source options available in the JVM—including Reactor, RxJava, and Ratpack—giving you an idea of where to begin with the reactive ecosystem. If Reactive is new to you, this should be an excellent introduction.
Engineering Netflix Global Operations in the CloudJosh Evans
Delivered at re:Invent 2015.
Operating a massively scalable, constantly changing, distributed global service is a daunting task. We innovate at breakneck speed to attract new customers and stay ahead of the competition. This means more features, more experiments, more deployments, more engineers making changes in production environments, and ever-increasing complexity. Simultaneously improving service availability and accelerating rate of change seems impossible on the surface. At Netflix, operations engineering is both a technical and organizational construct designed to accomplish just that by integrating disciplines like continuous delivery, fault injection, regional traffic management, crisis response, best practice automation, and real-time analytics. In this talk, designed for technical leaders seeking a path to operational excellence, we'll explore these disciplines in depth and how they integrate and create competitive advantages.
In this presentation, learn how Agile Infrastructure for OpenStack enables you to quickly stand up a dynamic self-service cloud infrastructure so you can easily take advantage of the flexibility, scalability, and efficiency of OpenStack.
You'll gain a better understanding of how Agile Infrastructure:
* Extends the core values of cloud: scale, guaranteed performance, automation, high availability and efficiency
* Ensures you deploy OpenStack using a process that's repeatable and error free
* Allows you to run production and test/dev workloads on one storage platform
* Provides higher utilization, better performance and more operational efficiency than legacy solutions
Groovy Options for Reactive Applications - Greach 2015Steve Pember
Performance demands placed on the web applications we build have drastically increased over the past few years. The Reactive approach has gained popularity recently for one simple reason: we need alternative designs and architectures to meet today’s demands. Speed is everything.
Reactive applications and Reactive programming are an alternative to the standard thread-based imperative programming model that can result in flexible, concise code. However, it can be difficult to shift one’s mind to think in Reactive terms. It doesn’t help that the descriptions around the web can be contradictory and the library documentation can be obscure.
In this talk, we’ll explore the concepts of Reactive and Reactive Programming. We’ll demonstrate some of the useful Reactive functions and examine some practical implementations – including how we’re currently using Reactive libraries in production code. Most importantly, we’ll look at some of the open source options available to us in the Groovy community, including Reactor, RxJava, and the Java 8 stream API. If Reactive is new to you, this should be an excellent introduction.
Reactive programming and Hystrix fault tolerance by Max MyslyvtsevJavaDayUA
Reactive programming is a new paradigm that provides asynchronous event-based flow control. Java implementation is called rxJava and is being developed by Netfix. They have also released Hystrix — a non-functional layer that provides fault tolerance and latency features which also exposes reactive API.
Monitoring Cassandra: Don't Miss a Thing (Alain Rodriguez, The Last Pickle) |...DataStax
Monitoring is critical to successfully running Apache Cassandra in production. Creating a comprehensive and insightful set of dashboards requires a deep knowledge of Cassandra internals that can be intimidating. Everyone however can benefit from knowing where to start looking and why. So that the next time there is a problem you have the right metrics and knowing which dashboards to look at.
In this talk Alain Rodriguez, Consultant at The Last Pickle, will discuss what to monitor in Apache Cassandra, how, and why. He will present examples from commercial products such as DataDog, and open source systems like Grafana.
About the Speaker
Alain Rodriguez Consultant, The Last Pickle
Alain has been working with Apache Cassandra since version 0.8. He was the first Engineer at teads.tv which had grown to 400+ employees by the time he left. During his time at Teads Alain managed and scaled Cassandra clusters across multiple AWS Regions, fully on his own, taking care of the data modeling as well as the troubleshooting and tuning. Alain frequently contributes to the Apache Cassandra users mailing list.
Springone2gx 2015 Reactive Options for GroovySteve Pember
Reactive applications and Reactive programming are an alternative to the standard thread-based imperative programming model that can result in flexible, concise code. The Reactive approach has gained popularity recently for one simple reason: we need alternative designs and architectures to meet today’s demands. However, it can be difficult to shift one’s mind to think in Reactive terms. It doesn’t help that the descriptions around the web can be contradictory and the library documentation can be obscure. In this talk, we’ll explore the concepts of Reactive and Reactive Programming. We’ll demonstrate some of the useful Reactive functions and examine some practical implementations - including how we’re currently using Reactive libraries in production code. Most importantly, we’ll look at some of the open source options available to us in the Groovy community, including Reactor, RxJava, and Ratpack. If Reactive is new to you, this should be an excellent introduction.
Making Glance tasks work for you - OpenStack Summit May 2015 VancouverBrian Rosmaita
It's not widely known that the OpenStack Images API v2 contains an implementation of a "tasks" API that can be customized by operators to enable asynchronous processing of long-running operations. For example, a deployer might want to enable end users to upload their own custom images ... but only after such images have been approved by some thorough, computation-intensive validation process. The Glance tasks API provides a common interface across OpenStack installations, but allows the implementation of tasks to be customizable to a particular cloud environment. Join Brian Rosmaita, Compute Control Plane Product Manager at Rackspace to see how Glance tasks are being used at Rackspace and to learn how you can use Glance tasks in your OpenStack cloud.
Do you need Ops in your new startup? If not now, then when? And...what is Ops?
Learn how to scale ruby-based distributed software infrastructure in the cloud to serve 4,000 requests per second, handle 400 updates per second, and achieve 99.97% uptime – all while building the product at the speed of light.
Unimpressed? Now try doing the above altogether without the Ops team, while growing your traffic 100x in 6 months and deploying 5-6 times a day!
It could be a dream, but luckily it's a reality that could be yours.
Operational Insight: Concepts and Examples (w/o Presenter Notes)royrapoport
The 2015-06-15 Operational Insight presentation, without presenter notes (because the way Keynote handles presenter notes makes them dominate the presentation)
Keeping Movies Running Amid Thunderstorms!Sid Anand
How does Netflix strive to deliver an uninterrupted service? This talk, delivered for the first time in November, 2011, covers some engineering design concepts that help us deliver features at a rapid pace while assuring high availability.
Same basic flow as the keynote, but with a lot more detail, and we had a lot more interactive discussion rather than a presentation format. See part 2 for some more specific detail and links to other presentations.
SV Forum Platform Architecture SIG - Netflix Open Source PlatformAdrian Cockcroft
Architecture overview of Netflix Cloud Architecture with a focus on the Open Source components that Netflix has put and is planning to release on http://netflix.github.com
For the Computer Measurement Group workshop in San Diego November 2013. Also presented to a student class at UC Santa Barbara. What is Cloud Native. Capacity and Performance benchmarks. Cost Optimization Techniques - content co-developed with Jinesh Varia of AWS.
I take our currently implemented real-time analytics platform which makes decisions and takes autonomous action within our environment and repurpose it for a hypothetical solution to a phishing problem at a hypothetical startup.
This presentatiom provides a method of mathematical representation of the traffic flow of network states. Anomalous behavior in this model is represented as a point, not grouped in clusters allocated by the "alpha-stream" process
Summary of past Cassandra benchmarks performed by Netflix and description of how Netflix uses Cassandra interspersed with a live demo automated using Jenkins and Jmeter that created two 12 node Cassandra clusters from scratch on AWS, one with regular disks and one with SSDs. Both clusters were scaled up to 24 nodes each during the demo.
The Land Sharks are on the Squawk Box (Where Did Postgres Come From?)EDB
This Turing Award talk intermixes a bicycle ride across America during the summer of 1988 with the design, construction and commercialization of Postgres during the late 80’s and early ‘90’s. Striking parallels are observed, leading to a discussion of what it takes to build a new DBMS. Also, indicated are the roles that perseverance and serendipity played in both endeavors.
Integrating RightScale, Windows, and .NET for Fun and Profit - RightScale Com...RightScale
Speakers:
Patrick McClory - Solutions Architect, RightScale
Patrick Moore - Operations Manager, Koupon Media
Gordon Bailey - Lead Infrastructure Engineer, Koupon Media
RightScale offers a robust REST-based API and, while it is available to all, having platform-specific clients certainly helps in the process of consuming and working with any given programming interface. We’ll demonstrate the RightScale .NET API 1.5 wrapper project with a few examples of how it can help to automate your process, integrate with existing tools, and even make your life easier when implemented within your daily operating applications.
Complex hierarchical relationships between entities can only be mapped with difficulty in a relational database and demanding queries are usually quite slow.
Graph databases are optimized for exactly these kinds of relationships and can provide high-performance results even with huge amounts of data. Moreover, not only the entities that are stored in the database, have attributes, but also their relationships. Queries can look at entities as well as their relationships.
Get to know the basics of graph databases, using Neo4j as an example, and see how it is used C# projects.
The world is not black and white – Impact of decisions over the lifetime of a...Eric Reiche
"Walking on water and developing software from a specification are easy if both are frozen." - Edward V. Berard
Decisions are context dependent. A right decision at the beginning may seem like a wrong decision 4 years later. This talk will compare the effect of architecture decisions of a small versus a mature project, how to make the "right" decision, and how to deal with problems when the right decision isn't right anymore.
KiZAN will bring 25 Raspberry Pi starter kits that run Windows 10 IoT Core. This will enable participants to build a really compelling IoT/Azure/Power BI story in a single day! Interet of Things (IoT) Raspberry Pi starter kit
We’ll start off the day with an introduction to IoT and build IoT devices (hands on). Next, we’ll build a simple temperature sensor, collecting ambient temperature readings, and stream the data to an Azure IoT Hub.
Once the data is in Azure, we’ll analyze it with Azure Stream Analytics, and ship it to an Azure SQL Database.
Finally, we’ll report on the data and build dashboards of our temperature readings using Power BI.
AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...Amazon Web Services
As systems and user bases grow, a once abundant resource can become scarce. While scaling out PlayStation services to millions of users at over a 100,000 requests/second, network throughput became a precious resource to optimize for. Alex and Dustin talk about how the microservices that power Playstation achieved low latency interactions while conserving on precious network bandwidth. These services powered by Amazon Elastic Load Balancing and Amazon DynamoDB benefitted from soft-state optimizations, a pattern that is used in complex interactions such as searching through a user’s social graph in sub 100 ms, or a user’s game library in 7 ms. As a developer utilizing Amazon Web services, you will discover new patterns and implementations which will better utilize your network, instances, and load balancers in order to deliver personalized experiences to millions of users while saving costs.
Abstract:
Cassandra is a new kind of database: it is more than a single-machine system. It naturally runs in a High-Availability configuration. All nodes in the system are symmetric; there is no single point of failure. As you add machines, failure becomes routine, and Cassandra is built to tolerate that with no interruptions.
Cassandra is linearly scalable with good performance characteristics for very small and very large data stores. Unlike earlier efforts, Cassandra is more than just a key-value store; it is a structured data store which can facilitate complex use cases and queries. Cassandra allows for random access to your data organized into rows and columns.
Cassandra is different, and exciting. This presentation will discuss the pros and cons of using Cassandra, and why it has seen such amazing adoption in the past year.
Bio:
Ben Coverston is Director of Operations at DataStax (formerly knows as Riptano), a provider of software, support, services, training, resources and help for Cassandra. He has been involved in enterprise software his entire career. Working in the airline industry, he helped to build some of the highest volume online booking sites in the world. He saw first hand the consequences of trying to solve real world scalability problems at the limit of what traditional relational databases are capable of.
Similar to Python Through the Back Door: Netflix Presentation at CodeMash 2014 (20)
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Python Through the Back Door: Netflix Presentation at CodeMash 2014
1. Python Through the
Back Door
!
CodeMash 2014
!
Roy Rapoport
@royrapoport rsr@netflix.com
www.linkedin.com/in/royrapoport
2. A Word About Me
• About 20 years in technology
• Systems engineering, networking,
software development, QA, release
management
• Time at Netflix: 1655 days (4y:6m:11d)
• Before at Netflix: Service Delivery in
the IT/Ops, troubleshooter, Builder of
Python Things[tm]
• Current role: Insight Engineering
•Real-Time Operational Insight
!2
4. People
“Netflix Company Profile
Now via self service*
Go to your favorite Python REPL and type the following:
import re, requests!
content = requests.get(“http://ir.netflix.com").content!
content = content.replace(“ ", " ")!
p = re.compile(r”.*over (d+) .*in (d+)”, re.S)!
m = re.match(p, content)!
print "Netflix is the world's leading Internet !
subscription service for enjoying TV and movies, !
with more than {} million subscribers in {} !
countries.”.format(m.group(1), m.group(2))!
*No whining. Remember that you’ll never again need to wait for me to update this
slide like you had to wait for database access when you started your last job.”
- Jay Zarfoss, http://www.slideshare.net/zarfide
!4
9. People
Policies
(How They Usually Work)
11/27/2006
“Sorry, but the standard monitor...is the HP 17 flat panel. I
actually told a director last week that they couldn't have a
19 for a new office so I am not picking on just you.”
9
10. People
Policies
(How They Usually Work)
!
6/18/2007
“There is a request for quantity 2 17” flat panels. We have
received direction from the CIO that no one will have
more than 1 flat panel monitor. I just wanted to let you
know that there will only be one monitor ordered ... The
17” is our only standard except for Legal.”
10
13. People
Policies
@nflx
!
01/30/2013, 15:22 PST
I'd like to request a 15” MBP w/ Retina Display. I don't know how much
you guys care about CPU specs -- it looks like the bump from 2.3GHz to
2.6GHz is reasonably priced at only about $100, so if it works for you
that'd be nice. 16GB RAM and at least 512GB drive.
!
01/31/2013
12:00 PST: “Forwarded to IT Purchasing to provide a quote to Roy for
the requested configuration.”
13:33 PST: “Requesting quote from vendor”
15:32 PST: “Attached is the quote, please approve and I’ll place order”
15:46 PST: “Thanks for the rapid response. Please order.”
15:52 PST: “Ordered. PO #...”
13
15. Problem
The Before Time
Dozens of SSL Certificates
Decentralized
Kept Expiring
Hilarity would ensue
Amazon Resources
“No Preset Limit”
You know when you hit it
Hilarity would ensue
15
16. Python
The Before Time
•
Well-developed Developer Ecosystem
•
•
DB Client
•
Credentials Management
•
Memory Object Cache
•
Server Infrastructure
•
•
Discovery
Telemetry
You wanted that for Java, right?
16
17. The Before Time
•
Just moved from IT/Ops
•
Problem
Python
Formally tasked with SSL cert
issue as quarterly goal
•
•
•
Limits issue “tacked” on
Happily hackily Pythonic
Presenter Selfie
Didn’t know Java
17
20. Python
Abstraction
•
“The process of separating
ideas from specific instances of
those ideas at work.”
•
Some abstraction: Good
•
Too much abstraction burns
your tongue*
•
Known bug
* Mixed metaphor is mixed
!20
23. Problem
Alerting
• Enterprise IT Solution
• Managed by the Enterprise IT Alerting People
• File Tickets
• Send alerts to NOC
• Completely separate from telemetry system
Copyright USAID Microlinks. CC Attribution 2.0
23
24. Problem
Alerting
• Enterprise IT Solution
• Managed by the Enterprise IT Alerting People
• File Tickets
• Send alerts to NOC
• Completely separate from telemetry system
Copyright: http://www.flickr.com/photos/s_w_ellis
CC Attribution 2.0 License
24
28. Python
But Now We Need…
• import Discovery.publish
• import EVCache
• import EpicMetrics
• import Archaius
• import Asgard.Registry
• import AKMS
28
29. Python
AKMS?
In [1]: import AKMS!
In [2]: ak = AKMS.AKMS(RoyWasHere)!
In [3]: ak.keys()!
Out[3]: ['MLQBAYLLDIGXPBQB', ‘eMr+Mdhv+E4xD+paPCxXF+’]!
In [4]: a, s = ak.keys()!
In [5]: s3_object = boto.connect_s3(a, s)!
In [6]: ak = AKMS.AKMS(RoyWasHere, version=2)!
In [7]: ak.keys()!
Out[7]: [‘yn[…]G’, ‘rV[…]bKfSUHDSA’, ‘reallyLongStringElided']!
In [8]: ak.expiration!
Out[8]: 1389165118!
In [9]: a, s, s2 = ak.keys()!
In [10]: s3_object = boto.connect_s3(a, s, security_token=s2)
29
30. People
So AKMS
• Server more paranoid than most
• Making Python library was a pain
• Remember Jay?
• High lateral trust
• Prioritization autonomy
• Never ask for permission
30
31. People
Lateral Trust
• Humans are good game players
• What are the rules?
• Zero-sum games: I want you to lose
• Stack ranking
• Fixed bonus / raise pools
31
32. People
Lateral Trust @nflx
• No fixed pools for anything
• No ranking (at all)
• Reviews != raises
• Smart people generally make good decisions
• Global optimization
32
33. People
Subordinate Trust @nflx
• Focus on results
• Unleash employees
• Encourage disagreement
• Accept dissent
• Job #1: Attract and retain world-class talent
33
34. People
Manager Trust @nflx
• Question, question, question
• Drive for context, not decisions
• Nobody is above questioning
34
35. Python
Field of Dreams
• Turned out I wasn’t the only one
• Striking the right balance between MVP
and future growth (maybe)
• And if it hadn’t … it’d still have been the
right choice
35
36. A Virtuous Cycle
• Requirement for high impact
• No process for permission
• Unorthodox language choice
• Lateral support for development
• Increased adoption
•…
• Profit!*
* (or at least a new standard)
36
Python
People
Problem
37. Tell me what you think.
You know you want to.
http://bit.ly/netflixcmpython
!37
![A Word About Me
• About 20 years in technology
• Systems engineering, networking,
software development, QA, release
management
• Time at Netflix: 1655 days (4y:6m:11d)
• Before at Netflix: Service Delivery in
the IT/Ops, troubleshooter, Builder of
Python Things[tm]
• Current role: Insight Engineering
•Real-Time Operational Insight
!2](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)