1. In the 1960s, Ken Thompson created the video game "Space Travel" while working on the Multics Operating System at Bell Labs. When Bell Labs withdrew from the project, Thompson rewrote Space Travel on an old PDP-7 machine. The tools created for the game later became the Unix operating system.
2. Virtualization successfully decoupled hardware from services, allowing easy provisioning of virtual machines (VMs) from standard templates. This simplified administration and reduced provisioning time from months to days or immediately.
3. The rise of public cloud and internal virtualization drove the creation of DevOps approaches to fully automate the software development lifecycle from code to deployment. This automation reduced friction
Achieving scale and performance using cloud native environmentRakuten Group, Inc.
ID Platform Product can be used by every Rakuten Group Companies and can easily serve millions of users. Multi-Region product challenges are many, example:
- Ensure 4 9’s availability
- Management across each region
- Alerting and Monitoring across each region
- Auto scaling (Scale up and Scale down) across each region
- Performance (vertical scale up)
- Cost
- DB Consistency Across Multiple Regions
- Resiliency
At Ecosystem Platform Layer for Rakuten, we handle each of these and this presentation is about how we handle these challenging scenarios.
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log InsightVMworld
VMworld 2013
Steve Flanders, VMware
Chengdu Huang, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
Plate spin migration and transformation prsesentation uploadApurva Shah
PlateSpin® Migrate is a powerful workload portability solution that automates the process of moving server workloads over
the network between physical servers, virtual hosts and image archives. PlateSpin Migrate remotely decouples workloads
from the underlying server hardware and streams them to and from physical or virtual hosts—all from a single point of control.
Integrating Hybrid Cloud Database-as-a-Service with Cloud Foundry’s Service ...VMware Tanzu
SpringOne Platform 2016
Speaker: Lenley Hensarling; SVP Strategy, EnterpriseDB
Enterprises want to enable continuous delivery and deployment of their digital products while also having the necessary security, robustness, monitoring, and management of the infrastructure. EnterpriseDB is integrating its Cloud Management provisioning capability with the Cloud Foundry Service Broker to allow data services and DBA groups to create templates for robust highly available PostgreSQL deployments while not impeding the speed and agility of the developer groups they serve. We’ll discuss how database provisioning through EDB’s Cloud Management can provide responsible DevOps models for the enterprise.
Achieving scale and performance using cloud native environmentRakuten Group, Inc.
ID Platform Product can be used by every Rakuten Group Companies and can easily serve millions of users. Multi-Region product challenges are many, example:
- Ensure 4 9’s availability
- Management across each region
- Alerting and Monitoring across each region
- Auto scaling (Scale up and Scale down) across each region
- Performance (vertical scale up)
- Cost
- DB Consistency Across Multiple Regions
- Resiliency
At Ecosystem Platform Layer for Rakuten, we handle each of these and this presentation is about how we handle these challenging scenarios.
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log InsightVMworld
VMworld 2013
Steve Flanders, VMware
Chengdu Huang, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
Plate spin migration and transformation prsesentation uploadApurva Shah
PlateSpin® Migrate is a powerful workload portability solution that automates the process of moving server workloads over
the network between physical servers, virtual hosts and image archives. PlateSpin Migrate remotely decouples workloads
from the underlying server hardware and streams them to and from physical or virtual hosts—all from a single point of control.
Integrating Hybrid Cloud Database-as-a-Service with Cloud Foundry’s Service ...VMware Tanzu
SpringOne Platform 2016
Speaker: Lenley Hensarling; SVP Strategy, EnterpriseDB
Enterprises want to enable continuous delivery and deployment of their digital products while also having the necessary security, robustness, monitoring, and management of the infrastructure. EnterpriseDB is integrating its Cloud Management provisioning capability with the Cloud Foundry Service Broker to allow data services and DBA groups to create templates for robust highly available PostgreSQL deployments while not impeding the speed and agility of the developer groups they serve. We’ll discuss how database provisioning through EDB’s Cloud Management can provide responsible DevOps models for the enterprise.
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!Continuent
With Multiple Active Primary MySQL Databases
Watch this on-demand webinar to learn the right way to deploy geo-distributed databases. We look at the pitfalls of deploying a single site and passive sites, and from there we show how to provide the best user experience by leveraging geo-distributed MySQL.
When considering geo-distributed MySQL database environments it is important to understand the nuances of having multiple active clusters deployed across sites and clouds. This webinar walks through the proper planning of geo-distributed MySQL for success.
Finally, you’ll learn about our best practices for multiple primary clusters, as well as failover and disaster recovery for MySQL.
AGENDA
- Why Geo-Distributed Databases
- Geo-Distributed MySQL Starts With High Performance Local Clusters
- Extend The Cluster To Multiple Datacenters/Clouds
- Best Practices For Multiple Primary Clusters
- Failover & Disaster Recovery
- Key Benefits
PRESENTER
Matthew Lang, Customer Success Director – Americas, Continuent, has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
Technical breakout during Confluent’s streaming event in Munich, presented by Sam Julian, Chief Cloud Engineer at E.On SE. This three-day hands-on course focused on how to build, manage, and monitor clusters using industry best-practices developed by the world’s foremost Apache Kafka™ experts. The sessions focused on how Kafka and the Confluent Platform work, how their main subsystems interact, and how to set up, manage, monitor, and tune your cluster.
Virtualizing Mission-critical Workloads: The PlateSpin StoryNovell
Explore the key roadblocks to virtualizing mission-critical workloads. Discuss the potential benefits, in terms of performance and service-level delivery, in migrating these workloads to an internal cloud. Learn how PlateSpin solutions pave the way to migrating mission-critical workloads with little or no downtime, integrate non-disruptive testing in the virtual environment, and provide real-time workload protection and recovery.
Accelerating Server Hardware Upgrades with PlateSpin Migrate P2PNovell
How long does it take to upgrade to a new server? Re-installing and configuring applications and patches is tedious, error-prone, time-consuming—and let's face it—boring. During this session, you will learn how physical-to-physical (P2P) conversions with PlateSpin Migrate can dramatically accelerate and automate this process, shorten the downtime window, and can even be scheduled to run unattended.
Service Ownership with PagerDuty and Rundeck: Help others help you TraciMyers5
Many engineering and operations teams would like to move to a Service Ownership: "You build it, you own it" operating model. However, as with many ancillary objectives driving DevOps across an organization, this is easier said than done. Often this is because teams lack the human-to-technology mechanisms that allow for a culture of service ownership.
Within the context of incident response, teams need to be able to clearly define who is responsible for tending to issues, how they're notified, and who to lean on for help. This is true for non-incident response scenarios too. How can teams operate at a fast pace and at a large scale, while still maintaining valid and safe service ownership? One of the keys to allowing for service ownership outside of incident response is by imbuing an organization with a culture of self-service operations. This is where a service owner builds and delegates self-service mechanisms for end-users (non service owners) to make use of a given service safer while also reducing the number of interruptions to the service creator/owner.
In this webinar, you'll learn:
How self-service helps organizations adopt a ‘You Build it, You Own it’ model
Necessary mechanisms for service owners to create self-service interfaces to address the needs of their service-users
How to apply self-service while continuing to maintain security and compliance standards
How to allow developers and SREs to safely delegate automation as self-service requests to other teams and IT users
Help developers regain productivity and quality of life by doing what they do best: coding
Lessons from the field: Catalog of Kafka Deployments | Joseph Niemiec, ClouderaHostedbyConfluent
Streaming architectures have been on the rise steadily and as a result, we have seen the adoption of Kafka go up too. With the diverse spread of use cases across multiple industries, we have seen a variety of Kafka deployments across our hundreds of Kafka customers. Along the way, we have learnt some best practices as well as what not to do in mission-critical architectures. Join Joe Niemiec, Sr. Product Manager at Cloudera, as he shares these insights in this session that covers topics such as - The many ways that Kafka has been deployed in the field Standalone clusters, multiple clusters in a single data center and multiple clusters geographically distributed performing replication Clusters of all sizes small and large, few messages to hundreds of thousands per second Discussion about architecture failure domains Configurations tuned and used in specific deployments
Deployment Checkup: How to Regularly Tune Your Cloud Environment - RightScale...RightScale
Speaker: Brian Adler - Sr. Services Architect, RightScale
Periodic reviews of your cloud deployments are as important as a regular oil change and tune up at your mechanic. We will cover our RightScale deployment checklist to root out issues with server utilization, cost optimization, multi-zone balancing, auto-scaling, high-availability configuration, security groups, IP address assignment, security patching, and manual changes. You will leave with an easy-to-follow maintenance schedule for your cloud environment.
How to Enable Industrial Decarbonization with Node-RED and InfluxDBInfluxData
Graphite Energy’s thermal energy storage (TES) platform encourages clients to offset their traditional energy consumption with low-cost renewable energy sources. Their customers include manufacturers, mines, steelmakers and aluminum plants. IIoT data is collected about energy usage, fuel consumption, temperatures, solar panels, wind farms, process steam and air dryers. Discover how Graphite Energy uses InfluxDB to monitor their zero-emission energy solution.
In this webinar, Byron Ross will dive into:
Graphite Energy’s approach to reducing their clients’ carbon footprint
Their methodology to collecting sensor data used to make their operations more green
Why they chose a time series database over a data historian
Learnings from the Field. Lessons from Working with Dozens of Small & Large D...HostedbyConfluent
If your data platform is powered only by batch data processing, you know you are always trailing your customer. Your databases aren’t always up to date. Your inability to have a synchronized data flow across systems leads to operational inefficiencies. And, your dreams of running advanced real-time AI and ML applications can’t be fulfilled. However, you might be wary of the implications of turning your product into an event-driven one. In this presentation we’ll share our experience transforming our CDP-based marketing orchestration engine to be both real-time and highly scalable with the Kafka ecosystem. We will look into how we saved resources with Connect when ingesting and syncing data with NoSQL databases, data warehouses and third-party platforms. What we did to turn ksqlDB into our data transformation, aggregation and querying hub, reducing latency and costs. How Streams helps us activate multiple real-time applications such as building identity graphs, updating materialized views in high frequency for efficient real-time lookups and inferencing machine learning models. Finally, we will look at how Confluent Cloud solved our pre-rollout sizing and scaling questions, significantly reducing time-to-market.
Strata SC 2014: Apache Mesos as an SDK for Building Distributed FrameworksPaco Nathan
O'Reilly Media - Strata SC 2014
Apache Mesos is an open source cluster manager that provides efficient resource isolation for distributed frameworks—similar to Google’s “Borg” and “Omega” projects for warehouse scale computing. It is based on isolation features in the modern kernel: “cgroups” in Linux, “zones” in Solaris.
Google’s “Omega” research paper shows that while 80% of the jobs on a given cluster may be batch (e.g., MapReduce), 55-60% of cluster resources go toward services. The batch jobs on a cluster are the easy part—services are much more complex to schedule efficiently. However by mixing workloads, the overall problem of scheduling resources can be greatly improved.
Given the use of Mesos as the kernel for a “data center OS”, two additional open source components Chronos (like Unix “cron”) and Marathon (like Unix “init.d”) serve as the building blocks for creating distributed, fault-tolerant, highly-available apps at scale.
This talk will examine case studies of Mesos uses in production at scale: ranging from Twitter (100% on prem) to Airbnb (100% cloud), plus MediaCrossing, Categorize, HubSpot, etc. How have these organizations leveraged Mesos to build better, more scalable and efficient distributed apps? Lessons from the Mesos developer community show that one can port an existing framework with a wrapper in approximately 100 line of code. Moreover, an important lesson from Spark is that based on “data center OS” building blocks one can rewrite a distributed system much like Hadoop to be 100x faster within a relatively small amount of source code.
These case studies illustrate the obvious benefits over prior approaches based on virtualization: scalability, elasticity, fault-tolerance, high availability, improved utilization rates, etc. Less obvious outcomes also include: reduced time for engineers to ramp-up new services at scale; reduced latency between batch and services, enabling new high-ROI use cases; and enabling dev/test apps to run on a production cluster without disrupting operations.
DCSF19 Transforming a 15+ Year Old Semiconductor Manufacturing EnvironmentDocker, Inc.
Jeanie Schwenk, Jireh Semiconductor
Jireh Semiconductor bought the Hillsboro fab and its contents including the manufacturing tools, servers, and software running the fab. The previous company had been winding down for years so server and software upgrades had not been on the radar for some time. In 2011 Jireh became the proud owner of the building, the tools, and its legacy software running on servers that weren’t even made any more.
That's when I started my adventure with Jireh in September 2016 with a charter to modernize the applications running the manufacturing facility process and move them into VMs with no impact to manufacturing. That led me down a path of exploration and questions. “What’s the goal?”
The goal wasn't to move to VMs. It was to become independent of the aging PA-RISC architecture, bring forward the ~230 java 1.4.2 applications (10-15 years old), scale to allow increased the load on the software and hardware in order to ramp the factory output to numbers never seen previously. And do it without manufacturing downtime.
The solution included a transition from waterfall and silo development to agile scrum. Rather than simply migrating to VMs, it became obvious the lynch pin for a successful software transition with the required uptime, flexibility, and scalability was Docker Enterprise.
Join me for this session where I'll talk about my journey modernizing 15+ year old applications and infrastructure at Jireh.
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!Continuent
With Multiple Active Primary MySQL Databases
Watch this on-demand webinar to learn the right way to deploy geo-distributed databases. We look at the pitfalls of deploying a single site and passive sites, and from there we show how to provide the best user experience by leveraging geo-distributed MySQL.
When considering geo-distributed MySQL database environments it is important to understand the nuances of having multiple active clusters deployed across sites and clouds. This webinar walks through the proper planning of geo-distributed MySQL for success.
Finally, you’ll learn about our best practices for multiple primary clusters, as well as failover and disaster recovery for MySQL.
AGENDA
- Why Geo-Distributed Databases
- Geo-Distributed MySQL Starts With High Performance Local Clusters
- Extend The Cluster To Multiple Datacenters/Clouds
- Best Practices For Multiple Primary Clusters
- Failover & Disaster Recovery
- Key Benefits
PRESENTER
Matthew Lang, Customer Success Director – Americas, Continuent, has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
Technical breakout during Confluent’s streaming event in Munich, presented by Sam Julian, Chief Cloud Engineer at E.On SE. This three-day hands-on course focused on how to build, manage, and monitor clusters using industry best-practices developed by the world’s foremost Apache Kafka™ experts. The sessions focused on how Kafka and the Confluent Platform work, how their main subsystems interact, and how to set up, manage, monitor, and tune your cluster.
Virtualizing Mission-critical Workloads: The PlateSpin StoryNovell
Explore the key roadblocks to virtualizing mission-critical workloads. Discuss the potential benefits, in terms of performance and service-level delivery, in migrating these workloads to an internal cloud. Learn how PlateSpin solutions pave the way to migrating mission-critical workloads with little or no downtime, integrate non-disruptive testing in the virtual environment, and provide real-time workload protection and recovery.
Accelerating Server Hardware Upgrades with PlateSpin Migrate P2PNovell
How long does it take to upgrade to a new server? Re-installing and configuring applications and patches is tedious, error-prone, time-consuming—and let's face it—boring. During this session, you will learn how physical-to-physical (P2P) conversions with PlateSpin Migrate can dramatically accelerate and automate this process, shorten the downtime window, and can even be scheduled to run unattended.
Service Ownership with PagerDuty and Rundeck: Help others help you TraciMyers5
Many engineering and operations teams would like to move to a Service Ownership: "You build it, you own it" operating model. However, as with many ancillary objectives driving DevOps across an organization, this is easier said than done. Often this is because teams lack the human-to-technology mechanisms that allow for a culture of service ownership.
Within the context of incident response, teams need to be able to clearly define who is responsible for tending to issues, how they're notified, and who to lean on for help. This is true for non-incident response scenarios too. How can teams operate at a fast pace and at a large scale, while still maintaining valid and safe service ownership? One of the keys to allowing for service ownership outside of incident response is by imbuing an organization with a culture of self-service operations. This is where a service owner builds and delegates self-service mechanisms for end-users (non service owners) to make use of a given service safer while also reducing the number of interruptions to the service creator/owner.
In this webinar, you'll learn:
How self-service helps organizations adopt a ‘You Build it, You Own it’ model
Necessary mechanisms for service owners to create self-service interfaces to address the needs of their service-users
How to apply self-service while continuing to maintain security and compliance standards
How to allow developers and SREs to safely delegate automation as self-service requests to other teams and IT users
Help developers regain productivity and quality of life by doing what they do best: coding
Lessons from the field: Catalog of Kafka Deployments | Joseph Niemiec, ClouderaHostedbyConfluent
Streaming architectures have been on the rise steadily and as a result, we have seen the adoption of Kafka go up too. With the diverse spread of use cases across multiple industries, we have seen a variety of Kafka deployments across our hundreds of Kafka customers. Along the way, we have learnt some best practices as well as what not to do in mission-critical architectures. Join Joe Niemiec, Sr. Product Manager at Cloudera, as he shares these insights in this session that covers topics such as - The many ways that Kafka has been deployed in the field Standalone clusters, multiple clusters in a single data center and multiple clusters geographically distributed performing replication Clusters of all sizes small and large, few messages to hundreds of thousands per second Discussion about architecture failure domains Configurations tuned and used in specific deployments
Deployment Checkup: How to Regularly Tune Your Cloud Environment - RightScale...RightScale
Speaker: Brian Adler - Sr. Services Architect, RightScale
Periodic reviews of your cloud deployments are as important as a regular oil change and tune up at your mechanic. We will cover our RightScale deployment checklist to root out issues with server utilization, cost optimization, multi-zone balancing, auto-scaling, high-availability configuration, security groups, IP address assignment, security patching, and manual changes. You will leave with an easy-to-follow maintenance schedule for your cloud environment.
How to Enable Industrial Decarbonization with Node-RED and InfluxDBInfluxData
Graphite Energy’s thermal energy storage (TES) platform encourages clients to offset their traditional energy consumption with low-cost renewable energy sources. Their customers include manufacturers, mines, steelmakers and aluminum plants. IIoT data is collected about energy usage, fuel consumption, temperatures, solar panels, wind farms, process steam and air dryers. Discover how Graphite Energy uses InfluxDB to monitor their zero-emission energy solution.
In this webinar, Byron Ross will dive into:
Graphite Energy’s approach to reducing their clients’ carbon footprint
Their methodology to collecting sensor data used to make their operations more green
Why they chose a time series database over a data historian
Learnings from the Field. Lessons from Working with Dozens of Small & Large D...HostedbyConfluent
If your data platform is powered only by batch data processing, you know you are always trailing your customer. Your databases aren’t always up to date. Your inability to have a synchronized data flow across systems leads to operational inefficiencies. And, your dreams of running advanced real-time AI and ML applications can’t be fulfilled. However, you might be wary of the implications of turning your product into an event-driven one. In this presentation we’ll share our experience transforming our CDP-based marketing orchestration engine to be both real-time and highly scalable with the Kafka ecosystem. We will look into how we saved resources with Connect when ingesting and syncing data with NoSQL databases, data warehouses and third-party platforms. What we did to turn ksqlDB into our data transformation, aggregation and querying hub, reducing latency and costs. How Streams helps us activate multiple real-time applications such as building identity graphs, updating materialized views in high frequency for efficient real-time lookups and inferencing machine learning models. Finally, we will look at how Confluent Cloud solved our pre-rollout sizing and scaling questions, significantly reducing time-to-market.
Strata SC 2014: Apache Mesos as an SDK for Building Distributed FrameworksPaco Nathan
O'Reilly Media - Strata SC 2014
Apache Mesos is an open source cluster manager that provides efficient resource isolation for distributed frameworks—similar to Google’s “Borg” and “Omega” projects for warehouse scale computing. It is based on isolation features in the modern kernel: “cgroups” in Linux, “zones” in Solaris.
Google’s “Omega” research paper shows that while 80% of the jobs on a given cluster may be batch (e.g., MapReduce), 55-60% of cluster resources go toward services. The batch jobs on a cluster are the easy part—services are much more complex to schedule efficiently. However by mixing workloads, the overall problem of scheduling resources can be greatly improved.
Given the use of Mesos as the kernel for a “data center OS”, two additional open source components Chronos (like Unix “cron”) and Marathon (like Unix “init.d”) serve as the building blocks for creating distributed, fault-tolerant, highly-available apps at scale.
This talk will examine case studies of Mesos uses in production at scale: ranging from Twitter (100% on prem) to Airbnb (100% cloud), plus MediaCrossing, Categorize, HubSpot, etc. How have these organizations leveraged Mesos to build better, more scalable and efficient distributed apps? Lessons from the Mesos developer community show that one can port an existing framework with a wrapper in approximately 100 line of code. Moreover, an important lesson from Spark is that based on “data center OS” building blocks one can rewrite a distributed system much like Hadoop to be 100x faster within a relatively small amount of source code.
These case studies illustrate the obvious benefits over prior approaches based on virtualization: scalability, elasticity, fault-tolerance, high availability, improved utilization rates, etc. Less obvious outcomes also include: reduced time for engineers to ramp-up new services at scale; reduced latency between batch and services, enabling new high-ROI use cases; and enabling dev/test apps to run on a production cluster without disrupting operations.
DCSF19 Transforming a 15+ Year Old Semiconductor Manufacturing EnvironmentDocker, Inc.
Jeanie Schwenk, Jireh Semiconductor
Jireh Semiconductor bought the Hillsboro fab and its contents including the manufacturing tools, servers, and software running the fab. The previous company had been winding down for years so server and software upgrades had not been on the radar for some time. In 2011 Jireh became the proud owner of the building, the tools, and its legacy software running on servers that weren’t even made any more.
That's when I started my adventure with Jireh in September 2016 with a charter to modernize the applications running the manufacturing facility process and move them into VMs with no impact to manufacturing. That led me down a path of exploration and questions. “What’s the goal?”
The goal wasn't to move to VMs. It was to become independent of the aging PA-RISC architecture, bring forward the ~230 java 1.4.2 applications (10-15 years old), scale to allow increased the load on the software and hardware in order to ramp the factory output to numbers never seen previously. And do it without manufacturing downtime.
The solution included a transition from waterfall and silo development to agile scrum. Rather than simply migrating to VMs, it became obvious the lynch pin for a successful software transition with the required uptime, flexibility, and scalability was Docker Enterprise.
Join me for this session where I'll talk about my journey modernizing 15+ year old applications and infrastructure at Jireh.
Cloud providers like Amazon or Goggle have great user experience to create and manage PaaS and IaaS services. But is it possible to reproduce same experience and flexibility locally, in on premise datacenter? This talk describes success story of creation private cloud based on DC/OS cluster. It is used to host and share different services like hadoop or kafka for development teams, dynamically manage services and resource pools with GKE integration.
3 years ago, Meetic chose to rebuild it's backend architecture using microservices and an event driven strategy. As we where moving along our old legacy application, testing features became gradually a pain, especially when those features rely on multiple changes across multiple components. Whatever the number of application you manage, unit testing is easy, as well as functional testing on a microservice. A good gherkin framework and a set of docker container can do the job. The real challenge is set in end-to-end testing even more when a feature can involve up to 60 different components.
To solve that issue, Meetic is building a Kubernetes strategy around testing. To do such a thing we need to :
- Be able to generate a docker container for each pull-request on any component of the stack
- Be able to create a full testing environment in the simplest way
- Be able to launch automated test on this newly created environment
- Have a clean-up process to destroy testing environment after tests To separate the various testing environment, we chose to use Kubernetes Namespaces each containing a variant of the Meetic stack. But when it comes to Kubernetes, managing multiple namespaces can be hard. Yaml configuration files need to be shared in a way that each people / automated job can access to them and modify them without impacting others.
This is typically why Meetic chose to develop it's own tool to manage namespace through a cli tool, or a REST API on which we can plug a friendly UI.
In this talk we will tell you the story of our CI/CD evolution to satisfy the need to create a docker container for each new pull request. And we will show you how to make end-to-end testing easier using Blackbeard, the tool we developed to handle the need to manage namespaces inspired by Helm.
ASZ-3034 Build a WebSphere Linux Cloud on System z: From Roll-Your-Own to Pre...WASdev Community
Do you need the most reliable, secure, and cost-effective on-premise cloud platform? Look no further: a cloud based on WebSphere and Linux on System z is the answer. This presentation traces the evolution of successful server consolidation to Linux on System z, from brute-force physical moves to virtual topology to sophisticated workload placement. We'll cover techniques and considerations to ensure a rich, dense, enterprise environment. The material is derived from interactions with our enterprise mainframe customers running world-class data centers.
We will briefly describe the new Enterprise Cloud System that unites leading IBM software, storage, and server technologies into one simple, flexible, and secure factory-integrated solution.
We will show examples of System z based cloud environments which provide everything you expect from System z: extreme reliability, secure, geo-dispersed, high performance clouds. We will describe application development and deployment patterns that both help and hurt in a virtualized cloud environment. From the admin perspective we will explore heap and GC tuning, idle server tuning, and stacking options. We will also present a very effective performance tuning approach for large scale virtualized environments.
We also present WebSphere Liberty profile performance in a virtualized environment, relative to a traditional WebSphere application server.
OS for AI: Elastic Microservices & the Next Gen of MLNordic APIs
AI has been a hot topic lately, with advances being made constantly in what is possible, there has not been as much discussion of the infrastructure and scaling challenges that come with it. How do you support dozens of different languages and frameworks, and make them interoperate invisibly? How do you scale to run abstract code from thousands of different developers, simultaneously and elastically, while maintaining less than 15ms of overhead?
At Algorithmia, we’ve built, deployed, and scaled thousands of algorithms and machine learning models, using every kind of framework (from scikit-learn to tensorflow). We’ve seen many of the challenges faced in this area, and in this talk I’ll share some insights into the problems you’re likely to face, and how to approach solving them.
In brief, we’ll examine the need for, and implementations of, a complete “Operating System for AI” – a common interface for different algorithms to be used and combined, and a general architecture for serverless machine learning which is discoverable, versioned, scalable and sharable.
Making Clouds: Turning OpenNebula into a ProductNETWAYS
What does it takes to bring innovations like private clouds to small and medium enterprises? In the course of this talk we will present our experience in creating a self-service toolkit for creating a complete virtualization and cloud platform based on OpenNebula, as well as our experience gathered in tens of installations of all sizes. From scalable storage (with benchmarks!) to autonomic optimization, we will present what in our view is needed to bring private clouds to everyone, what components and additions we created to better solve our customers’ problems (from replacing industrial control systems to medium scale virtual desktop infrastructures), and why OpenNebula has been chosen over other competing cloud toolkits.
OpenNebulaConf 2013 - Making Clouds: Turning OpenNebula into a Product by Car...OpenNebula Project
What does it takes to bring innovations like private clouds to small and medium enterprises? In the course of this talk we will present our experience in creating a self-service toolkit for creating a complete virtualization and cloud platform based on OpenNebula, as well as our experience gathered in tens of installations of all sizes. From scalable storage (with benchmarks!) to autonomic optimization, we will present what in our view is needed to bring private clouds to everyone, what components and additions we created to better solve our customers’ problems (from replacing industrial control systems to medium scale virtual desktop infrastructures), and why OpenNebula has been chosen over other competing cloud toolkits.
Bio:
Carlo Daffara the Technical director of Cloudweavers, and formerly head of research and development at Conecta, a consulting firm specializing in open source systems and distributed computing; Italian member of the European Working Group on Libre Software and co-coordinator of the working group on SMEs of the EU ICT task force on competitiveness. Since 1999, works as evaluator for IST programme submissions in the field of component-based software engineering, GRIDs and international cooperation. Coordinator of the open source platforms technical area of the IEEE technical committee on scalable computing, co-chair of the SIENA EU cloud initiative roadmap editorial board and part of the editorial review board of the International Journal of Open Source Software & Processes (IJOSSP).
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageMayaData Inc
Webinar Session - https://youtu.be/_5MfGMf8PG4
In this webinar, we share how the Container Attached Storage pattern makes performance tuning more tractable, by giving each workload its own storage system, thereby decreasing the variables needed to understand and tune performance.
We then introduce MayaStor, a breakthrough in the use of containers and Kubernetes as a data plane. MayaStor is the first containerized data engine available that delivers near the theoretical maximum performance of underlying systems. MayaStor performance scales with the underlying hardware and has been shown, for example, to deliver in excess of 10 million IOPS in a particular environment.
ELC-E 2016 Neil Armstrong - No, it's never too late to upstream your legacy l...Neil Armstrong
You maintain or used to maintain a Linux based board or SoC off-tree ? Then there are plenty of reasons for you to push your changes to the mainline Linux. Some will say it’s too late, or too complex, or too expensive but the long-term benefits of regular upstreaming truly outpass these constraints especially it you have the right methods. In this presentation Neil will elaborate on this question.
Neil will then expose the various challenges about code upstreaming, like time constraints, copyright issues and the community aspect of the work. For example, vendor GPL code is generally lying on an obscure github repo, or in a hardly reachable tarball.
In parallel, Neil will present practical tips to easier your day to day upstream work and explicit this simple rule : the fastest the maximum patches are upstreamed, the less work you’ll have to actually maintain the port in the future.
Containerizing couchbase with microservice architecture on mesosphere.pptxRavi Yadav
Ravi Yadav, Mesosphere
Anil Kumar, Couchbase
Organizations focused on delivering exceptional customer experiences are building applications using microservice architectures because of the flexibility, speed of delivery, and maintainability that they provide. In this session, you will learn how Couchbase can fit into a microservice architecture using containers and orchestration. We will explore how Couchbase and Mesosphere work together to simplify application development and delivery. Additionally, you will see a demonstration of exactly how to create a Couchbase cluster on Mesosphere DC/OS Enterprise.
How kubernetes can help you quickly and automatically test and deploy new services
While Kubernetes is primarily associated with managing cloud-native applications and microservices, it can also play a role in IoT deployments. Here are a few reasons why Kubernetes is relevant in the context of IoT:
1 Scalability: IoT systems often involve a large number of devices generating massive amounts of data. Kubernetes provides automatic scaling capabilities, allowing IoT applications to scale horizontally by adding or removing instances based on demand. This helps manage the increasing workload efficiently.
2 Resilience and High Availability: IoT applications require high availability to ensure uninterrupted operations. Kubernetes offers features like load balancing, fault tolerance, and self-healing capabilities. It can automatically restart failed containers or replace them with healthy instances, ensuring that IoT services remain available and resilient.
3 Resource Optimization: IoT deployments typically involve a mix of hardware devices with varying capabilities. Kubernetes can optimize resource utilization by efficiently distributing workloads across devices. It allows you to define resource constraints and priorities, ensuring that devices with higher capabilities handle more demanding tasks.
4 Service Discovery and Load Balancing: In an IoT ecosystem, devices and services need to discover and communicate with each other. Kubernetes provides built-in service discovery mechanisms, such as DNS-based service discovery and load balancing, allowing devices to locate and interact with services dynamically.
5 Security and Updates: Security is a crucial aspect of IoT systems, and Kubernetes helps in managing security at scale. It provides features like role-based access control (RBAC), network policies, and secret management to enforce security measures across IoT deployments. Additionally, Kubernetes facilitates rolling updates, allowing for seamless updates and patches without downtime.
6 Flexibility and Portability: Kubernetes abstracts the underlying infrastructure, enabling IoT applications to be deployed consistently across different environments, whether it’s on-premises, in the cloud, or at the edge. This flexibility allows organizations to migrate or distribute their IoT workloads as needed, making it easier to adopt hybrid or multi-cloud strategies
Visão geral do hardware do servidor System z e Linux on z - Concurso MainframeAnderson Bassani
Apresentação realizada no evento de premiação do Concurso Mainframe 2014 que foi realizado em São Paulo, IBM Tutóia. Tópicos apresentados incluíram: hardware System zEC12 e zBC12, Linux on z, O que o System z faz que outras plataformas não fazem e um caso real de uma empresa desenvolvedora de Software.
Similar to Evolution of unix environments and the road to faster deployments (20)
In my presentation, I will summarize the applied and practical aspects of creating sustainable software products. What does it mean - "green" software for users and developers? I want to explain how creating “green” software can be driven by multiple organizational layers. And how building “green” software products can help the organization increase overall software product efficiency.
This presentation introduces the OWASP Top 10:2021.
It explains how to look at the data related to OWASP Top 10:2021, and provides detailed explanations of items with distinctive data. It also introduces the OWASP Project related to each item.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
2. 2
…
PDP-7
~1/2M USD (2018 equivalent) video console !
In the 1960s, while working on the Multics Operating System at Bell labs, Ken Thompson created the video
game “Space Travel”.
Bell Labs withdrew from the MULTICS project. In order to go on playing the game, Thompson found an old
PDP-7 machine and rewrote Space Travel on it.
The tools created to make Space Travel later became the Unix operating system.
Space Travel
Source: Wikipedia
4. 4
1995: Millions of $$$
SGI Challenge Array
1990: 50k US$ HP Workstation 1985: 32M USD Cray-2
with liquid cooling
• Many environments in 80’s and 90’s already heavily automated.
• Start of a shift from SHELL based automation to languages like Perl
• HW often had many functions to support automation (maybe sometime better than todays x86 HW).
• Tens to thousands of users on a single server/OS.
Upgrade/migration very complex.
Setup evolved over several years:
• very hard to clean up when upgrading/removing/rolling back software
• impossible to reproduce environments on a new server
Extremely high price for HW defined many operational patterns!
5. 5
Shared HW in labs with multi user support – How to achieve consistency?
Consistency across many servers/environments could be achieved in many ways
- Scripts managing groups of individual servers
- Diskless nodes (one filesystem)
- Distributed file systems allowed managing many computers “as one”
Example: HP DUX which allowed “one filesystem” even in a mixed HW environment
• Context dependent files allowed, for instance, different files per CPU type at the same location
• Distributed device nodes and distributed named pipes (cool stuff!)
However, tight coupling made the distributed filesystem a risk for the whole cluster
Only admin users could change system/application files.
Friction between “root” and the users gave birth to stories like
BOFH – The Bastard Operator From Hell!
7. 7
…
Desktop:
• Each user could get his own machine with local disk
• Users got more control of their own environment (there was a lack of mechanisms to avoid it!)
• Friction towards “root” dramatically reduced
Servers was in many ways a step backwards:
• Most 1990-2000 x86 HW had less automation support than Unix servers 10 years earlier
• Users paid “their own server” but many unix security aspects depended on control of root
-> friction between server ”owner” and operator
• More HW variations, more OS variations, less consistency in tools across HW and SW
-> admin workload per server increased (but HW cost savings made this ok…)
• By 2000, some companies started getting very good at automating x86 as well, but at significant dev cost
• However… cost/performance got significantly better!
8. 8
Traditional process to get new servers:
1. Meeting to share request
2. 2-5 meetings to discuss architecture
3. Budget, order, deliver, rack & cable
4. OS setup
5. Typically 3 to 6 months
6. Surprisingly often… servers got into next budget year….
9. 9
- High lead time for new HW
- Large number of custom HW variation caused big administrative overhead and extra cost
- “Ownership of HW” (HW Hugging) created mental barrier blocking efficient HW use
- Some core Unix dependencies such as NFS prevented “delegating root”
- Many technical solutions (NFS is again an example) also prevented efficient scaling and failure handling
often needed in Internet services
11. 11
Last 3 years :
Efficiency per clock +10%
Increase in Cores + 50%
0
5
10
15
20
25
30
0
500
1000
1500
2000
2500
3000
3500
4000
CoreCount
ClockFrequency
Clock Frequency Core Count Clock Efficiency
Increase in CPU cores main contributor to CPU since 2004
Time to virtualize!
12. 12
“Of the AWS instances monitored by 2nd Watch, 38 percent are small. There's a
significant gap between the next most numerous instance size, which is medium
(making up 19 percent of total instances monitored).”
Source: 2014 report from AWS management system provider 2nd Watch:
How powerful is a “small” instance?
System Multi-Core
Geekbench result
Dhrystone
EC2 ”t2.small” 2,822 35,003,920
iPhone X 10,641
Huawei P20 Pro 6753 100,945,863
13. 13
VM
Virtualization successfully decoupled HW from service!
• No more server hugging!
• Easy to keep “spare resources” for immediate deploy from VM image.
• Could use standard HW regardless of VM config. Dramatically simplified admins life.
• Provisioning time down from months to days or even immediately.
• Commercial products like vSphere made things “easy”
Server ServerServer
Hypervisor
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM VM
VM
VM
VM
VM
VM
VM
VM
VM VM
VM
14. 14
Huge shift!
• 1 mid-high spec 2010 server could quickly turn into 30-50 smaller VMs
• From many services per server -> 1 service per VM
• Many new tools on the scene to automate and track.
• Make image -> Clone! Clooone! Cloooooooooone!
• Predictable cloning, however maintaining a predictable and reproducible state when number of hosts
grew from 1 physical to 30-50 VMs did not get easier.
15. 15
• Endless resources! (Budget?? What’s that??)
• Allowed dev teams to bypass server admins. Great organizational hack but terrible for security..
• Good APIs for automated generation of VMs
• Not cheap, but in some way this benefits the user.
Many companies adopted automation just to be cost efficient through dynamic scaling.
16. 16
Public cloud and internal virtualization allowed (and even forced) the creation of DevOps
- DevOps is an approach rather than a distinct role (a lot of confusion and disagreement here…)
- The core of DevOps describes Toolchains for automation
- Code -> Build -> Test -> Package -> Release -> Configure -> Monitor (and repeat from start!)
- All parts of the toolchain should be automated a much as possible.
Most of friction between infrastructure and development literally gets automated away…
17. 17
Predictable environments
- Human change to environments cannot be allowed (humans are not predictable!)
- Automate tests as much as possible, including performance and failure test.
- All deploy operations should be automated
- Always build from a predictable baseline. Never change after build. Build again instead
- Immutable and disposable infrastructure!
- Build environment once, deploy many
Build Deploy Run Destroy
Deploy Run Destroy
Deploy Run Destroy
Development
QA
Production * X
Test
18. 18
Official Build
Unit test
Packaging
Private Branch Private Build Unit Test Merge to official
Branch
Deploy to CI Integration tests
Deploy to QA Integration tests Manual tests
Smoke tests Release
Provision CI
Environment
Provision QA
Environment
Provision Prod
Environment
When successfully done – Can often drive 3-10x increases in dev output
19. 19
Great abstraction for immutable systems!
- Can layer filesystems to dramatically speed up build, test and deploy of environments
- Easier, faster and less error prone to distribute than installing “thousands” of packages
- Very light weight (vs VMs)
- Minimal performance overhead
Host OS
Guest OS
Libraries
Application
Guest OS
Libraries
Application
Server
Host OS
Container control layer
Libraries
Application
Libraries
Application
Server
Hypervisor
Typical VM Setup Typical Container Setup
21. 21
In the past:
- Rough plan
- 4-10 Architecture meetings
- Cost Calculation
- If cost<Budget
- Approve HW cost
- Wait 6 months
- Launch service
- If cost>Budget
- Wait for next year
- Hope you did not forget the
whole plan by then…
Maybe 1-4 updates of dev/stg
per month
Today, compute is just a dependency:
(Imaginary deployment profile)
Instances: 100
Memory: 4G
Cores: 4
OS: Centos 7.4
Container image:
myapp: 5.24
>deploy myapp.profile
Wait 2-3 minutes and app runs on 100
containers
Can re-create dev/stg/test many times a
day.
22. 22
…
Facility
Network
Server
OS
Application Ugly Line of
Organizational
Friction!
BOFH
Clueless Developer
One of the biggest benefits of devops style automation is reduction in
friction between dev and infrastructure teams!
Friction in this case defined as “actions you have to take only to be able
to proceed but adds no value in terms of improving quality”
Developers
Cloud Platform Team
25. 25
Typical Internet service
- 1-2 releases per quarter: you may be able to maintain service quality
- > 3 releases per quarter: you may be improving service quality
- Fastest improving service parts in Rakuten: 20-30 A/B tests per quarter.
~50% failure ratios on tests normal
26. 26
Service A
Modern automation often allow developers to move faster than planning and decision making.
20 A/B tests per quarter = ~ 3 days for each test if sequentially done (Plan/DEV/QA/AB/Release)
How do you structure your teams for such faster processes?
PDM/PJM
DEV
QA
Infra
PDM/PJM
DEV
QA
Infra
B CA
PDM/PJM
DEV
QA
Infra
BA
Infra
DEV
DEV
Cross-Functional? Matrix? Broken Tetris structure
Something else?
27. 27
Traditional style.
VMs with semi automated deploy and QA
2-3 A/B tests per quarter. Flat for 2 years.
Change to containers. Full CI pipeline with more than
1000 testcases and automated deployment.
More than 20 A/B tests per quarter.
New cross functional team structure, same team size.
CVR data for one of Rakutens major site parts.
28. 28
Speed vs stability and risk. Need to keep balance!
Storage need to be handled different from stateless applications
Lower layers needs to provide flexibility and self service so higher layers can RUN by themselves!
StorageStorage
APIAPI
Control/AdminControl/Admin
App 1App 1 App 2App 2 App 3App 3
. . .
Slow Moving
Fast Moving
HWHW
FacilityFacility
29. 29
1/2M USD video
console turned
into Unix
1969 1980-1995
Multi User / Multi
service very
expensive Machines
1992-2000
Move starts towards
dedicated purpose
machines. Still 1
machine many services
2004-2010
Virtualization starts:
1 service process per
VM getting normal
Devops describes an
approach to make safer
and faster operation at
scale
Containers make operations
at scale faster than ever.
Application vs. Infrastructure
containers concepts
Public cloud takes off
and makes VMs easier
than ever!
2008-2012 2010 - 2012 2015 -> 2018 ->
Near future likely to bring
- Faster processes
- Include new things like
networking in the automation