The best response to a system outage is not "What did you do?", but "What did we learn?" This session will walk through three system-wide outages at Google, at Stitch Fix, and at WeWork—their incidents, aftermaths, and recoveries. In all cases, many things went right and a few went wrong; also in all cases, because of blameless cultures, we buckled down, learned a lot, and made substantial improvements in the systems for the future. Looking back with the perspective of 20-20 hindsight, all of these incidents were seminal events that changed the focus and trajectory of engineering at each organization. You will leave with a set of actionable suggestions in dealing with customers, engineering teams, and upper management. You will also enjoy a few war stories from the trenches.
One Terrible Day at Google, and How It Made Us BetterRandy Shoup
In October 2012, Google App Engine had an 8-hour global outage. This session walks through the incident and the "Reliability Fixit" it inspired in its aftermath. Learn how the team came together, and over the next 6 months, reduced reliability issues by 10x. Also take away broader insights around engineering tradeoffs, managing an incident, and driving improvement.
This presentation introduces the idea of a "Minimal Viable Architecture". As a company and product evolves, its architecture should evolve as well. We talk about the different phases of a product -- from the idea phase, to the starting phase, scaling phase, and optimizing phase. For each phase, we discuss the goals and constraints on the business, and we suggest an appropriate software architecture to match. Throughout the presentation, we use examples from eBay, Google, StitchFix, and others.
As the research in Accelerate and in the DevOps Handbook shows, high-performing organizations deliver more rapidly, more repeatably, and more reliably. And as an organization scales, it becomes more and more important to get the product development process right. Drawing on the speaker's experiences leading high-performing organizations at Google and eBay, this session discusses the upstream parts of that process, focusing on organization, problem definition, and prioritization. We will discuss forming small, cross-functional teams with clear areas of responsibility. Then we will discuss the importance of clearly defining the problem we are trying to solve as a team. Finally, we will cover focus and prioritization -- how we decide what to do when. You will take away actionable techniques you can apply in your own organization.
Learning from Learnings: Anatomy of Three IncidentsRandy Shoup
The best response to a system outage is not "What did you do?", but "What did we learn?" This session will walk through three system-wide outages at Google, at Stitch Fix, and at WeWork—their incidents, aftermaths, and recoveries. In all cases, many things went right and a few went wrong; also in all cases, because of blameless cultures, we buckled down, learned a lot, and made substantial improvements in the systems for the future. Looking back with the perspective of 20-20 hindsight, all of these incidents were seminal events that changed the focus and trajectory of engineering at each organization. You will leave with a set of actionable suggestions in dealing with customers, engineering teams, and upper management. You will also enjoy a few war stories from the trenches.
Faster is Better. High-performing organizations deploy both substantially faster and substantially more reliably, and thus are 2.5x more likely to achieve business goals. This keynote covers how to move fast at large scale:
* Organizing for Speed
* What to Build and What NOT to Build
* When to Build
* How to Build
* Delivering and Operating
Keynote at Reversim Summit 2017 in Tel Aviv, Israel.
DevOps is far more about culture and organization than it is about technology and tooling. This talk will discuss the speaker's experiences leading high-performing engineering teams at Google, eBay, and Stitch Fix, and will offer suggestions for other organizations to level up their DevOps game.
https://www.meetup.com/SV-ELC/events/240087808/
Modern software-service models take advantage of the great benefits in having the same team both build the software as well as operate it in production -- "You Build It; You Run It" is the Amazon mantra. What does this mean in practice?
Organizationally, it means small teams with well-defined areas of responsibility, directly aligned with the business. The teams are cross-functional, meaning that each team has all the skill sets it requires to do its job, while at the same time relying on other teams for supporting services, tools, and libraries.
Process-wise, it means doubling down on practices like test-driven development and continuous delivery. Using continuous delivery practices, high-performing teams can and do release their applications and services multiple times a day. This enables them to iterate rapidly, experiment courageously, and fail more quickly.
Culturally, it means end-to-end ownership. Each team owns its software end-to-end, from design to development to deployment to retirement. The same engineers who are responsible for the features are responsible for quality, performance, operations, and maintenance. This ownership puts incentives in the right place to encourage building maintainable, observable, and operable systems from the start.
All these techniques and approaches are available to everyone, and practical examples in this talk will help other organizations on their journey.
Scaling Your Architecture for the Long TermRandy Shoup
This talk from the virtual 2020 CTO Summit (https://www.ctoconnection.com/summits) covers several architecture lessons to help you survive and thrive through the scaling phase of your company:
* Modular Architecture
* Event-Driven Communication
* Quality and Reliability
* Continuous Delivery
Minimum Viable Architecture - Good Enough is Good EnoughRandy Shoup
The “right” architecture and organization depends on the size and scale of your company. The only constant is change, and what works for 5 engineers does not work for 5000. Based upon lessons from Google and eBay, learn how to evolve both technology and organization together successfully.
This presentation is based on many hard-won lessons by the speaker, who led large-scale engineering teams at Google and eBay, but also co-founded a tiny startup and tried (unsuccessfully) to apply the same techniques. This session hopes to help others from making the same mistakes by introducing the concept of “Minimal Viable Architecture”. It outlines the common architectural evolution of a company or project through the search, execution, and scaling phases, and discusses the appropriate technologies, disciplines, and organizational structures at each phase. You'll start with a monolith, and end up with microservices, and that's completely and entirely appropriate.
One Terrible Day at Google, and How It Made Us BetterRandy Shoup
In October 2012, Google App Engine had an 8-hour global outage. This session walks through the incident and the "Reliability Fixit" it inspired in its aftermath. Learn how the team came together, and over the next 6 months, reduced reliability issues by 10x. Also take away broader insights around engineering tradeoffs, managing an incident, and driving improvement.
This presentation introduces the idea of a "Minimal Viable Architecture". As a company and product evolves, its architecture should evolve as well. We talk about the different phases of a product -- from the idea phase, to the starting phase, scaling phase, and optimizing phase. For each phase, we discuss the goals and constraints on the business, and we suggest an appropriate software architecture to match. Throughout the presentation, we use examples from eBay, Google, StitchFix, and others.
As the research in Accelerate and in the DevOps Handbook shows, high-performing organizations deliver more rapidly, more repeatably, and more reliably. And as an organization scales, it becomes more and more important to get the product development process right. Drawing on the speaker's experiences leading high-performing organizations at Google and eBay, this session discusses the upstream parts of that process, focusing on organization, problem definition, and prioritization. We will discuss forming small, cross-functional teams with clear areas of responsibility. Then we will discuss the importance of clearly defining the problem we are trying to solve as a team. Finally, we will cover focus and prioritization -- how we decide what to do when. You will take away actionable techniques you can apply in your own organization.
Learning from Learnings: Anatomy of Three IncidentsRandy Shoup
The best response to a system outage is not "What did you do?", but "What did we learn?" This session will walk through three system-wide outages at Google, at Stitch Fix, and at WeWork—their incidents, aftermaths, and recoveries. In all cases, many things went right and a few went wrong; also in all cases, because of blameless cultures, we buckled down, learned a lot, and made substantial improvements in the systems for the future. Looking back with the perspective of 20-20 hindsight, all of these incidents were seminal events that changed the focus and trajectory of engineering at each organization. You will leave with a set of actionable suggestions in dealing with customers, engineering teams, and upper management. You will also enjoy a few war stories from the trenches.
Faster is Better. High-performing organizations deploy both substantially faster and substantially more reliably, and thus are 2.5x more likely to achieve business goals. This keynote covers how to move fast at large scale:
* Organizing for Speed
* What to Build and What NOT to Build
* When to Build
* How to Build
* Delivering and Operating
Keynote at Reversim Summit 2017 in Tel Aviv, Israel.
DevOps is far more about culture and organization than it is about technology and tooling. This talk will discuss the speaker's experiences leading high-performing engineering teams at Google, eBay, and Stitch Fix, and will offer suggestions for other organizations to level up their DevOps game.
https://www.meetup.com/SV-ELC/events/240087808/
Modern software-service models take advantage of the great benefits in having the same team both build the software as well as operate it in production -- "You Build It; You Run It" is the Amazon mantra. What does this mean in practice?
Organizationally, it means small teams with well-defined areas of responsibility, directly aligned with the business. The teams are cross-functional, meaning that each team has all the skill sets it requires to do its job, while at the same time relying on other teams for supporting services, tools, and libraries.
Process-wise, it means doubling down on practices like test-driven development and continuous delivery. Using continuous delivery practices, high-performing teams can and do release their applications and services multiple times a day. This enables them to iterate rapidly, experiment courageously, and fail more quickly.
Culturally, it means end-to-end ownership. Each team owns its software end-to-end, from design to development to deployment to retirement. The same engineers who are responsible for the features are responsible for quality, performance, operations, and maintenance. This ownership puts incentives in the right place to encourage building maintainable, observable, and operable systems from the start.
All these techniques and approaches are available to everyone, and practical examples in this talk will help other organizations on their journey.
Scaling Your Architecture for the Long TermRandy Shoup
This talk from the virtual 2020 CTO Summit (https://www.ctoconnection.com/summits) covers several architecture lessons to help you survive and thrive through the scaling phase of your company:
* Modular Architecture
* Event-Driven Communication
* Quality and Reliability
* Continuous Delivery
Minimum Viable Architecture - Good Enough is Good EnoughRandy Shoup
The “right” architecture and organization depends on the size and scale of your company. The only constant is change, and what works for 5 engineers does not work for 5000. Based upon lessons from Google and eBay, learn how to evolve both technology and organization together successfully.
This presentation is based on many hard-won lessons by the speaker, who led large-scale engineering teams at Google and eBay, but also co-founded a tiny startup and tried (unsuccessfully) to apply the same techniques. This session hopes to help others from making the same mistakes by introducing the concept of “Minimal Viable Architecture”. It outlines the common architectural evolution of a company or project through the search, execution, and scaling phases, and discusses the appropriate technologies, disciplines, and organizational structures at each phase. You'll start with a monolith, and end up with microservices, and that's completely and entirely appropriate.
DOES15 - Randy Shoup - Ten (Hard-Won) Lessons of the DevOps TransitionGene Kim
Randy Shoup, Consulting CTO
DevOps is no longer just for Internet unicorns any more. Today many large enterprises are transitioning from the slow and siloed traditional IT approach to modern DevOps practices, and getting substantial improvements in agility, velocity, scalability, and efficiency. But this transition is not without its challenges and pitfalls, and those of us who have led this journey have the scar tissue to prove it.
A successful transition to DevOps practices ultimately involves changes to organization, to culture, and to architecture. Organizationally, we want to create multi-skilled teams with end-to-end ownership and shared on-call responsibilities. Culturally, we want to prioritize solving problems and improving the product over closing tickets. Architecturally, we want to move to an infrastructure with independently testable and deployable components.
The ten practical lessons outlined in this session synthesize the speaker’s experiences leading teams at eBay, Google, and KIXEYE, as well as from his current consulting practice.
Evolving Architecture and Organization - Lessons from Google and eBayRandy Shoup
Keynote at DevOpsDays Cuba
Successful Internet companies are built on a foundation of excellent culture, efficient organization, and solid technology. As a company needs to scale, all of these parts of the foundation need to grow and scale with it. This session covers modern best practices at innovative companies in Silicon Valley for scaling culture, organization, and technology. Driven primarily by the presenter's experience ranging from small Valley startups to Google and eBay, it discusses:
* Organizing small, fast-moving engineering teams
* Building a scalable system out of smaller microservices
* Maintaining a culture of ownership and collaboration
* Developing effective engineering processes of continuous integration and continuous delivery
What if we designed our organizations like we design our systems? Applying scalability principles that we know from building large-scale distributed systems, as well as practical lessons learned at eBay and Google, this session covers how we can design and evolve our engineering organizations to scale.
How do effective large-scale service ecosystems work? Keynote Presentation at Istanbul Tech Talks 2018
How to Design Services
* Systems of record
* Interface specification
* Interface backward / forward compatibility
Service Ecosystems
* Layered services
* "Standardization" through encouragement
* Vendor-customer relationships between teams
Operating and Deploying Services
* Data Migration
* Automated Pipelines
* Incremental Deployment
* Feature Flags
Service Architectures At Scale - QCon London 2015Randy Shoup
Over time, almost all large, well-known web sites have evolved their architectures from an early monolithic application to a loosely-coupled ecosystem of polyglot microservices. While first-order goals are almost always driven by the needs of scalability and velocity, this evolution also produces second-order effects on the organization as well. This session will discuss modern service architectures at scale, using specific examples from both Google and eBay.
It covers some interesting -- and perhaps nonintuitive -- lessons learned in building and operating these sites. It concludes with a number of experience-based recommendations for other smaller organizations evolving to -- and sustaining -- an effective service ecosystem.
Machine learning has become an important tool in the modern software toolbox, and high-performing organizations are increasingly coming to rely on data science and machine learning as a core part of their business. eBay introduced machine learning to its commerce search ranking and drove double-digit increases in revenue. Stitch Fix built a multibillion dollar clothing retail business in the US by combining the best of machines with the best of humans. And WeWork is bringing machine-learned approaches to the physical office environment all around the world. In all cases, algorithmic techniques started simple and slowly became more sophisticated over time. This talk will use these examples to derive an agile approach to machine learning, and will explore that approach across several different dimensions. We will set the stage by outlining the kinds of problems that are most amenable to machine-learned approaches as well as describing some important prerequisites, including investments in data quality, a robust data pipeline, and experimental discipline. Next, we will choose the right (algorithmic) tool for the right job, and suggest how to incrementally evolve the algorithmic approaches we bring to bear. Most fancy cutting-edge recommender systems in the real world, for example, started out with simple rules-based techniques or basic regression. Finally, we will integrate machine learning into the broader product development process, and see how it can help us to accelerate business results
Monoliths, Migrations, and MicroservicesRandy Shoup
This talk describes several common challenges of software systems at scale:
* How to break up a monolithic application or a monolithic database into microservices.
* How to approach shared data, joins, and transactions in a microservices ecosystem
Minimum Viable Architecture -- Good Enough is Good Enough in a StartupRandy Shoup
I have spent the last decade building large-scale systems at eBay and Google -- and talking publicly about it -- and this presentation is about why a startup should completely ignore what I said! In an early-stage startup, it is not only not worth architecting for a future of massive scale; it is actively counterproductive. This presentation from the SF Startup CTO Summit outlines the common architectural evolution of a startup through the search, execution, and scaling phases, and discusses the appropriate technologies and disciplines at each phase. It ends with some real-world examples from eBay, Twitter, and Amazon to illustrate the point.
Managing Data at Scale - Microservices and EventsRandy Shoup
An ambitious attempt at BuildStuff España 2018 to cover, in 50 minutes:
* Migrating to Microservices
* Challenges of Data in Microservices (including shared data, joins, and transactions)
* Challenges of Event-Driven Systems (including event duplication and event ordering)
DevOpsDays Silicon Valley 2014 - The Game of OperationsRandy Shoup
Operating online games is fun and challenging. Games are some of the spikiest workloads around, and real-time really means *real-time*. Randy shares many of the DevOps techniques his team has put into practice at KIXEYE: Cloud infrastructure, Service teams, and DevOps Culture. He talks about elastic workloads, micro-services, configuration automation, and a common service "chassis". He further discusses the organizational and technical disciplines of team autonomy, internal vendor-customer relationships, and, of course, "you build it, you run it"!
There are many excellent courses, books and videos that can help you learn Agile and Scrum, but they all tend to ignore some common challenges teams run into. For example, non-functional requirements. How should we deal with them? Technical stories are also a big challenge. Good luck finding concrete, unbiased information on the pros and cons of the different approaches. Forecasting is another great example. How do you estimate or forecast an entire project?
This session will cover a number of these "elephants" in the room and provide concrete and unbiased advise on how to deal with them. All recommendations are based on real-world experiences and will include an honest discussion of relevant pros and cons.
Scaling Your Architecture with Services and EventsRandy Shoup
This session is a deep dive into the modern best practices around asynchronous decoupling, resilience, and scalability that allow us to implement a large-scale software system from the building blocks of events and services, based on the speaker's experiences implementing such systems at Google, eBay, and other high-performing technology organizations. We will outline the various options for handling event delivery and event ordering in a distributed system. We will cover data and persistence in an event-driven architecture. Finally, we will describe how to combine events, services, and so-called 'serverless' functions into a powerful overall architecture. You will leave with practical suggestions to help you accelerate your development velocity and drive business results.
Why Enterprises Are Embracing the CloudRandy Shoup
After being deeply involved in public cloud for the last several years, as both a provider and a consumer, I have been very pleasantly surprised at the rate at which large enterprises are rapidly moving to the cloud. For all the right reasons, even the most regulated and risk-averse of industries -- banking, for example -- are rapidly moving workloads out of their own owned data centers. Public cloud is not just for the "unicorns", but for the "horses" as well. This short vignette, presented at the GOTO Aarhus 2014 conference, tries to explain why this trend will continue and accelerate, and why we should be excited about it.
ONE-SIZE DOESN'T FIT ALL - EFFECTIVELY (RE)EVALUATE A DATA SOLUTION FOR YOUR ...DevOpsDays Tel Aviv
When it comes to data solutions, one-size doesn’t fit all. Choosing the right best-matching database, or data tools, can be a game changer for your system. How can you take such decision effectively? The system, the company, the product, and probably your team - all are evolving, and the best solution for today may not fit tomorrow’s needs. In order to pick a data solution for longer term, you should evaluate the optional data tools according to several factors. These factors will reflect the requirements looking forward.
At this session, we will share such use case, of evaluating data solution, when we redesigned one of Oribi features from scratch. Our goal was to avoid a data explosion crisis, while the system kept scaling up. Having so many solutions out there - we needed to make sure that we are choosing the one that will support the increasing load farthest.
Eventually we picked up ten criteria factors, which we used to compare and choose the best solution effectively. Join the session to hear what were these factors, and get prepared for choosing the next data solution for your system.
Presentation on the challenges, approaches, and reality of DevOps in large enterprises. Given at DevOps Summit 2014. Better with voice track - contact me.
Continuous Delivery in a Legacy Shop—One Step at a TimeTechWell
Not every continuous delivery (CD) initiative starts with someone saying “Drop everything. We’re going to do DevOps.” Sometimes, you have to grow your process incrementally. And sometimes you don’t set out to grow at all—you are just fixing problems with your process, trying to make things better. Gene Gotimer discusses techniques and the chain of tools he has used to bring a DevOps mindset and CD practices into a legacy environment. Gene discusses how his team started fixing problems and making process improvements in development. From there, they tackled one problem after another, each time making the release a little better and a little less risky. They incrementally brought their practices through other environments until the project was confidently delivering working and tested releases every two weeks. Gene shares their journey and the tools they used to build quality into the product, the releases, and the release process.
Presented at the Oklahoma Council for Information Technology (CoIT) Conference 2017. Want to get a taste of the ups and downs of a higher education IAM Program after a year? This presentation gives you a taste of OU's Journey.
Many scrum teams struggle with estimations and so did mine. Story points, man days, scaling. What is a story point? How to find the best task for reference?
So we decided to try something different. To try not to estimate, but use historical data instead.
Come and hear more about our jurney.
In every successful technology businesses Jeff has worked in, the key challenge has been understanding how to scale technology and when to tackle the technical debt that inevitably accrues as a company runs ever faster and faster in pursuit of its business objectives. Jeff draws on his experience to help you understand what challenges emerge as a company moves from a Developer Centric environment to become more business focused. How can you get the business people to have influence on a developer centric environment? How can you manage the challenges that marketing will present?! What principles can you apply to be aware of problems early? How do you trade Agile Practioners vs Architectural Astronauts in a fast growing business? What are the technical debt trade-offs, what problems can you buy yourself out of? What problems will kill you if you don’t move now?
DOES15 - Randy Shoup - Ten (Hard-Won) Lessons of the DevOps TransitionGene Kim
Randy Shoup, Consulting CTO
DevOps is no longer just for Internet unicorns any more. Today many large enterprises are transitioning from the slow and siloed traditional IT approach to modern DevOps practices, and getting substantial improvements in agility, velocity, scalability, and efficiency. But this transition is not without its challenges and pitfalls, and those of us who have led this journey have the scar tissue to prove it.
A successful transition to DevOps practices ultimately involves changes to organization, to culture, and to architecture. Organizationally, we want to create multi-skilled teams with end-to-end ownership and shared on-call responsibilities. Culturally, we want to prioritize solving problems and improving the product over closing tickets. Architecturally, we want to move to an infrastructure with independently testable and deployable components.
The ten practical lessons outlined in this session synthesize the speaker’s experiences leading teams at eBay, Google, and KIXEYE, as well as from his current consulting practice.
Evolving Architecture and Organization - Lessons from Google and eBayRandy Shoup
Keynote at DevOpsDays Cuba
Successful Internet companies are built on a foundation of excellent culture, efficient organization, and solid technology. As a company needs to scale, all of these parts of the foundation need to grow and scale with it. This session covers modern best practices at innovative companies in Silicon Valley for scaling culture, organization, and technology. Driven primarily by the presenter's experience ranging from small Valley startups to Google and eBay, it discusses:
* Organizing small, fast-moving engineering teams
* Building a scalable system out of smaller microservices
* Maintaining a culture of ownership and collaboration
* Developing effective engineering processes of continuous integration and continuous delivery
What if we designed our organizations like we design our systems? Applying scalability principles that we know from building large-scale distributed systems, as well as practical lessons learned at eBay and Google, this session covers how we can design and evolve our engineering organizations to scale.
How do effective large-scale service ecosystems work? Keynote Presentation at Istanbul Tech Talks 2018
How to Design Services
* Systems of record
* Interface specification
* Interface backward / forward compatibility
Service Ecosystems
* Layered services
* "Standardization" through encouragement
* Vendor-customer relationships between teams
Operating and Deploying Services
* Data Migration
* Automated Pipelines
* Incremental Deployment
* Feature Flags
Service Architectures At Scale - QCon London 2015Randy Shoup
Over time, almost all large, well-known web sites have evolved their architectures from an early monolithic application to a loosely-coupled ecosystem of polyglot microservices. While first-order goals are almost always driven by the needs of scalability and velocity, this evolution also produces second-order effects on the organization as well. This session will discuss modern service architectures at scale, using specific examples from both Google and eBay.
It covers some interesting -- and perhaps nonintuitive -- lessons learned in building and operating these sites. It concludes with a number of experience-based recommendations for other smaller organizations evolving to -- and sustaining -- an effective service ecosystem.
Machine learning has become an important tool in the modern software toolbox, and high-performing organizations are increasingly coming to rely on data science and machine learning as a core part of their business. eBay introduced machine learning to its commerce search ranking and drove double-digit increases in revenue. Stitch Fix built a multibillion dollar clothing retail business in the US by combining the best of machines with the best of humans. And WeWork is bringing machine-learned approaches to the physical office environment all around the world. In all cases, algorithmic techniques started simple and slowly became more sophisticated over time. This talk will use these examples to derive an agile approach to machine learning, and will explore that approach across several different dimensions. We will set the stage by outlining the kinds of problems that are most amenable to machine-learned approaches as well as describing some important prerequisites, including investments in data quality, a robust data pipeline, and experimental discipline. Next, we will choose the right (algorithmic) tool for the right job, and suggest how to incrementally evolve the algorithmic approaches we bring to bear. Most fancy cutting-edge recommender systems in the real world, for example, started out with simple rules-based techniques or basic regression. Finally, we will integrate machine learning into the broader product development process, and see how it can help us to accelerate business results
Monoliths, Migrations, and MicroservicesRandy Shoup
This talk describes several common challenges of software systems at scale:
* How to break up a monolithic application or a monolithic database into microservices.
* How to approach shared data, joins, and transactions in a microservices ecosystem
Minimum Viable Architecture -- Good Enough is Good Enough in a StartupRandy Shoup
I have spent the last decade building large-scale systems at eBay and Google -- and talking publicly about it -- and this presentation is about why a startup should completely ignore what I said! In an early-stage startup, it is not only not worth architecting for a future of massive scale; it is actively counterproductive. This presentation from the SF Startup CTO Summit outlines the common architectural evolution of a startup through the search, execution, and scaling phases, and discusses the appropriate technologies and disciplines at each phase. It ends with some real-world examples from eBay, Twitter, and Amazon to illustrate the point.
Managing Data at Scale - Microservices and EventsRandy Shoup
An ambitious attempt at BuildStuff España 2018 to cover, in 50 minutes:
* Migrating to Microservices
* Challenges of Data in Microservices (including shared data, joins, and transactions)
* Challenges of Event-Driven Systems (including event duplication and event ordering)
DevOpsDays Silicon Valley 2014 - The Game of OperationsRandy Shoup
Operating online games is fun and challenging. Games are some of the spikiest workloads around, and real-time really means *real-time*. Randy shares many of the DevOps techniques his team has put into practice at KIXEYE: Cloud infrastructure, Service teams, and DevOps Culture. He talks about elastic workloads, micro-services, configuration automation, and a common service "chassis". He further discusses the organizational and technical disciplines of team autonomy, internal vendor-customer relationships, and, of course, "you build it, you run it"!
There are many excellent courses, books and videos that can help you learn Agile and Scrum, but they all tend to ignore some common challenges teams run into. For example, non-functional requirements. How should we deal with them? Technical stories are also a big challenge. Good luck finding concrete, unbiased information on the pros and cons of the different approaches. Forecasting is another great example. How do you estimate or forecast an entire project?
This session will cover a number of these "elephants" in the room and provide concrete and unbiased advise on how to deal with them. All recommendations are based on real-world experiences and will include an honest discussion of relevant pros and cons.
Scaling Your Architecture with Services and EventsRandy Shoup
This session is a deep dive into the modern best practices around asynchronous decoupling, resilience, and scalability that allow us to implement a large-scale software system from the building blocks of events and services, based on the speaker's experiences implementing such systems at Google, eBay, and other high-performing technology organizations. We will outline the various options for handling event delivery and event ordering in a distributed system. We will cover data and persistence in an event-driven architecture. Finally, we will describe how to combine events, services, and so-called 'serverless' functions into a powerful overall architecture. You will leave with practical suggestions to help you accelerate your development velocity and drive business results.
Why Enterprises Are Embracing the CloudRandy Shoup
After being deeply involved in public cloud for the last several years, as both a provider and a consumer, I have been very pleasantly surprised at the rate at which large enterprises are rapidly moving to the cloud. For all the right reasons, even the most regulated and risk-averse of industries -- banking, for example -- are rapidly moving workloads out of their own owned data centers. Public cloud is not just for the "unicorns", but for the "horses" as well. This short vignette, presented at the GOTO Aarhus 2014 conference, tries to explain why this trend will continue and accelerate, and why we should be excited about it.
ONE-SIZE DOESN'T FIT ALL - EFFECTIVELY (RE)EVALUATE A DATA SOLUTION FOR YOUR ...DevOpsDays Tel Aviv
When it comes to data solutions, one-size doesn’t fit all. Choosing the right best-matching database, or data tools, can be a game changer for your system. How can you take such decision effectively? The system, the company, the product, and probably your team - all are evolving, and the best solution for today may not fit tomorrow’s needs. In order to pick a data solution for longer term, you should evaluate the optional data tools according to several factors. These factors will reflect the requirements looking forward.
At this session, we will share such use case, of evaluating data solution, when we redesigned one of Oribi features from scratch. Our goal was to avoid a data explosion crisis, while the system kept scaling up. Having so many solutions out there - we needed to make sure that we are choosing the one that will support the increasing load farthest.
Eventually we picked up ten criteria factors, which we used to compare and choose the best solution effectively. Join the session to hear what were these factors, and get prepared for choosing the next data solution for your system.
Presentation on the challenges, approaches, and reality of DevOps in large enterprises. Given at DevOps Summit 2014. Better with voice track - contact me.
Continuous Delivery in a Legacy Shop—One Step at a TimeTechWell
Not every continuous delivery (CD) initiative starts with someone saying “Drop everything. We’re going to do DevOps.” Sometimes, you have to grow your process incrementally. And sometimes you don’t set out to grow at all—you are just fixing problems with your process, trying to make things better. Gene Gotimer discusses techniques and the chain of tools he has used to bring a DevOps mindset and CD practices into a legacy environment. Gene discusses how his team started fixing problems and making process improvements in development. From there, they tackled one problem after another, each time making the release a little better and a little less risky. They incrementally brought their practices through other environments until the project was confidently delivering working and tested releases every two weeks. Gene shares their journey and the tools they used to build quality into the product, the releases, and the release process.
Presented at the Oklahoma Council for Information Technology (CoIT) Conference 2017. Want to get a taste of the ups and downs of a higher education IAM Program after a year? This presentation gives you a taste of OU's Journey.
Many scrum teams struggle with estimations and so did mine. Story points, man days, scaling. What is a story point? How to find the best task for reference?
So we decided to try something different. To try not to estimate, but use historical data instead.
Come and hear more about our jurney.
In every successful technology businesses Jeff has worked in, the key challenge has been understanding how to scale technology and when to tackle the technical debt that inevitably accrues as a company runs ever faster and faster in pursuit of its business objectives. Jeff draws on his experience to help you understand what challenges emerge as a company moves from a Developer Centric environment to become more business focused. How can you get the business people to have influence on a developer centric environment? How can you manage the challenges that marketing will present?! What principles can you apply to be aware of problems early? How do you trade Agile Practioners vs Architectural Astronauts in a fast growing business? What are the technical debt trade-offs, what problems can you buy yourself out of? What problems will kill you if you don’t move now?
Effective Microservices In a Data-centric WorldRandy Shoup
From a talk at GOTOChicago 2017, these slides discuss the speaker's experiences at Stitch Fix with
* Organizational, Process, and Cultural prerequisites for being successful with Microservices: small teams, TDD / CD, DevOps
* How to handle shared data when your data is split among microservices
* How to handle "joins" across microservices
* How to simulate "transactions" across microservices
Slides link: https://gotochgo.com/3/sessions/79/slides
Video link: https://gotochgo.com/3/sessions/79/video
Software Release Orchestration and the EnterpriseXebiaLabs
Or, How ING Streamlined and Increased Software Deployments to Twice a Day
Enterprises are realizing that doing DevOps right requires a streamlined Continuous Delivery pipeline that spans many groups beyond Dev and Ops. Finding a way to automate and control modern DevOps processes while maintaining visibility is a huge a challenge.
Hear from Andréas Prins, IT Manager at ING and Andrew Phillips, VP of DevOps Strategy at XebiaLabs, as they discuss the challenges enterprises are facing and offer actionable advice on how to:
More easily manage complex, distributed releases across technical and non-technical teams
Gain better control and oversight of your DevOps automation and overall software delivery process
Provide visibility into your Continuous Delivery process for everyone involved in your DevOps initiative
Release more quickly, identify bottlenecks, reduce errors and lower the risk of release failures
Performing an R12 Upgrade in a Highly Customized Environment with a Worldwide...AXIA Consulting Inc.
Performing an upgrade is a challenging task. Performing an upgrade where the environment contains a significant number of customizations adds significant complexity and risk. This presentation will detail the approach, steps and pitfalls that are encountered and how the upgrade can still be successful. This upgrade contains a number of additional challenges, including the need to support processes for over 40 countries.
Data Science Salon: Quit Wasting Time – Case Studies in Production Machine Le...Formulatedby
Presented by Yashas Vaidya, Sr Data Scientist at DataIku
Next DSS MIA Event - https://datascience.salon/miami/
The steps to taking a machine learning model to production. Modern architectures and technologies for building production machine learning. An overview of the talent and processes for creating and maintaining production machine learning.
How to overcome challenges in it system evolutionGrupa Unity
This presentation shows the problems and challenges in IT system evolution and the ways to deal with them, from Architect & Team Leader and Project & Portfolio Manager perspective
Standard Bank: How APM Supports DevOps, Agile and Engineering Transformation ...AppDynamics
Standard Bank's goal is to be the leading financial services organisation in, for, and across Africa. They’re investing heavily in their omni-channel capability delivered through self-service channels that sit on a modern software stack. Their strategy requires that they frequently deliver new functionality to customers across countries, channels, and business domains. To support this, Standard Bank has moved to modern engineering practices, including DevOps, test-driven development, and Agile. In this session, Andrew and Lenro from Standard Bank will outline their journey and discuss how APM has helped their teams, promoted a DevOps culture and driven transformation. They’ll cover:
-How we structure our development teams and what DevOps means to us
-How APM unblocks delivery and help deliver quality software to our customers
-APM as a DevOps enabler, helping bridge the gap between operations and development
To learn more, visit: www.appdynamics.com
Hanno Jarvet - VSM, Planning and Problem Solving - ConFuDevConFu
Value stream mapping is a Lean technique used to analyse and design the flow of materials and information required to bring a product or service to a consumer. It can be used for nearly any value chain, line of business and group of processes to optimize their results and efficiency.
During the hands on work-shop each participant will have the opportunity to work with their actual business problems and walk away with a clear roadmap on what to improve and why.
How Microsoft ALM Tools Can Improve Your Bottom LineImaginet
Improved efficiencies, enhanced productivity, reduction of wasted time and effort, and improved team collaboration. Each of these benefits that result from adopting a successful ALM strategy will all help your bottom line. Come find out how at this free webinar!
Doing Analytics Right - Building the Analytics EnvironmentTasktop
Implementing analytics for development processes is challenging. As in discussed in the previous webinars, the right analytics are determined by the goals of the organization, not by the available data. So implementing your analytics solutions will require an efficient analytics and data architecture, including the ability to combine and stage data from heterogeneous sources. An architecture that excludes the ability to gain access to the necessary data will create a barrier to deploying your newly designed analytics program, and will force you back into the “light is brighter here” anti-pattern.
This webinar will describe the technical considerations of implementing the data architecture for your analytics program, and explain how Tasktop can help.
Standard Bank: Agile, DevOps, Engineering Transformation and the Role of AppD...AppDynamics
Standard Bank's goal is to be the leading financial services organization in, for, and across Africa. We are investing heavily in a global digital strategy oriented around omni-channel capability delivered through self-service channels.
This strategy requires that we frequently deliver new functionality to customers across countries, channels, and business domains. To support this we must adopt sound engineering practices such as DevOps, test-driven development, and Agile.
In this session, we will highlight some of the key Agile engineering practices that support the journey and the role that AppDynamics has played in enabling these practices.
Key takeaways:
o How AppDynamics unblocks delivery and help to deliver quality software to our customers
o AppDynamics as a DevOps enabler, helping bridge the gap between operations and development
o Scenarios in which AppDynamics has prevented outages and reduced the amount of time it would take to restore service
For more information, go to: www.appdynamics.com
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityRandy Shoup
Building distributed systems that work is hard. And scaling those systems by multiple orders of magnitude is even harder. Using examples from internet-scale consumer properties like Google, Amazon, and eBay, this talk deep-dives into the counterintuitive idea that the key to success in large-scale architecture is simplicity.
We first discuss simple components like modular services, orthogonal domain logic, and service layering. Next we discuss simple interactions between components, leveraging event-driven models, immutable logs, and asynchronous dataflow. Then we explore techniques that simplify making changes the system, including incremental changes, continuous testing, canary deployments, and feature flags.
In the final part of the talk, we show how all these ideas work together with specific architectural examples from Amazon, Netflix, and Walmart.
You will take away actionable insights you can immediately put into practice in your own systems.
Similar to Anatomy of Three Incidents -- Commonalities and Lessons (20)
Breaking Codes, Designing Jets, and Building TeamsRandy Shoup
Throughout engineering history, focused and empowered teams have consistently achieved the near-impossible. Alan Turing, Tommy Flowers, and their teams at Bletchley Park broke Nazi codes, saved their country, and brought down the Third Reich. Kelly Johnson and the Lockheed Skunk Works designed and built the XP-80 in 143 days, and later produced the U-2, the SR-71, and the F-22. Xerox PARC invented Smalltalk, graphical user interfaces, Ethernet, and the laser printer. What can this history teach us? Well, basically everything.
Effective teams have a purpose - a clearly defined problem which the entire team focuses on and owns end-to-end. Effective teams have an organizational culture that prioritizes collaboration and learning. And most importantly, effective teams are made up of people from diverse backgrounds and experiences.
If this sounds a lot like DevOps, or true little-a agile, that's no coincidence. But too few organizations actually practice these three-quarter-century-old ideas despite the overwhelming evidence that they work. So let's relearn those history lessons.
From the DevOps Enterprise Summit 2015, this presentation covers hard-won lessons of transitioning an engineering organization to DevOps. See video at https://www.youtube.com/watch?v=6tREbJl8e_Y.
Lessons:
1. Reorganize around Ownership
2. Lose the Ticket Culture
3. Replace Approvals with Code
4. Enforce a Service Mentality
5. Charge for Usage
6. Prioritize Quality
7. Start Investing in Testing
8. Actively Manage Technical Debt
9. Share On-Call Duties
10. Make Post-Mortems Truly Blameless
DevOps is no longer just for Internet unicorns any more. Today many large enterprises are transitioning from the slow and siloed traditional IT approach to modern DevOps practices, and getting substantial improvements in agility, velocity, scalability, and efficiency. But this transition is not without its challenges and pitfalls, and those of us who have led this journey have the scar tissue to prove it.
A successful transition to DevOps practices ultimately involves changes to organization, to culture, and to architecture. Organizationally, we want to create multi-skilled teams with end-to-end ownership and shared on-call responsibilities. Culturally, we want to prioritize solving problems and improving the product over closing tickets. Architecturally, we want to move to an infrastructure with independently testable and deployable components.
The ten practical lessons outlined in this session synthesize the speaker’s experiences leading teams at eBay, Google, and KIXEYE, as well as from his former consulting practice.
From a talk at the SF CTO Summit 2017 (https://www.ctoconnection.com/summits/sf2017), these slides cover the speaker's experience at Stitch Fix with managing data in a microservices environment. Areas include:
* Breaking up a monolithic database into services
* Using events as a first-class part of your architecture
* Sharing data among microservices
* Handling "joins" among microservices
* Simulating "transactions" among microservices using the Saga pattern
One of the most powerful trends in software today is building large systems out of composable microservices. Many large-scale web companies have migrated over time to this architecture – and for good reason. But, as with any powerful technique, microservices come with their own brand of tradeoffs, and it is important to be aware of them before deciding whether they are appropriate in any particular case. They are not for every scale of problem, for every stage of company, or for every team.
This session takes a pragmatic approach to microservices, and compares them to the alternatives at different stages of company evolution. Using examples both from Google and eBay as well as from smaller organizations, it makes practical suggestions about whether, when, and how an organization should consider adopting a microservices architecture. Assuming microservices are the appropriate choice, it outlines an experience-based, incremental approach to making a successful rearchitecture to microservices.
From the Monolith to Microservices - CraftConf 2015Randy Shoup
Most large-scale web companies have evolved their system architecture from a monolithic application and monolithic database to a set of loosely coupled microservices. Using examples from Google, eBay, and other large-scale sites, this talk outlines the pros and cons of these different stages of evolution, and makes practical suggestions about when and how other organizations should consider migrating to microservices. It continues with some more advanced implications of a microservices architecture, including SLAs, cost-allocation, and vendor-customer relationships within the organization. It concludes by exploring a set of common service anti-patterns.
Concurrency at Scale: Evolution to Micro-ServicesRandy Shoup
Most large-scale web companies have evolved their system architecture from a monolithic application and monolithic database to a set of loosely coupled micro-services. Using examples from Google, eBay, and KIXEYE, this talk outlines the pros and cons of these different stages of evolution, and makes practical suggestions about when and how other organizations should consider migrating to micro-services. It concludes with some more advanced implications of a micro-services architecture, including SLAs, cost-allocation, and vendor-customer relationships within the organization.
QCon New York 2014 - Scalable, Reliable Analytics Infrastructure at KIXEYERandy Shoup
As a maker of real-time strategy games for web and mobile, KIXEYE's business depends on deep insights into how players play our games. By analyzing player behavior in a rich and flexible way, we are able to better target our efforts around user acquisition, game balance, player retention, and game monetization. By storing and analyzing data in standard ways, our data scientists are better able to take learnings from one game and apply them to another.
This presentation describes KIXEYE's newly-minted modern analytics infrastructure soup-to-nuts, from Kafka queues through Hadoop 2 to Hive and Redshift. It outlines our efforts around queryability, extensibility, scalability, standardization, and stability and outage recovery. It further shares our lessons learned in building, testing, operating, and enhancing this mission-critical piece of our infrastructure.
QCon Tokyo 2014 - Virtuous Cycles of Velocity: What I Learned About Going Fas...Randy Shoup
eBay and Google operate some of the largest Internet sites on the planet, and each maintains its leadership through continuous innovation in infrastructure and products. While substantially different in their detailed approaches, both organizations sustain their feature velocity through a combination of organizational culture, process, and people. This session will explore how these large-scale sites do it, and will offer some concrete suggestions on how other organizations -- both large and small -- can do the same.
The Importance of Culture: Building and Sustaining Effective Engineering Org...Randy Shoup
Randy is a 25-year veteran of Silicon Valley, having led engineering organizations at eBay, Google, Oracle, and a number of other companies. Through the lens of his personal experience from hands-on engineer to architect to CTO, at organizations ranging from tiny startups to global giants, Randy will discuss several important aspects of engineering cultures, which both support and hinder the ability to innovate: hiring and retention, ownership and collaboration, quality and discipline, and learning and experimentation.
Randy will suggest some learnings about what has worked well -- and what has not -- in creating and sustaining an effective engineering culture. He will further offer some concrete suggestions on how other organizations -- both large and small -- can evolve their cultures as well.
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Strategies for Successful Data Migration Tools.pptxvarshanayak241
Data migration is a complex but essential task for organizations aiming to modernize their IT infrastructure and leverage new technologies. By understanding common challenges and implementing these strategies, businesses can achieve a successful migration with minimal disruption. Data Migration Tool like Ask On Data play a pivotal role in this journey, offering features that streamline the process, ensure data integrity, and maintain security. With the right approach and tools, organizations can turn the challenge of data migration into an opportunity for growth and innovation.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
Your Digital Assistant.
Making complex approach simple. Straightforward process saves time. No more waiting to connect with people that matter to you. Safety first is not a cliché - Securely protect information in cloud storage to prevent any third party from accessing data.
Would you rather make your visitors feel burdened by making them wait? Or choose VizMan for a stress-free experience? VizMan is an automated visitor management system that works for any industries not limited to factories, societies, government institutes, and warehouses. A new age contactless way of logging information of visitors, employees, packages, and vehicles. VizMan is a digital logbook so it deters unnecessary use of paper or space since there is no requirement of bundles of registers that is left to collect dust in a corner of a room. Visitor’s essential details, helps in scheduling meetings for visitors and employees, and assists in supervising the attendance of the employees. With VizMan, visitors don’t need to wait for hours in long queues. VizMan handles visitors with the value they deserve because we know time is important to you.
Feasible Features
One Subscription, Four Modules – Admin, Employee, Receptionist, and Gatekeeper ensures confidentiality and prevents data from being manipulated
User Friendly – can be easily used on Android, iOS, and Web Interface
Multiple Accessibility – Log in through any device from any place at any time
One app for all industries – a Visitor Management System that works for any organisation.
Stress-free Sign-up
Visitor is registered and checked-in by the Receptionist
Host gets a notification, where they opt to Approve the meeting
Host notifies the Receptionist of the end of the meeting
Visitor is checked-out by the Receptionist
Host enters notes and remarks of the meeting
Customizable Components
Scheduling Meetings – Host can invite visitors for meetings and also approve, reject and reschedule meetings
Single/Bulk invites – Invitations can be sent individually to a visitor or collectively to many visitors
VIP Visitors – Additional security of data for VIP visitors to avoid misuse of information
Courier Management – Keeps a check on deliveries like commodities being delivered in and out of establishments
Alerts & Notifications – Get notified on SMS, email, and application
Parking Management – Manage availability of parking space
Individual log-in – Every user has their own log-in id
Visitor/Meeting Analytics – Evaluate notes and remarks of the meeting stored in the system
Visitor Management System is a secure and user friendly database manager that records, filters, tracks the visitors to your organization.
"Secure Your Premises with VizMan (VMS) – Get It Now"
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
Worried about document security while sharing them in Salesforce? Fret no more! Here are the top-notch security standards XfilesPro upholds to ensure strong security for your Salesforce documents while sharing with internal or external people.
To learn more, read the blog: https://www.xfilespro.com/how-does-xfilespro-make-document-sharing-secure-and-seamless-in-salesforce/
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
6. App Engine Reliability Fixit
• Step 1: Identify the Problem
o All team leads and senior engineers met in a room with a whiteboard
o Enumerated all known and suspected reliability issues
o Too much technical debt had accumulated
o Reliability issues had not been prioritized
o Identify 8-10 themes
@randyshoup
7. • Step 2: Understand the Problem
o Each theme assigned to a senior engineer to investigate
o Timeboxed for 1 week
o After 1 week, all leads came back with
• Detailed list of issues
• Recommended steps to address them
• Estimated order-of-magnitude of effort (1 day, 1 week, 1 month, etc.)
App Engine Reliability Fixit
@randyshoup
8. • Step 3: Consensus and Prioritization
o Leads discussed themes and prioritized work
o Assigned engineers to tasks
App Engine Reliability Fixit
@randyshoup
9. • Step 4: Implementation and Follow-up
o Engineers worked on assigned tasks
o Simple spreadsheet of task status, which engineers updated weekly
o Minimal effort from management (~1 hour / week) to summarize progress at
weekly team meeting
App Engine Reliability Fixit
@randyshoup
10. • Results
o 10x reduction in reliability issues
o Improved team cohesion and camaraderie
o Broader participation and ownership of the future health of the platform
o Still remembered several years later
App Engine Reliability Fixit
@randyshoup
12. Stitch Fix – Oct / Nov 2016
• (11/08/2016) Spectre unavailable for ~3 minutes [Shared Database]
• (11/05/2016) Spectre unavailable for ~5 minutes [Shared Database]
• (10/25/2016) All systems unavailable for ~5 minutes [Shared Database]
• (10/24/2016) All systems unavailable for ~5 minutes [Shared Database]
• (10/21/2016) All systems unavailable for ~3 ½ hours [DDOS attack]
• (10/18/2016) All systems unavailable for ~3 minutes [Shared Database]
• (10/17/2016) All systems unavailable for ~20 minutes [Shared Database]
• (10/13/2016) Minx escalation broken for ~2 hours [Zendesk outage]
• (10/11/2016) Label printing unavailable for ~10 minutes [FedEx outage]
• (10/10/2016) Label printing unavailable for ~15 minutes [FedEx outage]
• (10/10/2016) All systems unavailable for ~10 minutes [Shared Database]
@randyshoup
13. Database Stability Problems
• 1. Applications contended on common tables
• 2. Scalability limited by database connections
• 3. One application could take down entire company
@randyshoup
14. Stability Retrospective
• Step 1: Identify the Problem
• Step 2: Understand the Problem
• Step 3: Consensus and Prioritization
• Step 4: Implementation and Follow-Up
• Results
@randyshoup
15. Stability Solutions
• 1. Focus on expensive queries
o Log
o Eliminate
o Rewrite
o Reduce
• 2. Manage database connections via connection concentrator
• 3. Stability and Scalability Program
o Ongoing 25% investment in services migration
@randyshoup
17. Login Issues - 2019
• Problem: Some members unable to log in
• Inconsistent representations across different services in the
system
• Over time, simple system interactions grew increasingly
complex and convoluted
• Not enough graceful degradation or automated repair
@randyshoup
18. Login Retrospective
• Step 1: Identify the Problem
• Step 2: Understand the Problem
• Step 3: Consensus and Prioritization
• Step 4: Implementation and Follow-Up
@randyshoup
19. Login Solutions
• 1. Clean up user data
o Find inconsistencies
o Track inconsistency metrics
o Identify and fix contributing processes and applications
• 2. User state machines
o Define user journeys as explicit state machines
o Refine and correct via cross-functional feedback
o Implement state machines in code
• 3. “Pandora” Program
o Rewrite core identity system into set of user capabilities
@randyshoup
20. Common Elements
• Unintentional, long-term accumulation of
small, individually reasonable decisions
• “Compelling event” catalyzes long-term
change
• Blameless culture makes learning and
improvement possible
• Structured post-incident approach
@randyshoup
29. During the Incident
• Focus on restoring service
o Everything else is secondary, and should wait
• Shield the team
• Clear, structured communication
o Even when there is nothing to report!
@randyshoup https://myresources.itrevolution.com/id006657105/A-Framework-for-Incident-Response
30. After the Incident
• Blameless postmortem
• Identify and understand the
contributing factors
• Action items and Learnings
• Follow Up!
@randyshoup https://myresources.itrevolution.com/id006657105/A-Framework-for-Incident-Response
31. Psychological Safety
• Team is safe for interpersonal
risk-taking
• “Being able to show and employ
one’s self without fear of
negative consequences”
• More important than any other
factor in team success
32. “Finally we can prioritize
fixing that broken system!”
@randyshoup
33. Inclusive Decisionmaking
• Make better business decisions
87% of the time
• Make decisions 2x faster with
1/2 the meetings
• Deliver 60% better business
results
Cloverpop Inclusive Decisionmaking study, 2016
As we improve diversity, decisionmaking improves
@randyshoup
38. “Incidents are unplanned
investments, and they are also
opportunities. Your challenge
is to maximize the ROI on the
sunk cost.”
@randyshoup
-- John Allspaw, Adaptive Capacity Labs
39. Improvement Budget
• Explicit resource investment
o Agree on an up-front investment
(e.g., 25%, 30% of engineering efforts)
• Retain autonomy, Provide transparency
o Making these decisions is exactly why they hired you
@randyshoup