The ability to grow (and shrink) according to the needs and the available resources is an essential part of designing applications. In this talk we'll cover the fundamental elements of scalability, including aspects involving people, processes and technology. With sound and proven principles and some advice on how to shape your organisation, set the right processes and design your application, this session is a must-see for developers and technical leads alike.
Hardware planning & sizing for sql serverDavide Mauri
Purchasing a dedicated server to SQL Server is still a necessary operation. The cloud is a great choice but if you need to create a data warehouse of non-trivial size or if you have the need for optimal performance and control of your production database server, the choice of on-premise server is still an optimal choice. So, how not to throw away money on unnecessary hardware? In this session we will see how each component works together to form a balanced hardware (this is the key word!), without bottlenecks, maximizing the investment made. We'll talk about SAN, CPU, HBA, Fibre Channel, Memory and everything you thought you knew well...
CSI – IT2020, IIT Mumbai, October 6th 2017
Computer Society of India, Mumbai Chapter
The presentation focuses on Microservices architecture and the comparison between MicroService with Standard Monolithic Apps and SOA based Apps. It also gives a quick outline of Domain Driven Design, Event Sourcing and CQRS, Functional Reactive Programming and comparison of SAGA pattern with 2 Phase Commit.
http://www.csimumbai.org/it2020-17/index.html
Pros and Cons of a MicroServices Architecture talk at AWS ReInventSudhir Tonse
Netflix morphed from a private datacenter based monolithic application into a cloud based Microservices architecture. This talk highlights the pros and cons of building software applications as suites of independently deployable services, as well as practical approaches for overcoming challenges - especially in the context of an elastic but ephemeral cloud ecosystem. What were the lessons learned while building and managing these services? What are the best practices and anti-patterns?
Where SOA and Monolitch EAR have failed. It's not simple to have your Apps scaling automagically without a very complex architecture. We're going to show pros and cons of so called Cloud-Native Applications based on Microservices, Caas, DevOps, Continuous Delivery....
When prioritizing requirements in a project, have you ever been in a situation in which virtually all requirements are High Priority or Critical? As you can imagine, ALL requirements being High priority is as "good" as NO requirements having ANY priority at all. Hmm, not very helpful, isn't it? Is there anything we can do about that?
In this presentation/workshop we'll go through some ideas and practices on how to improve the requirements prioritization process.
Agenda topics:
- Why are we talking about Requirements Prioritization?
- What are we talking about?
- Who cares? Why?
- When do (should) we do it?
- How do we do it? Some useful techniques...
- Pitfalls & "Best" Practices
The workshop goes beyond the knowledge presented in this document, working as team with a faster and better Prioritization Process. The outcomes of that experiment in a future presentation.
The ability to grow (and shrink) according to the needs and the available resources is an essential part of designing applications. In this talk we'll cover the fundamental elements of scalability, including aspects involving people, processes and technology. With sound and proven principles and some advice on how to shape your organisation, set the right processes and design your application, this session is a must-see for developers and technical leads alike.
Hardware planning & sizing for sql serverDavide Mauri
Purchasing a dedicated server to SQL Server is still a necessary operation. The cloud is a great choice but if you need to create a data warehouse of non-trivial size or if you have the need for optimal performance and control of your production database server, the choice of on-premise server is still an optimal choice. So, how not to throw away money on unnecessary hardware? In this session we will see how each component works together to form a balanced hardware (this is the key word!), without bottlenecks, maximizing the investment made. We'll talk about SAN, CPU, HBA, Fibre Channel, Memory and everything you thought you knew well...
CSI – IT2020, IIT Mumbai, October 6th 2017
Computer Society of India, Mumbai Chapter
The presentation focuses on Microservices architecture and the comparison between MicroService with Standard Monolithic Apps and SOA based Apps. It also gives a quick outline of Domain Driven Design, Event Sourcing and CQRS, Functional Reactive Programming and comparison of SAGA pattern with 2 Phase Commit.
http://www.csimumbai.org/it2020-17/index.html
Pros and Cons of a MicroServices Architecture talk at AWS ReInventSudhir Tonse
Netflix morphed from a private datacenter based monolithic application into a cloud based Microservices architecture. This talk highlights the pros and cons of building software applications as suites of independently deployable services, as well as practical approaches for overcoming challenges - especially in the context of an elastic but ephemeral cloud ecosystem. What were the lessons learned while building and managing these services? What are the best practices and anti-patterns?
Where SOA and Monolitch EAR have failed. It's not simple to have your Apps scaling automagically without a very complex architecture. We're going to show pros and cons of so called Cloud-Native Applications based on Microservices, Caas, DevOps, Continuous Delivery....
When prioritizing requirements in a project, have you ever been in a situation in which virtually all requirements are High Priority or Critical? As you can imagine, ALL requirements being High priority is as "good" as NO requirements having ANY priority at all. Hmm, not very helpful, isn't it? Is there anything we can do about that?
In this presentation/workshop we'll go through some ideas and practices on how to improve the requirements prioritization process.
Agenda topics:
- Why are we talking about Requirements Prioritization?
- What are we talking about?
- Who cares? Why?
- When do (should) we do it?
- How do we do it? Some useful techniques...
- Pitfalls & "Best" Practices
The workshop goes beyond the knowledge presented in this document, working as team with a faster and better Prioritization Process. The outcomes of that experiment in a future presentation.
The rise of NoSQL is characterized with confusion and ambiguity; very much like any fast-emerging organic movement in the absence of well-defined standards and adequate software solutions. Whether you are a developer or an architect, many questions come to mind when faced with the decision of where your data should be stored and how it should be managed. The following are some of these questions: What does the rise of all these NoSQL technologies mean to my enterprise? What is NoSQL to begin with? Does it mean "No SQL"? Could this be just another fad? Is it a good idea to bet the future of my enterprise on these new exotic technologies and simply abandon proven mature Relational DataBase Management Systems (RDBMS)? How scalable is scalable? Assuming that I am sold, how do I choose the one that fit my needs best? Is there a middle ground somewhere? What is this Polyglot Persistence I hear about? The answers to these questions and many more is the subject of this talk along with a survey of the most popular of NoSQL technologies. Be there or be square.
Developing applications with a microservice architecture (SVforum, microservi...Chris Richardson
Here is the version of my microservices talk that that I gave on September 17th at the SVforum Cloud SIG/Microservices meetup.
To learn more see http://microservices.io and http://plainoldobjects.com
Any team that has made the jump from building monoliths to building microservices knows the complexities you must overcome to build a system that is functional and maintainable. Building a microservice architecture that is low latency and only communicates using REST APIs is even more tricky, with high latency for requests being a common concern. This talk explains how you can use events as the backbone of your microservice architecture and build an efficient, event-driven system. It covers how to get started with designing your microservice architecture and the key requirements any system needs to fulfil. It also introduces the different patterns you will encounter in event-driven architectures and the advantages and disadvantages of these choices. Finally it explains why Apache Kafka is a great choice for event-driven microservices.
Explore the challenges of migrating existing applications to a cloud infrastructure, then present proven strategies for mitigating the risks You'll learn how to:
- Prioritize application migration
- Plan for “big-blocks”
- Assess your existing applications' ‘fit’ for cloud
- Leverage a centralized development testing platform to align your business goals with coding decisions
- Create a cloud migration policy
- Use process to mitigate the risks associated with cloud migration
Agile 2013 - Lean Change for Enabling Agile TransformationsAlexis Hui
Experience report summarizing our experiences with agile transformation in mid-large sized IT organizations and challenges we faced with current methods available. As a result, we saw a need for a better change approach to help us and others in the agile community with agile transformations. Borrowing thinking and tools from Lean Startup, Kanban and Kotter we have defined a structured framework known as Lean Change. The premise behind our thinking is that successful agile transformation requires learning and feedback as the keys for success. Lean Change is founded on three concepts, co-creation of changes through negotiated change, experiment based objectives using minimum viable changes, and validated learning to guide changes through a structured validation lifecycle.
Composable Software Architecture with SpringSam Brannen
What does the architecture of a modern enterprise Java application look like? What have we as a community learned from our past? What does it mean to design a composable architecture? And how can Spring help developers meet the needs of enterprise applications in 2013 and beyond?
In this keynote presentation at Java Breeze, core Spring Framework committer Sam Brannen invites the audience to explore what it means to design, develop, and test modern enterprise Java applications following a composable software architecture model. Along the way, Sam will show how the Spring ecosystem and programming model fit into the larger picture of modern enterprise Java applications.
Lessons learned processing 70 billion data points a day using the hybrid cloudDataWorks Summit
NetApp receives 70 billion data points of telemetry information each day from its customer’s storage systems. This telemetry data contains configuration information, performance counters, and logs. All of this data is processed using multiple Hadoop clusters, and feeds a machine learning pipeline and a data serving infrastructure that produces insights for customers via an application called Active IQ. We describe the evolution of our Hadoop infrastructure from a traditional on-premises architecture to the hybrid cloud, and lessons learned.
We’ll discuss the insights we are able to produce for our customers, and the techniques used. Finally, we describe the data management challenges with our multi-petabyte Hadoop data lake. We solved these problems by building a unified data lake on-premises and using the NetApp Data Fabric to seamlessly connect to public clouds for data science and machine learning compute resources.
Architecting a truly hybrid cloud implementation allowed NetApp to free up our data scientists to use any software on any cloud, kept the customer log data safe on NetApp Private Storage in Equinix, resulted in faster ability to innovate and release new code and provided flexibility to use any public cloud at the same time with data on NetApp in Equinix.
Speaker
Pranoop Erasani, NetApp, Senior Technical Director, ONTAP
Shankar Pasupathy, NetApp, Technical Director, ACE Engineering
This is a talk I gave at PLoP 2017 - http://www.hillside.net/plop/2017/index.php?nav=program
The microservice architecture is growing in popularity. It is an architectural style that structures an application as a set of loosely coupled services that are organized around business capabilities. Its goal is to enable the continuous delivery of large, complex applications. However, the microservice architecture is not a silver bullet and it has some significant drawbacks.
The goal of the microservices pattern language is to enable software developers to apply the microservice architecture effectively. It is a collection of patterns that solve architecture, design, development and operational problems. In this talk, I’ll provide an overview of the microservice architecture and describe the motivations for the pattern language. You will learn about the key patterns in the pattern language.
Monitoring at scale - Intuitive dashboard designLorenzo Alberton
At a certain scale, millions of events happen every second, and all of them are important to evaluate the health of the system. If not handled correctly, such a volume of information can overwhelm both the infrastructure that needs to support them, and people who have to make a sense out of thousands of signals and make decisions upon them, fast. By understanding how our rational mind works, how people process information, we can present data so it's more evident and intuitive. This talk will explain how to collect useful metrics, and to create the perfect monitoring dashboard to organise and display them, letting our intuition operate automatically and quickly, and saving attention and mental effort to activities that demand it.
The rise of NoSQL is characterized with confusion and ambiguity; very much like any fast-emerging organic movement in the absence of well-defined standards and adequate software solutions. Whether you are a developer or an architect, many questions come to mind when faced with the decision of where your data should be stored and how it should be managed. The following are some of these questions: What does the rise of all these NoSQL technologies mean to my enterprise? What is NoSQL to begin with? Does it mean "No SQL"? Could this be just another fad? Is it a good idea to bet the future of my enterprise on these new exotic technologies and simply abandon proven mature Relational DataBase Management Systems (RDBMS)? How scalable is scalable? Assuming that I am sold, how do I choose the one that fit my needs best? Is there a middle ground somewhere? What is this Polyglot Persistence I hear about? The answers to these questions and many more is the subject of this talk along with a survey of the most popular of NoSQL technologies. Be there or be square.
Developing applications with a microservice architecture (SVforum, microservi...Chris Richardson
Here is the version of my microservices talk that that I gave on September 17th at the SVforum Cloud SIG/Microservices meetup.
To learn more see http://microservices.io and http://plainoldobjects.com
Any team that has made the jump from building monoliths to building microservices knows the complexities you must overcome to build a system that is functional and maintainable. Building a microservice architecture that is low latency and only communicates using REST APIs is even more tricky, with high latency for requests being a common concern. This talk explains how you can use events as the backbone of your microservice architecture and build an efficient, event-driven system. It covers how to get started with designing your microservice architecture and the key requirements any system needs to fulfil. It also introduces the different patterns you will encounter in event-driven architectures and the advantages and disadvantages of these choices. Finally it explains why Apache Kafka is a great choice for event-driven microservices.
Explore the challenges of migrating existing applications to a cloud infrastructure, then present proven strategies for mitigating the risks You'll learn how to:
- Prioritize application migration
- Plan for “big-blocks”
- Assess your existing applications' ‘fit’ for cloud
- Leverage a centralized development testing platform to align your business goals with coding decisions
- Create a cloud migration policy
- Use process to mitigate the risks associated with cloud migration
Agile 2013 - Lean Change for Enabling Agile TransformationsAlexis Hui
Experience report summarizing our experiences with agile transformation in mid-large sized IT organizations and challenges we faced with current methods available. As a result, we saw a need for a better change approach to help us and others in the agile community with agile transformations. Borrowing thinking and tools from Lean Startup, Kanban and Kotter we have defined a structured framework known as Lean Change. The premise behind our thinking is that successful agile transformation requires learning and feedback as the keys for success. Lean Change is founded on three concepts, co-creation of changes through negotiated change, experiment based objectives using minimum viable changes, and validated learning to guide changes through a structured validation lifecycle.
Composable Software Architecture with SpringSam Brannen
What does the architecture of a modern enterprise Java application look like? What have we as a community learned from our past? What does it mean to design a composable architecture? And how can Spring help developers meet the needs of enterprise applications in 2013 and beyond?
In this keynote presentation at Java Breeze, core Spring Framework committer Sam Brannen invites the audience to explore what it means to design, develop, and test modern enterprise Java applications following a composable software architecture model. Along the way, Sam will show how the Spring ecosystem and programming model fit into the larger picture of modern enterprise Java applications.
Lessons learned processing 70 billion data points a day using the hybrid cloudDataWorks Summit
NetApp receives 70 billion data points of telemetry information each day from its customer’s storage systems. This telemetry data contains configuration information, performance counters, and logs. All of this data is processed using multiple Hadoop clusters, and feeds a machine learning pipeline and a data serving infrastructure that produces insights for customers via an application called Active IQ. We describe the evolution of our Hadoop infrastructure from a traditional on-premises architecture to the hybrid cloud, and lessons learned.
We’ll discuss the insights we are able to produce for our customers, and the techniques used. Finally, we describe the data management challenges with our multi-petabyte Hadoop data lake. We solved these problems by building a unified data lake on-premises and using the NetApp Data Fabric to seamlessly connect to public clouds for data science and machine learning compute resources.
Architecting a truly hybrid cloud implementation allowed NetApp to free up our data scientists to use any software on any cloud, kept the customer log data safe on NetApp Private Storage in Equinix, resulted in faster ability to innovate and release new code and provided flexibility to use any public cloud at the same time with data on NetApp in Equinix.
Speaker
Pranoop Erasani, NetApp, Senior Technical Director, ONTAP
Shankar Pasupathy, NetApp, Technical Director, ACE Engineering
This is a talk I gave at PLoP 2017 - http://www.hillside.net/plop/2017/index.php?nav=program
The microservice architecture is growing in popularity. It is an architectural style that structures an application as a set of loosely coupled services that are organized around business capabilities. Its goal is to enable the continuous delivery of large, complex applications. However, the microservice architecture is not a silver bullet and it has some significant drawbacks.
The goal of the microservices pattern language is to enable software developers to apply the microservice architecture effectively. It is a collection of patterns that solve architecture, design, development and operational problems. In this talk, I’ll provide an overview of the microservice architecture and describe the motivations for the pattern language. You will learn about the key patterns in the pattern language.
Monitoring at scale - Intuitive dashboard designLorenzo Alberton
At a certain scale, millions of events happen every second, and all of them are important to evaluate the health of the system. If not handled correctly, such a volume of information can overwhelm both the infrastructure that needs to support them, and people who have to make a sense out of thousands of signals and make decisions upon them, fast. By understanding how our rational mind works, how people process information, we can present data so it's more evident and intuitive. This talk will explain how to collect useful metrics, and to create the perfect monitoring dashboard to organise and display them, letting our intuition operate automatically and quickly, and saving attention and mental effort to activities that demand it.
Modern Algorithms and Data Structures - 1. Bloom Filters, Merkle TreesLorenzo Alberton
The first part of a series of talks about modern algorithms and data structures, used by nosql databases like HBase and Cassandra. An explanation of Bloom Filters and several derivates, and Merkle Trees.
Graphs in the Database: Rdbms In The Social Networks AgeLorenzo Alberton
Despite the NoSQL movement trying to flag traditional databases as a dying breed, the RDBMS keeps evolving and adding new powerful weapons to its arsenal. In this talk we'll explore Common Table Expressions (SQL-99) and how SQL handles recursion, breaking the bi-dimensional barriers and paving the way to more complex data structures like trees and graphs, and how we can replicate features from social networks and recommendation systems. We'll also have a look at window functions (SQL:2003) and the advanced reporting features they make finally possible.
NoSQL databases get a lot of press coverage, but there seems to be a lot of confusion surrounding them, as in which situations they work better than a Relational Database, and how to choose one over another. This talk will give an overview of the NoSQL landscape and a classification for the different architectural categories, clarifying the base concepts and the terminology, and will provide a comparison of the features, the strengths and the drawbacks of the most popular projects (CouchDB, MongoDB, Riak, Redis, Membase, Neo4j, Cassandra, HBase, Hypertable).
Trees In The Database - Advanced data structuresLorenzo Alberton
Storing tree structures in a bi-dimensional table has always been problematic. The simplest tree models are usually quite inefficient, while more complex ones aren't necessarily better. In this talk I briefly go through the most used models (adjacency list, materialized path, nested sets) and introduce some more advanced ones belonging to the nested intervals family (Farey algorithm, Continued Fractions, and other encodings). I describe the advantages and pitfalls of each model, some proprietary solutions (e.g. Oracle's CONNECT BY) and one of the SQL Standard's upcoming features, Common Table Expressions.
Overview of agile values
This presentation shows some core concepts that make agile software development different.
This will help your team familiar with agile concepts and start boosting your team performance.
Introduction to Scrum presentation which outlines common issues in software development, what is Scrum, and an introduction to the Scrum framework. This presentation has been used for training and presentations to both technology and business audiences.
This presentation was used in "Agile workshop for FPT Aptech Hanoi students" in December 2012.
This doc covers most of core practices of an agile developer.
This is my latest presentation on "Scrum managing through complexity" given at Luxembourg Sacred Heart University Executive MBA Class (Jan. 17th 2012).
This is a part of the Operational Excellence Module.
The Essential Product Owner - Partnering with the teamCprime
Bob Galen shares real-world stories where he’s seen “effectively partnered” teams and Product Owners truly deliver balanced value for their business stakeholders. In this session he’ll show you how story mapping and release planning can truly set the stage for effective team workflow—establishing a “Big Picture” for everyone to shoot for. How establishing shared goals, both at the iteration and release levels, truly cements the partnership between team and Product Owner. And finally, how setting a tempo of regular, focused backlog grooming sessions establishes a mechanism for the team and Product Owner to explore well-nuanced and high value backlogs.
Consistently delivering and maintaining well performing applications doesn't just happen, it requires a solid architecture, sound development, continual attention, diligence and expertise. It also requires appropriate testing, not simply of release-candidate builds, but of designs, units, integrations, and physical components... both during development and in production. The question is, how can a team accomplish all of that under all of today's pressure to deliver quickly and cheaply?
Join Scott Barber for this Keynote Address to hear about what successful organizations are doing to consistently deliver well performing applications, to learn the underlying principles and practices that enable those organizations to create, test, and maintain those well performing applications without breaking either the budget or the schedule, and what the key items are that virtually every team can implement right away, to dramatically improve the consistency and overall performance of their applications.
Randy Clark - a Black Belt and PowerSteering Software's Director of Six Sigma - shares best practice advice for how Process Excellence leaders can guarantee repeated victory with project replication, the process of taking successful projects and duplicating the results in other locations or business areas. View the slides to learn valuable tips to help your organization repeat the projects proven to yield financial touchdowns, significantly increase project completion percentages, and eliminate time consuming and costly fumbles.
Similar to Scaling Teams, Processes and Architectures (20)
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Monitoring Java Application Security with JDK Tools and JFR Events
Scaling Teams, Processes and Architectures
1. Lorenzo Alberton
@lorenzoalberton
Scaling Teams,
Processes and
Architectures
Managing growth
London Scalability Group,
Innovation Warehouse, 16th April 2012
1
4. Staffing
Never compromise.
Only hire people smarter than you.
http://www.earthrangers.com/content/wildwire/toxic_spill.jpg 4
5. Staffing
Hire people who can fit
the company culture.
Promote fun in your
working environment.
http://www.earthrangers.com/content/wildwire/toxic_spill.jpg 4
6. Staffing
Beware of
toxic people
http://www.earthrangers.com/content/wildwire/toxic_spill.jpg 4
7. Team Size and Structure
Micromanaging managers Poor communication
too small Overworked team members Low morale too big
Can’t accomplish much Low productivity
5
8. Team Size and Structure
Micromanaging managers Poor communication
too small Overworked team members Low morale too big
Can’t accomplish much Low productivity
CTO
functional
PM PM PM
Designer Developer Tester
Designer Developer Tester
Designer Developer Tester
Designer Developer Tester
Designers Developers Testers
5
9. Team Size and Structure
Micromanaging managers Poor communication
too small Overworked team members Low morale too big
Can’t accomplish much Low productivity
CTO
functional
matrix
PM PM PM
Proj 1 PM Designer Developer Tester
Proj 2 PM Designer Developer Tester
Proj 3 PM Designer Developer Tester
Proj 4 PM Designer Developer Tester
Designers Developers Testers
5
11. Why are processes critical?
Improve management of teams and employees
Standardise actions in repetitive tasks
Reduce mundane decisions to focus on grander ideas
Allow the team to react quickly to crisis
Determine system capacity and scalability needs
7
12. Why are processes critical?
Improve management of teams and employees
Standardise actions in repetitive tasks
Reduce mundane decisions to focus on grander ideas
Allow the team to react quickly to crisis
Determine system capacity and scalability needs
Challenge
7
13. Why are processes critical?
Improve management of teams and employees
Standardise actions in repetitive tasks
Reduce mundane decisions to focus on grander ideas
Allow the team to react quickly to crisis
Determine system capacity and scalability needs
Challenge
right amount
7
14. Why are processes critical?
Improve management of teams and employees
Standardise actions in repetitive tasks
Reduce mundane decisions to focus on grander ideas
Allow the team to react quickly to crisis
Determine system capacity and scalability needs
Challenge
right amount right process
7
15. Why are processes critical?
Improve management of teams and employees
Standardise actions in repetitive tasks
Reduce mundane decisions to focus on grander ideas
Allow the team to react quickly to crisis
Determine system capacity and scalability needs
Challenge
right amount right process right time
7
20. Risk Management
Risk is cumulative
Determine limits and tolerance
10
21. Load / Stress Testing
Load testing
- identify, document and eliminate
bottlenecks through a strict controlled
process of measurement and analysis
- measure system’s response and stability
- verify the app can meet the desired
performance objectives (SLA)
Stress testing
- determine the app’s stability when
subjected to above-normal loads
- verify the app’s behaviour when close
to the breaking point
- test the application recoverability
(negative testing)
11
22. Barrier Conditions
Code reviews
Manual and automated QA processes
Performance and stress testing
Release documentation checks (runbook)
Dev, Test, Stage and Live environments
Instrumentation checks
Protection from significant failures
12
29. Architectural Principles
+1
N + 1 design for rollback to be disabled
to be for multiple
monitored live sites
14
30. Architectural Principles
+1
N + 1 design for rollback to be disabled
to be for multiple use mature
monitored live sites technology
14
31. Architectural Principles
+1
N + 1 design for rollback to be disabled
to be for multiple use mature
monitored live sites technology
asynchronous
design
14
32. Architectural Principles
+1
N + 1 design for rollback to be disabled
to be for multiple use mature
monitored live sites technology
asynchronous stateless
design systems
14
33. Architectural Principles
+1
N + 1 design for rollback to be disabled
to be for multiple use mature
monitored live sites technology
asynchronous stateless buy when
design systems non core
14
34. Stateless, Asynchronous Systems
http://upload.wikimedia.org/wikipedia/commons/4/46/Synchronized_swimming_-_Russian_team.jpg 15
38. Fault Isolative Structures
Increase availability
Limit impact of
failures
Easier debugging
Functions
causing
repetitive
problems
First
16
39. Fault Isolative Structures
Increase availability
Limit impact of
failures
Easier debugging
Functions Natural layout
causing or topology
repetitive of the site
problems
First
16
44. Managing “Big Data”
storage costs
people and software
power and space
processing power
backup time and costs
18
45. Managing “Big Data”
The more storage
...the more
storage management
storage costs
people and software
power and space
processing power
backup time and costs
18
46. Managing “Big Data”
The more storage
...the more
storage management
storage costs
people and software
power and space
processing power
backup time and costs
Evaluate data retention policy
Consider multi-tiered storage
Distribute data/ work (Hadoop, M/R)
18
48. Monitoring: Measure Everything
1. Is there a problem? User experience / Business metrics monitors
2. Where is the problem? System monitors (threshold - variance)
3. What is the problem? Application monitors
19
49. Monitoring: Measure Everything
1. Is there a problem? User experience / Business metrics monitors
2. Where is the problem? System monitors (threshold - variance)
3. What is the problem? Application monitors
Keep Signal vs. Noise ratio high
19
50. Monitoring: Measure Everything
StatsD
1. Is there a problem? User experience / Business metrics monitors
2. Where is the problem? System monitors (threshold - variance)
3. What is the problem? Application monitors
Keep Signal vs. Noise ratio high
19
61. 0mq PUB-SUB (High Availability)
DC 1
Publisher 1
Publisher 2
DC 2
27
62. Internal “Firehose”
Publishers Subscribers
Alice’s John’s
Y Z timeline Inbox
X
subscribe
to topic X
Data Bus
subscribe
to topic Y
System Fred’s Tech
Monitor Followers Blog Feed
28
65. References
M. L. Abbot, M. T. Fisher,
“The Art Of Scalability”,
Addison Wesley
http://theartofscalability.com/
http://www.slideshare.net/quipo/the-art-of-scalability-managing-
growth
http://www.slideshare.net/postwait/scalable-internet-architecture
http://bit.ly/IJKwuc
http://agile.dzone.com/news/approaches-organizational
https://bitly.com/vCSd49
31
66. Lorenzo Alberton
@lorenzoalberton
Thank you!
lorenzo@alberton.info
http://www.alberton.info/talks
Questions?
32
Editor's Notes
\n
Let’s start by focusing on the true foundation: people and process, without which true scalability cannot be built.\nPeople are the most important element of scalability, as without people there are no processes and no technology.\n
\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Leadership is the influencing of an organisation to accomplish a specific objective (down to personal characteristics + skills + experience + actions).\nLook for solid software engineers with good understanding of CS topics, and exceptional devops. Create fun working environment. We solve serious, challenging problems. We also want to have fun. Avoid rockstars. "Hard work beats talent when talent doesn't work hard." - Tim Notke\nFocus and dedication.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
Overlapping responsibilities create wasted effort and value-destroying conflicts.\nKey scale-related responsibilities for any organisation include:\n- setting measurable goals; - staffing the team with the appropriate skills; - defining and implementing a scalable architecture.\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
There’s a link between organisational structure and scalability, it has a big impact in personal productivity.\nThe goal is to minimise the friction caused by organisational or team boundaries, without limiting the throughput, and at the same time making innovation and the work flow easy.\nThe team can be organised into 2 structures:- functional (employees divided by their primary function; homogeneity, simplicity of responsibilities, adherence to standards; Drawbacks: no single project owner, poor cross-functional communication).\n- matrix (similar, but with a second dimension that includes a new management structure; better communication, project ownership; Drawbacks: multiple bosses, distraction from a person’s primary discipline).\n
\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
Processes serve 3 general purposes:\n- they augment the management of our teams and employees\n- they standardise employee’s actions while performing repetitive tasks\n- they free employees up from daily mundane decisions to concentrate on grander ideas.\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
In order to navigating your way out of the woods, you need to know the point from which you are starting.\nHeadroom = amount of free capacity that exists within your system before you start having problems such as degradation of performance or an outage. Because your app is a system that involves many different components as a db, a firewall, application servers, in order to truly understand headroom you need to first understand the headroom of each of these.\n1) Identify major components. 2) Identify responsible team. 3) Determine usage and capacity. 4) Determine growth rate. Work together for a better analysis and to find the best solution\n
The intent of change management is to limit the impact of changes by controlling them through their release into the production environment and logging them as they are introduced to production. Gut feeling / finger in the air: thanks to experience + innate ability: fast, but not accurate.\nSemaphore: Assign a risk level of green/yellow/red to each small component, then assign an overall colour: methodical, repeatable, documentable, no longer relying on a single person, accurate. Better: Failure Mode and Effect Analysis.\n\n\n
Risk is cumulative. You might want to establish some limits to the amount of risk that you are willing to allow at a particular time of the day or customer volume.\nAlso consider the human factor, i.e. the level of risk tolerance that a person can have within a certain time frame.\n\n
The purpose of load testing is to identify, document and eliminate bottlenecks in the system through a strict controlled process of measurement and analysis. Load testing is the process of putting load or user demand on a system to measure its response and stability, to verify that the app can meet the desired performance objectives (SLA: service level agreement).\n Establish success criteria (concurrent usage, response time, ...)\n Establish the test environment (as close as possible to the production environment)\n Define the tests (Pareto rule 20% - 80%) to cover different things (endurance, most used, most visible, different components)\n Identify what needs to be monitored / what data needs to be collected\n Run, Analyse, Report to Engineers\n Repeat Tests and Analysis\nStress testing is a process used to determine an application’s stability when subjected to above-normal loads, to verify the behaviour when close to the breaking point of the application.\nPositive testing is where the load is progressively increased to overwhelm the system’s resources.\nNegative testing takes away resources such as memory, threads, connections, testing the application recoverability.\nThe way to insure that the headroom calculations remain accurate is to conduct performance testing on all your releases to insure you are not introducing unexpected load increases.\n
Good processes for the promotion of systems into the production environment have the capability of protecting you from significant failures. Developing effective barrier conditions and coupling them with a process and capability to roll back production changes are necessary components within any highly available service and are critical to the success of your scalability goals.\n
\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
- N+1 design (ensure that everything you develop has at least one additional instance of that system in the event of failure)\n- Designing the capability to roll back into an app helps limit the scalability impact of any given release.\n- Designing to disable features adds the flexibility of keeping the most recent release in production while limiting / containing the impact of offending features or functionality.\n- Design to be monitored: you want your system to identify when it’s performing differently than it normally operates in addition to telling you when it’s not functioning properly.\n- Design for multiple live sites: it usually costs less than the operation of a hot site and a cold disaster recovery site.\n- Use mature technology: early adopters risk a lot in finding the bugs; availability and reliability are important.\n- Asynchronous design: asynchronous systems tend to be more fault tolerant to extreme load.\n- Stateless Systems (if necessary, store state with the end users)\n- Buy when non-core\n- Scale out not up (with commodity hardware; horizontal split in terms of data, transactions and customers).\n- Design for any technology, not for a specific product/vendor\n
Synchronous calls, if used excessively or incorrectly cause undue burden on the system and prevent it from scaling.\nSystems designed to interact synchronously have a higher failure rate than asynchronous ones. Their ability to scale is tied to the slowest system in the chain of communications. It’s better to use callbacks, and timeouts to recover gracefully should they not receive responses in a timely fashion.\nSynchronisation is when two or more pieces of work must be in a specific order to accomplish a task. Asynchronous coordination between the original method and the invoked method requires a mechanism that the original method determines when or if a called method has completed executing (callbacks). Ensure they have a chance to recover gracefully with timeouts should they not receive responses in a timely fashion.\nA related problem is stateful versus stateless applications. An application that uses state relies on the current condition of execution as a determinant of the next action to be performed. \nThere are 3 basic approaches to solving the complexities of scaling an application that uses session data: 1) Avoidance (using no sessions or sticky sessions) avoid replication: Share-nothing architecture; 2) Decentralisation (store session data in the browser’s cookie or in a db whose key is referenced by a hash in the cookie); 3) Centralisation (store cookies in the db / memcached).\n\n
You must be able to isolate and limit the effects of failures within any system, by segmenting the components. Decouple decouple decouple! A swim lane represent both a barrier and a guide (ensure that swimmers don’t interfere with each other. Help guide the swimmer toward their objective with minimal effort). AKA Shard.\nThey increase availability by limiting the impact of failures to a subset of functionality, make incidents easier to detect, identify and resolve. The fewer the things are shared between lanes, the more isolative and beneficial the swim lane becomes to both scalability and availability. They should not have lines of communication crossing lane boundaries, and should always move in the direction of the communication. When designing swim lanes, always address the transactions making the company money first (e.g. Search&Browse vs Shopping Cart), then move functions causing repetitive problems into swim lanes; finally consider the natural layout or topology of the site for opportunities to swim lanes (e.g. customer boundaries within an app / environment. If you have a tenant who is very busy, assign it a swim lane; other tenants with a low utilisation can be all put into another swim lane).\n
You must be able to isolate and limit the effects of failures within any system, by segmenting the components. Decouple decouple decouple! A swim lane represent both a barrier and a guide (ensure that swimmers don’t interfere with each other. Help guide the swimmer toward their objective with minimal effort). AKA Shard.\nThey increase availability by limiting the impact of failures to a subset of functionality, make incidents easier to detect, identify and resolve. The fewer the things are shared between lanes, the more isolative and beneficial the swim lane becomes to both scalability and availability. They should not have lines of communication crossing lane boundaries, and should always move in the direction of the communication. When designing swim lanes, always address the transactions making the company money first (e.g. Search&Browse vs Shopping Cart), then move functions causing repetitive problems into swim lanes; finally consider the natural layout or topology of the site for opportunities to swim lanes (e.g. customer boundaries within an app / environment. If you have a tenant who is very busy, assign it a swim lane; other tenants with a low utilisation can be all put into another swim lane).\n
You must be able to isolate and limit the effects of failures within any system, by segmenting the components. Decouple decouple decouple! A swim lane represent both a barrier and a guide (ensure that swimmers don’t interfere with each other. Help guide the swimmer toward their objective with minimal effort). AKA Shard.\nThey increase availability by limiting the impact of failures to a subset of functionality, make incidents easier to detect, identify and resolve. The fewer the things are shared between lanes, the more isolative and beneficial the swim lane becomes to both scalability and availability. They should not have lines of communication crossing lane boundaries, and should always move in the direction of the communication. When designing swim lanes, always address the transactions making the company money first (e.g. Search&Browse vs Shopping Cart), then move functions causing repetitive problems into swim lanes; finally consider the natural layout or topology of the site for opportunities to swim lanes (e.g. customer boundaries within an app / environment. If you have a tenant who is very busy, assign it a swim lane; other tenants with a low utilisation can be all put into another swim lane).\n
You must be able to isolate and limit the effects of failures within any system, by segmenting the components. Decouple decouple decouple! A swim lane represent both a barrier and a guide (ensure that swimmers don’t interfere with each other. Help guide the swimmer toward their objective with minimal effort). AKA Shard.\nThey increase availability by limiting the impact of failures to a subset of functionality, make incidents easier to detect, identify and resolve. The fewer the things are shared between lanes, the more isolative and beneficial the swim lane becomes to both scalability and availability. They should not have lines of communication crossing lane boundaries, and should always move in the direction of the communication. When designing swim lanes, always address the transactions making the company money first (e.g. Search&Browse vs Shopping Cart), then move functions causing repetitive problems into swim lanes; finally consider the natural layout or topology of the site for opportunities to swim lanes (e.g. customer boundaries within an app / environment. If you have a tenant who is very busy, assign it a swim lane; other tenants with a low utilisation can be all put into another swim lane).\n
You must be able to isolate and limit the effects of failures within any system, by segmenting the components. Decouple decouple decouple! A swim lane represent both a barrier and a guide (ensure that swimmers don’t interfere with each other. Help guide the swimmer toward their objective with minimal effort). AKA Shard.\nThey increase availability by limiting the impact of failures to a subset of functionality, make incidents easier to detect, identify and resolve. The fewer the things are shared between lanes, the more isolative and beneficial the swim lane becomes to both scalability and availability. They should not have lines of communication crossing lane boundaries, and should always move in the direction of the communication. When designing swim lanes, always address the transactions making the company money first (e.g. Search&Browse vs Shopping Cart), then move functions causing repetitive problems into swim lanes; finally consider the natural layout or topology of the site for opportunities to swim lanes (e.g. customer boundaries within an app / environment. If you have a tenant who is very busy, assign it a swim lane; other tenants with a low utilisation can be all put into another swim lane).\n
What is the best way to handle large volumes of traffic? Answer: “Establish the right organisation, implement the right processes and follow the right architectural principles”. Correct, but the best way is not to have to handle it at all. The key to achieving this is through pervasive use of caching. The cache hit ratio is important to understand its effectiveness. The cache can be updated/refreshed via a batch job or on a cache-miss. If the cache is filled, some algorithms (LRU, MRU...) will decide on which entry to evict. When the data changes, the cache can be updated through a write-back or write-through policy. There are 3 cache types:\n- Object caches: used to store objects for the app to be reused, usually serialized objects. The app must be aware of them. Layer in front of the db / external services. Marshalling is a process where the object is transformed into a data format suitable for transmitting or storing.\n- Application caches: A) Proxy caches, usually implemented by ISPs, universities or corporations; it caches for a limited number of users and for an unlimited number of sites. B) Reverse proxy caches (opposite): it caches for an unlimited number of users and for a limited number of applications; the configuration of the specific app will determine what can be cached. HTTP headers give much control over caching (Last-Modified, Etag, Cache-Control).\n- Content Delivery Networks: they speed up response time, off load requests from your application’s origin server, and usually lower costs. The total capacity of the CDN’s strategically placed servers can yield a higher capacity and availability than the network backbone. The way it works is that you place the CDN’s domain name as an alias for your server by using a canonical name (CNAME) in your DNS entry\n
What is the best way to handle large volumes of traffic? Answer: “Establish the right organisation, implement the right processes and follow the right architectural principles”. Correct, but the best way is not to have to handle it at all. The key to achieving this is through pervasive use of caching. The cache hit ratio is important to understand its effectiveness. The cache can be updated/refreshed via a batch job or on a cache-miss. If the cache is filled, some algorithms (LRU, MRU...) will decide on which entry to evict. When the data changes, the cache can be updated through a write-back or write-through policy. There are 3 cache types:\n- Object caches: used to store objects for the app to be reused, usually serialized objects. The app must be aware of them. Layer in front of the db / external services. Marshalling is a process where the object is transformed into a data format suitable for transmitting or storing.\n- Application caches: A) Proxy caches, usually implemented by ISPs, universities or corporations; it caches for a limited number of users and for an unlimited number of sites. B) Reverse proxy caches (opposite): it caches for an unlimited number of users and for a limited number of applications; the configuration of the specific app will determine what can be cached. HTTP headers give much control over caching (Last-Modified, Etag, Cache-Control).\n- Content Delivery Networks: they speed up response time, off load requests from your application’s origin server, and usually lower costs. The total capacity of the CDN’s strategically placed servers can yield a higher capacity and availability than the network backbone. The way it works is that you place the CDN’s domain name as an alias for your server by using a canonical name (CNAME) in your DNS entry\n
What is the best way to handle large volumes of traffic? Answer: “Establish the right organisation, implement the right processes and follow the right architectural principles”. Correct, but the best way is not to have to handle it at all. The key to achieving this is through pervasive use of caching. The cache hit ratio is important to understand its effectiveness. The cache can be updated/refreshed via a batch job or on a cache-miss. If the cache is filled, some algorithms (LRU, MRU...) will decide on which entry to evict. When the data changes, the cache can be updated through a write-back or write-through policy. There are 3 cache types:\n- Object caches: used to store objects for the app to be reused, usually serialized objects. The app must be aware of them. Layer in front of the db / external services. Marshalling is a process where the object is transformed into a data format suitable for transmitting or storing.\n- Application caches: A) Proxy caches, usually implemented by ISPs, universities or corporations; it caches for a limited number of users and for an unlimited number of sites. B) Reverse proxy caches (opposite): it caches for an unlimited number of users and for a limited number of applications; the configuration of the specific app will determine what can be cached. HTTP headers give much control over caching (Last-Modified, Etag, Cache-Control).\n- Content Delivery Networks: they speed up response time, off load requests from your application’s origin server, and usually lower costs. The total capacity of the CDN’s strategically placed servers can yield a higher capacity and availability than the network backbone. The way it works is that you place the CDN’s domain name as an alias for your server by using a canonical name (CNAME) in your DNS entry\n
\n
\n
\n
\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
Logging at scale is useless. Too much noise. Instrumentation is essential.\nYou need to identify bottlenecks quickly or suffer prolonged and painful outages. The question of "How come we didn't catch that earlier?" addresses the incident, not the problem. The alternative question "What in our process is flawed that allowed us to launch the service without the appropriate monitoring to catch such an issue?" addresses the people and the processes that allowed the event you just had and every other event for which you didn't have appropriate monitoring.\nDesigning to be monitored is an approach wherein one builds monitoring into the application rather than around it. "How do we know when it's starting to behave poorly?" First, you need to answer the question "Is there a problem?" with user experience and business metrics monitors (lower click-through rate, shopping cart abandonment rate, ...). Then you need to identify where the problem is with system monitors (the problem with this is that it's usually relying on threshold alerts - i.e. checking if something is behaving outside of our expectations - rather than alerting on when it's performing significantly differently than in the past). Finally you need to identify what is the problem thanks to application monitoring. \nNot all monitoring data is valuable, too much of it only creates noise, while wasting time and resources. It's advisable to only save a summary of the reports over time to keep costs down while still providing value. In the ideal world, incidents and crises are predicted and avoided by a robust monitoring solution.\n
\n
\n
\n
\n
Use queues and workers to make processes asynchronous, distribute data to parallel workers. \n
happy to talk about any of them\n
\n
\n
\n
listeners can only subscribe to one or more topics. Different output channels.\nZeroMQ v3: filtering done on the publisher side\n
An interesting idea if you have a highly dynamic site / service, with each update affecting several other users / pages, is to have an internal data bus that carries all the information, with updates labelled with topics, and all the services/users subscribing to the relevant topics.\n
We collect millions of events every second.\nThe importance of people: devops who know what to monitor, how, how to use and write tools, and have 100% dedication.\nWe use different technologies. It’s very easy to set up a new ZeroMQ listener.\nWe use StatsD (from Flickr / Etsy), Zenoss, Graphite\n