Software risk impact is more predictable than you might think. This session discusses similarities of uncertainty in various industries and relates this back to how we can measure and analyze impediments and risk for agile software teams.
Forecasting using data workshop slides for the Deliver conference in Winnipeg October 2016. This session introduces practical exercises for probabilistic forecasting. http://www.prdcdeliver.com
I love the smell of data in the morning (getting started with data science) ...Troy Magennis
Data Science 101 for software development. I know it misses the purist view of Data Science, but this is intended to get you started! First presented at Agile 2017 in Florida.
What is the story with agile data keynote agile 2018 (Magennis)Troy Magennis
This document discusses using data to improve agile practices and outcomes. It argues that agile has lost the "data war" by not capturing and utilizing data from teams effectively. It suggests that data needs to be handled safely to avoid embarrassing people and destroying the utility of historical data. Better ways are needed to measure outcomes rather than just output, and to balance predictability with creativity. The document also discusses visualizing and managing dependencies, comparing performance across teams, and using the right metrics depending on a team's characteristics and challenges. The overarching message is that data needs to be used carefully and conversationally to drive the right actions and improve agile practices.
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...Troy Magennis
To meet expectations and optimize flow, managing risk is an important part of Kanban. Anticipating and adapting to things that "go wrong" and the uncertainty they cause is topic of this session. We look at techniques for quantifying what risks should be considered important to deal with.
Although discouraging, forecasting size, effort, staff and cost is sometimes necessary. Of course we have to do as little of this as possible, but when we do, we have to do it well with the data we have available. Forecasting is made difficult by un-reliable information as inputs to our process – the amount of work is uncertain, the historical data we are basing our forecasts on is biased and tainted, the situation seems hopeless. But it isn't. Good decisions can be made on imperfect data, and this session discusses how. This session shows immediately usable and simple techniques to capture, analyze, cleanse and assess data, and then use that data for reliable forecasting.
Second and hopefully draft of LKCE 2014 talk.
The document discusses using metrics to improve decision making for software projects, explaining that metrics should be focused on outcomes that teams can influence and that predict future performance, and provides examples of different types of metrics and modeling techniques that can help teams forecast delivery and make better decisions.
How to use data to improve software development teams and processes. Presented at the Prairie Dev Con Deliver conference October 2016. http://www.prdcdeliver.com
Data driven coaching - Agile 2016 (troy magennis)Troy Magennis
Team data and dashboards can be misused and cause more pain than results. Having the team run blind to its historical data though is often worse, with solely opinions and gut-feel driving process change. Helping your teams see and understand a holistic balance of their data will give your coaching advice context and encourage team constant improvement through experiments and reflection.
Coaching dashboards are about balancing trade-offs. Trading something your team is great at for something they want (or need) to improve. Having the team complete the feedback loop and confirm than an experiment had the intended impact, will process improvement be continuous and sustainable.
This presentation shows how to expose data to teams in order for them to retrospect productively, determine if a process experiment is panning out as expected, and to vigorously explore process change opportunities. Recent research shows strong relationships of certain metrics to process and practices, and this session demonstrates how these metrics have and can be tied to timely coaching advice.
The real-world dashboards demonstrated in this session show most common problems and how to avoid them with before and after shots and quotes from the teams impacted by them.
In this session you will –
- Learn how you can not only gather data, but use it to improve the process, with examples!
- Learn how your can tie data insights to coaching advice (data driven coaching)
- Learn how you can detect, predict and avoid data gaming and dashboard misuse
- Learn from my mistakes, and mistakes I’ve seen others with real examples of Agile coaching dashboards (good and bad)
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
Forecasting using data workshop slides for the Deliver conference in Winnipeg October 2016. This session introduces practical exercises for probabilistic forecasting. http://www.prdcdeliver.com
I love the smell of data in the morning (getting started with data science) ...Troy Magennis
Data Science 101 for software development. I know it misses the purist view of Data Science, but this is intended to get you started! First presented at Agile 2017 in Florida.
What is the story with agile data keynote agile 2018 (Magennis)Troy Magennis
This document discusses using data to improve agile practices and outcomes. It argues that agile has lost the "data war" by not capturing and utilizing data from teams effectively. It suggests that data needs to be handled safely to avoid embarrassing people and destroying the utility of historical data. Better ways are needed to measure outcomes rather than just output, and to balance predictability with creativity. The document also discusses visualizing and managing dependencies, comparing performance across teams, and using the right metrics depending on a team's characteristics and challenges. The overarching message is that data needs to be used carefully and conversationally to drive the right actions and improve agile practices.
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...Troy Magennis
To meet expectations and optimize flow, managing risk is an important part of Kanban. Anticipating and adapting to things that "go wrong" and the uncertainty they cause is topic of this session. We look at techniques for quantifying what risks should be considered important to deal with.
Although discouraging, forecasting size, effort, staff and cost is sometimes necessary. Of course we have to do as little of this as possible, but when we do, we have to do it well with the data we have available. Forecasting is made difficult by un-reliable information as inputs to our process – the amount of work is uncertain, the historical data we are basing our forecasts on is biased and tainted, the situation seems hopeless. But it isn't. Good decisions can be made on imperfect data, and this session discusses how. This session shows immediately usable and simple techniques to capture, analyze, cleanse and assess data, and then use that data for reliable forecasting.
Second and hopefully draft of LKCE 2014 talk.
The document discusses using metrics to improve decision making for software projects, explaining that metrics should be focused on outcomes that teams can influence and that predict future performance, and provides examples of different types of metrics and modeling techniques that can help teams forecast delivery and make better decisions.
How to use data to improve software development teams and processes. Presented at the Prairie Dev Con Deliver conference October 2016. http://www.prdcdeliver.com
Data driven coaching - Agile 2016 (troy magennis)Troy Magennis
Team data and dashboards can be misused and cause more pain than results. Having the team run blind to its historical data though is often worse, with solely opinions and gut-feel driving process change. Helping your teams see and understand a holistic balance of their data will give your coaching advice context and encourage team constant improvement through experiments and reflection.
Coaching dashboards are about balancing trade-offs. Trading something your team is great at for something they want (or need) to improve. Having the team complete the feedback loop and confirm than an experiment had the intended impact, will process improvement be continuous and sustainable.
This presentation shows how to expose data to teams in order for them to retrospect productively, determine if a process experiment is panning out as expected, and to vigorously explore process change opportunities. Recent research shows strong relationships of certain metrics to process and practices, and this session demonstrates how these metrics have and can be tied to timely coaching advice.
The real-world dashboards demonstrated in this session show most common problems and how to avoid them with before and after shots and quotes from the teams impacted by them.
In this session you will –
- Learn how you can not only gather data, but use it to improve the process, with examples!
- Learn how your can tie data insights to coaching advice (data driven coaching)
- Learn how you can detect, predict and avoid data gaming and dashboard misuse
- Learn from my mistakes, and mistakes I’ve seen others with real examples of Agile coaching dashboards (good and bad)
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
This document discusses forecasting software delivery using probabilistic methods rather than estimations. It defines key terms like prediction, estimation, forecast, and Monte Carlo simulation. The problems with traditional estimations are outlined, such as being inaccurate and not accounting for variability. Forecasting is presented as a better alternative that uses historical data to generate probability distributions of possible outcomes. Examples are given of forecasting delivery times for single issues, multiple issues with different resources, and rolling forecasts as new data comes in. The benefits of forecasting are noted as being more factual, repeatable, and reducing stress compared to estimations. Implications are that issue tracking and development practices need to support forecasting to improve predictability.
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...CXL
Individual tests drive insights & ROI, but the most sophisticated optimizers look beyond what an individual test is telling them and use data to optimize their overall testing performance.
In this talk, Claire will dive into the specifics of how to track, improve, and drive insight from performance metrics for your conversion program, so you can not only run better tests, but get more out of your investment in CRO.
Statistics for UX Professionals - Jessica CameronUser Vision
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
State of the Art in Machine Learning, by Thomas Dietterich, Distinguished Professor Emeritus in the School of EECS at Oregon State University and Chief Scientist of BigML.
*MLSEV 2020: Virtual Conference.
CYCLE TIME ANALYTICS: RELIABLE #NOESTIMATES FORECASTING USING DATA, TROY MAGE...Lean Kanban Central Europe
If you are struggling to forecast project delivery dates and cost, or you want to eliminate the story estimation process because you feel it is waste, or you need to build the business case for hiring more staff, then this session is relevant to you. All estimates have uncertainty, and understanding how multiple uncertain factors compound is the first step to improving project and team predictability. A major benefit of Lean is the low weight capture of cycle time metrics. This session looks at how to use historical cycle time data to answer questions of forecasting and staff skill balancing. This session compares the benefits of using cycle time for analysis over current planning techniques such as velocity, burn-down charts, and cumulative flow diagrams. This session takes you on a journey of what to do after capturing cycle time data or what to do if you have no history to rely upon. Reducing reliance on developer estimation (popularized by the twitter hashtag of #NoEstimates movement) is good general advice, having the tools to plan and manage teams and projects is still important to maintain support at the executive level. This session details the approaches to getting the numbers you need to have whilst minimizing un-necesary overhead and estimating ONLY this factors that matter most.
Agile Analysis 101: Agile Stats v Command & Control MathsAxelisys Limited
Introducing Agile teams to Statistical Analysis. It's the tool that will help them self-manage and I introduce simple methods to measure efficacy. We also compare and contrast the traditional use of mathematics for command and control versus statistics and learning for contemporary agile development and EA.
Most teams need to answer questions like “When will it be done? What can I get by date X?”. However, common estimation approaches often fail to give us the predictability we want, and tend to introduce bad behaviours like hard deadlines and hiding uncertainty.
In this session, I’ll show you how, step by step and with real life examples, my team uses their historical data and metrics to forecast the future and answer these questions with confidence.
Download slides at: http://bit.ly/2pD9rfQ
Book discount link: http://leanpub.com/metricsforbusinessdecisions/c/MATTIA20-BZXib2F
Searching for Anomalies, by Thomas Dietterich, Distinguished Professor Emeritus in the School of EECS at Oregon State University and Chief Scientist of BigML.
*MLSEV 2020: Virtual Conference.
[CXL Live 16] How to Utilize Your Test Capacity? by Ton WesselingCXL
Ton Wesseling gave a presentation at ConversionXL Live in Austin on March 31st 2016 about utilizing test capacity. He discussed optimizing conversions through the ROAR model of risk, optimization, automation and re-thinking. Wesseling emphasized fully using a company's test capacity for impactful A/B tests and separating that capacity for IT releases, campaigns and behavioral learning. He advised celebrating failures to encourage risk-taking and continuous learning.
Leveraging Analytics In Gaming - Tiny Mogul GamesInMobi
'Analytics In Gaming' and how you can use it to improve the game's acquisition, retention and engagement' by Rajdeep Gumaste, Product Manager - Tiny Mogul Games.
Kanban Metrics in practice for leading Continuous ImprovementMattia Battiston
Why should I bother collecting metrics? How can they help me? My CFD is pretty and colourful, but what is it actually trying to tell me?
CFD, control chart, lead time distribution, percentiles...Metrics can be daunting to start with but if you know how to interpret them they can drive continuous improvement and forecast the future and take your Kanban system to the next level! It’s much easier than you think, no need for complex maths or expensive software.
At Sky Network Services a few teams are using Kanban and metrics. In this talk I’ll share our experience: what metrics we use, how we use each one of them, what little data we collect to get a whole lot of value, what pitfalls we encountered.
Downloads
Powerpoint: https://goo.gl/4CkKJd
PDF: https://goo.gl/VDW93U
Kanban Metrics in practice at Sky Network ServicesMattia Battiston
Why should I bother collecting metrics? How can they help me? My CFD is pretty and colourful, but what is it actually trying to tell me?
CFD, control chart, lead time distribution, percentiles...Metrics can be daunting to start with but if you know how to interpret them they can really take your Kanban system to the next level - drive continuous improvement and forecast the future! It’s much easier than you think, no need for complex maths or expensive software.
At Sky Network Services a few teams are using Kanban and metrics. In this talk I’ll share our experience: what metrics we use, how we use each one of them, what little data we collect to get a whole lot of value, what pitfalls we encountered.
Downloads
Powerpoint: https://goo.gl/19wOjU
PDF: https://goo.gl/AM69MF
Skepticism at work - Logical Fallacies. ASQ BuffaloASQ Buffalo NY
http://asqbuffalo.org
We should only believe things for which there is adequate evidence. But what evidence is adequate? We decide based on our experiences, beliefs, judgments, and reasoning. However, we often commit logical fallacies (errors in reasoning). When this happens either sound evidence is rejected or questionable evidence is accepted while leaving us feeling that we have made the right decision. Recognizing these errors can help us avoid making wrong decisions and promote data based decision making.
Black Swan Risk Management - Aditya YadavAditya Yadav
This document discusses managing risks from "black swan" events, which are rare events with severe consequences that are often rationalized with hindsight. The author argues that a probabilistic or statistical approach is inappropriate for black swan risk management. Instead, organizations should use scenario-based modeling to simulate assumption failures, identify model sanity checks, and prepare reactive measures. The key is having a general consensus on risk themes and practices for different categories of model breakdowns, rather than rigid procedures, so people understand risks mentally.
Statistics in the age of data science, issues you can not ignoreTuri, Inc.
This document discusses issues in statistics that data scientists can and cannot ignore when working with large datasets. It begins by outlining the talk and defining key terms in data science. It then explains that model assessment, such as estimating model performance on new data, becomes easier with more data as statistical adjustments are not needed. However, more data and variables are not always better, as noise, collinearity, and overfitting can still occur. Several examples are given where common machine learning algorithms can be fooled into achieving high accuracy on training data even when the target variable is random. The conclusion emphasizes that data science, statistics, and domain expertise each provide unique perspectives, and effective teams need to understand all views.
The document discusses some of the risks and challenges of data visualization and analytics programs in organizations. It argues that while complex data visualizations can work, they are difficult to implement successfully from scratch. Additionally, stakeholders may claim the benefits from outside ideas while only superficially complying with analytics recommendations. The document provides steps for organizations to truly realize change through data-driven insights, such as having leadership buy-in and starting with small, test-based implementations.
Replication in Data Science - A Dance Between Data Science & Machine Learning...June Andrews
We use Iterative Supervised Clustering as a simple building block for exploring Pinterest's Content. But simplicity can unlock great power and with this building block we show the shocking result of how hard it is to replicated data science conclusions. This begs us to challenge the future for When is Data Science a House of Cards?
5 Whys: Originally developed by Sakichi Toyoda and used within the Toyota Motor Corporation during the evolution of its manufacturing methodologies, 5 Whys is a basic component of problem-solving. By asking ‘Why’ 5 times it encourages the problem solver to avoid assumptions and logic traps and trace the chain of causality from the effect seen through to a root cause. The real root cause should point toward a process that is not working well or does not exist.
MLSEV Virtual. Automating Model SelectionBigML, Inc
1) Bayesian parameter optimization uses machine learning to predict the performance of untrained models based on parameters from previous models to efficiently search the parameter space.
2) However, there are still important issues like choosing the right evaluation metric, ensuring no information leakage between training and test data, and selecting the appropriate model for the problem and available data.
3) Automated model selection requires sufficient data to make accurate predictions; with insufficient data, the process can fail.
This document discusses achieving "ready-ready" stories by having more than a sprint's worth of stories prepared in advance. It recommends having a chief product owner approve stories that meet INVEST criteria and can be estimated quickly. Stories should be tested, have API/UI mockups approved, and be achievable within a sprint. Analyzing defects can help improve the ready-ready process by addressing issues like communication, knowledge, or availability. Presenting the ready-ready process to the organization should provide direction, gain support, and outline key performance indicators to help achieve goals. Starting small experiments is encouraged to test the approach.
This document discusses forecasting software delivery using probabilistic methods rather than estimations. It defines key terms like prediction, estimation, forecast, and Monte Carlo simulation. The problems with traditional estimations are outlined, such as being inaccurate and not accounting for variability. Forecasting is presented as a better alternative that uses historical data to generate probability distributions of possible outcomes. Examples are given of forecasting delivery times for single issues, multiple issues with different resources, and rolling forecasts as new data comes in. The benefits of forecasting are noted as being more factual, repeatable, and reducing stress compared to estimations. Implications are that issue tracking and development practices need to support forecasting to improve predictability.
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...CXL
Individual tests drive insights & ROI, but the most sophisticated optimizers look beyond what an individual test is telling them and use data to optimize their overall testing performance.
In this talk, Claire will dive into the specifics of how to track, improve, and drive insight from performance metrics for your conversion program, so you can not only run better tests, but get more out of your investment in CRO.
Statistics for UX Professionals - Jessica CameronUser Vision
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
State of the Art in Machine Learning, by Thomas Dietterich, Distinguished Professor Emeritus in the School of EECS at Oregon State University and Chief Scientist of BigML.
*MLSEV 2020: Virtual Conference.
CYCLE TIME ANALYTICS: RELIABLE #NOESTIMATES FORECASTING USING DATA, TROY MAGE...Lean Kanban Central Europe
If you are struggling to forecast project delivery dates and cost, or you want to eliminate the story estimation process because you feel it is waste, or you need to build the business case for hiring more staff, then this session is relevant to you. All estimates have uncertainty, and understanding how multiple uncertain factors compound is the first step to improving project and team predictability. A major benefit of Lean is the low weight capture of cycle time metrics. This session looks at how to use historical cycle time data to answer questions of forecasting and staff skill balancing. This session compares the benefits of using cycle time for analysis over current planning techniques such as velocity, burn-down charts, and cumulative flow diagrams. This session takes you on a journey of what to do after capturing cycle time data or what to do if you have no history to rely upon. Reducing reliance on developer estimation (popularized by the twitter hashtag of #NoEstimates movement) is good general advice, having the tools to plan and manage teams and projects is still important to maintain support at the executive level. This session details the approaches to getting the numbers you need to have whilst minimizing un-necesary overhead and estimating ONLY this factors that matter most.
Agile Analysis 101: Agile Stats v Command & Control MathsAxelisys Limited
Introducing Agile teams to Statistical Analysis. It's the tool that will help them self-manage and I introduce simple methods to measure efficacy. We also compare and contrast the traditional use of mathematics for command and control versus statistics and learning for contemporary agile development and EA.
Most teams need to answer questions like “When will it be done? What can I get by date X?”. However, common estimation approaches often fail to give us the predictability we want, and tend to introduce bad behaviours like hard deadlines and hiding uncertainty.
In this session, I’ll show you how, step by step and with real life examples, my team uses their historical data and metrics to forecast the future and answer these questions with confidence.
Download slides at: http://bit.ly/2pD9rfQ
Book discount link: http://leanpub.com/metricsforbusinessdecisions/c/MATTIA20-BZXib2F
Searching for Anomalies, by Thomas Dietterich, Distinguished Professor Emeritus in the School of EECS at Oregon State University and Chief Scientist of BigML.
*MLSEV 2020: Virtual Conference.
[CXL Live 16] How to Utilize Your Test Capacity? by Ton WesselingCXL
Ton Wesseling gave a presentation at ConversionXL Live in Austin on March 31st 2016 about utilizing test capacity. He discussed optimizing conversions through the ROAR model of risk, optimization, automation and re-thinking. Wesseling emphasized fully using a company's test capacity for impactful A/B tests and separating that capacity for IT releases, campaigns and behavioral learning. He advised celebrating failures to encourage risk-taking and continuous learning.
Leveraging Analytics In Gaming - Tiny Mogul GamesInMobi
'Analytics In Gaming' and how you can use it to improve the game's acquisition, retention and engagement' by Rajdeep Gumaste, Product Manager - Tiny Mogul Games.
Kanban Metrics in practice for leading Continuous ImprovementMattia Battiston
Why should I bother collecting metrics? How can they help me? My CFD is pretty and colourful, but what is it actually trying to tell me?
CFD, control chart, lead time distribution, percentiles...Metrics can be daunting to start with but if you know how to interpret them they can drive continuous improvement and forecast the future and take your Kanban system to the next level! It’s much easier than you think, no need for complex maths or expensive software.
At Sky Network Services a few teams are using Kanban and metrics. In this talk I’ll share our experience: what metrics we use, how we use each one of them, what little data we collect to get a whole lot of value, what pitfalls we encountered.
Downloads
Powerpoint: https://goo.gl/4CkKJd
PDF: https://goo.gl/VDW93U
Kanban Metrics in practice at Sky Network ServicesMattia Battiston
Why should I bother collecting metrics? How can they help me? My CFD is pretty and colourful, but what is it actually trying to tell me?
CFD, control chart, lead time distribution, percentiles...Metrics can be daunting to start with but if you know how to interpret them they can really take your Kanban system to the next level - drive continuous improvement and forecast the future! It’s much easier than you think, no need for complex maths or expensive software.
At Sky Network Services a few teams are using Kanban and metrics. In this talk I’ll share our experience: what metrics we use, how we use each one of them, what little data we collect to get a whole lot of value, what pitfalls we encountered.
Downloads
Powerpoint: https://goo.gl/19wOjU
PDF: https://goo.gl/AM69MF
Skepticism at work - Logical Fallacies. ASQ BuffaloASQ Buffalo NY
http://asqbuffalo.org
We should only believe things for which there is adequate evidence. But what evidence is adequate? We decide based on our experiences, beliefs, judgments, and reasoning. However, we often commit logical fallacies (errors in reasoning). When this happens either sound evidence is rejected or questionable evidence is accepted while leaving us feeling that we have made the right decision. Recognizing these errors can help us avoid making wrong decisions and promote data based decision making.
Black Swan Risk Management - Aditya YadavAditya Yadav
This document discusses managing risks from "black swan" events, which are rare events with severe consequences that are often rationalized with hindsight. The author argues that a probabilistic or statistical approach is inappropriate for black swan risk management. Instead, organizations should use scenario-based modeling to simulate assumption failures, identify model sanity checks, and prepare reactive measures. The key is having a general consensus on risk themes and practices for different categories of model breakdowns, rather than rigid procedures, so people understand risks mentally.
Statistics in the age of data science, issues you can not ignoreTuri, Inc.
This document discusses issues in statistics that data scientists can and cannot ignore when working with large datasets. It begins by outlining the talk and defining key terms in data science. It then explains that model assessment, such as estimating model performance on new data, becomes easier with more data as statistical adjustments are not needed. However, more data and variables are not always better, as noise, collinearity, and overfitting can still occur. Several examples are given where common machine learning algorithms can be fooled into achieving high accuracy on training data even when the target variable is random. The conclusion emphasizes that data science, statistics, and domain expertise each provide unique perspectives, and effective teams need to understand all views.
The document discusses some of the risks and challenges of data visualization and analytics programs in organizations. It argues that while complex data visualizations can work, they are difficult to implement successfully from scratch. Additionally, stakeholders may claim the benefits from outside ideas while only superficially complying with analytics recommendations. The document provides steps for organizations to truly realize change through data-driven insights, such as having leadership buy-in and starting with small, test-based implementations.
Replication in Data Science - A Dance Between Data Science & Machine Learning...June Andrews
We use Iterative Supervised Clustering as a simple building block for exploring Pinterest's Content. But simplicity can unlock great power and with this building block we show the shocking result of how hard it is to replicated data science conclusions. This begs us to challenge the future for When is Data Science a House of Cards?
5 Whys: Originally developed by Sakichi Toyoda and used within the Toyota Motor Corporation during the evolution of its manufacturing methodologies, 5 Whys is a basic component of problem-solving. By asking ‘Why’ 5 times it encourages the problem solver to avoid assumptions and logic traps and trace the chain of causality from the effect seen through to a root cause. The real root cause should point toward a process that is not working well or does not exist.
MLSEV Virtual. Automating Model SelectionBigML, Inc
1) Bayesian parameter optimization uses machine learning to predict the performance of untrained models based on parameters from previous models to efficiently search the parameter space.
2) However, there are still important issues like choosing the right evaluation metric, ensuring no information leakage between training and test data, and selecting the appropriate model for the problem and available data.
3) Automated model selection requires sufficient data to make accurate predictions; with insufficient data, the process can fail.
This document discusses achieving "ready-ready" stories by having more than a sprint's worth of stories prepared in advance. It recommends having a chief product owner approve stories that meet INVEST criteria and can be estimated quickly. Stories should be tested, have API/UI mockups approved, and be achievable within a sprint. Analyzing defects can help improve the ready-ready process by addressing issues like communication, knowledge, or availability. Presenting the ready-ready process to the organization should provide direction, gain support, and outline key performance indicators to help achieve goals. Starting small experiments is encouraged to test the approach.
Black Magic of the Advanced Scrum MasterGil Nahmias
The document discusses tactics for Scrum Masters to establish trust and leadership. It provides tips for getting stakeholders to like and trust the Scrum Master such as smiling, remembering details, getting feedback, and establishing trust first. It also covers handling resistance, motivating teams, and establishing mastery, autonomy and purpose. The document warns against unethical tactics like applying time pressure or psychological manipulation.
The document discusses various code quality metrics that can be used to understand software quality, including defects density, unit test density, code and test coverage, cyclomatic complexity, fan-in and fan-out, and WTFs per minute. While metrics can help identify issues, they cannot determine the precise cause. Metrics should be used carefully to avoid incentivizing behaviors like hiding bugs. Maintaining pride in craftsmanship is important for quality.
High Performance Teams: The 4 KPIs of SuccessQELIedu
This document discusses the keys to developing high performance teams. It identifies 4 key performance indicators (KPIs) of success: 1) having a common vision and clear actions, 2) clear accountability and performance reporting, 3) leveraging diversity and leading by example, and 4) awareness and support of individual work/life goals. A case study shows that implementing a 3-phase high performance team program focused on these KPIs led to improved job demands, satisfaction, engagement, and a six month ROI of $254,951.50 for a team. Developing high performance teams can help organizations thrive through innovation, savings, and growth rather than just survive challenges.
This document provides details about a course on random variables and stochastic processes. It includes:
- An overview of the course content which will cover probability theory, random variables, distributions, and stochastic processes.
- Information about assignments, quizzes, grading policy, textbooks, and the instructor's office hours.
- Examples and explanations of key concepts from probability theory that will be covered, including sample spaces, probability values, events, and complements of events. Applications to games of chance, software errors, and power plant operations are discussed.
- The goal of developing mathematical tools to analyze and characterize random signals and stochastic processes is stated.
IT's A/B-testing and lean-startup techniques can learn a lot from experimental design and statistics. For those of you not that confident or familiar with such techniques, here is a little intro to help you on your way :)
This document discusses Naive Bayes classifiers and k-nearest neighbors (kNN) algorithms. It begins with an overview of Naive Bayes, including how it makes strong independence assumptions between attributes. Several examples are provided to illustrate Naive Bayes classification. The document then covers kNN, explaining that it is an instance-based learning method that classifies new examples based on their similarity to training examples. Parameters like the number of neighbors k and distance metrics are discussed.
CAPA management, corrective and preventive action, Rootcause analysis, RCA, Problem mapping, FMEA, Failure Mode effect and Analysis, Fault Tree analysis, Fishbone : ISHIKAWA, CTQ Tree (Critical to Quality Tree), AFFINITY DIAGRAM, 5 Why’s, Human errors,
This document provides guidance on writing skills and best practices for research methods and communications. It discusses writing different types of documents like essays, cover letters, and dissertations. It provides a marking scheme for essay-style questions and criteria for different grade levels. It offers tips for structuring writing clearly with separate introduction, one paragraph per idea, and smooth transitions. The document also discusses using functions and loops in R programming and gives examples. Finally, it promotes practices like reproducible research, sustainable software, and taking notes in Markdown.
Monte Carlo Schedule Risk Analysis: The Concept, Benefits, and Limitations. How Monte Carlo schedule risk analysis works; how to perform Monte Carlo simulations of project schedules.
For more information how to perform schedule risk analysis using RiskyProject software please visit Intaver Institute web site: http://www.intaver.com.
About Intaver Institute.
Intaver Institute Inc. develops project risk management and project risk analysis software. Intaver's flagship product is RiskyProject: project risk management software. RiskyProject integrates with Microsoft Project, Oracle Primavera, other project management software or can run standalone. RiskyProject comes in three configurations: RiskyProject Lite, RiskyProject Professional, and RiskyProject Enterprise.
ISSTA'16 Summer School: Intro to StatisticsAndrea Arcuri
This document provides an introduction to statistics, data analysis, and visualization in R and Latex. It discusses why statistics are important for experiments and compares different statistical tests that can be used to analyze data, including the Fisher Exact test, Wilcoxon-Mann-Whitney U-test, and Student's t-test. It also addresses issues around sample sizes and distributions, emphasizing that statistics are useful for handling limited data and determining practical versus statistical significance.
More than three years after Elizabeth Holmes was first indicted and nearly four months after her trial kicked off, the founder and former CEO of failed blood testing startup Theranos was found guilty on four out of 11 federal fraud and conspiracy charges.
The verdict comes after a stunning downfall that saw Holmes, once hailed as the next Steve Jobs, go from being a tech industry icon to being a rare Silicon Valley entrepreneur on trial for fraud.
A Stanford University dropout, Holmes – inspired by her own fear of needles – started the company at the age of 19, with a mission of creating a cheaper, more efficient alternative to a traditional blood test. Theranos promised patients the ability to test for conditions like cancer and diabetes with just a few drops of blood. She attracted hundreds of millions of dollars in funding, a board of well-known political figures, and key retail partners.
But a Wall Street Journal investigation poked holes into Theranos’ testing and technology, and the dominoes fell from there. Holmes and her former business partner, Ramesh “Sunny” Balwani, were charged in 2018 by the US government with multiple counts of wire fraud and conspiracy to commit wire fraud. (Both pleaded not guilty.)
Here are the highlights of the rise and fall of Elizabeth Holmes and Theranos.
Lecture 2: Data, pre-processing and post-processing
Chapters 2,3 from the book “Introduction to Data Mining” by Tan, Steinbach, Kumar.
Chapter 1 from the book Mining Massive Datasets by Anand Rajaraman and Jeff Ullman
chapter-00-01.ppt analytical chemistry for collegejoygalero
This document provides information about an introductory quantitative analysis chemistry course. It outlines the course details including instructors, meeting times, exams, grading breakdown, textbook, and lab schedule. It also describes the course expectations for labs, lab notebooks, and homework problems. Key concepts covered in lectures and experimental techniques are briefly introduced.
This document provides information and steps for performing a root cause analysis when investigating failures or mishaps. It defines key terms like proximate cause, root cause, and root cause analysis. The root cause analysis process involves clearly defining the undesired outcome, gathering data, creating a timeline, developing a causal factors tree to identify all potential underlying causes, and determining the root causes and solutions to prevent recurrence.
This document discusses the use of statistics and probabilities in corpus linguistics. It explains that statistics can provide useful tools for linguists to better understand languages. Probabilities in particular can be used to estimate word frequencies and develop probabilistic models of spelling. The document also discusses best practices for annotating corpora, including annotating with sufficient data to achieve statistical significance and avoiding errors like testing machine learning models on the same data they were trained on.
The document describes the Kepner-Tregoe methodology, a structured approach for problem solving, decision making, and risk analysis. It was developed in the 1960s and has been used by teams like those that solved problems during the Apollo 13 mission. The methodology involves systematically gathering information, prioritizing objectives, generating and evaluating alternatives, and verifying solutions. It provides step-by-step guidance for tasks like defining problems, identifying potential causes, testing solutions, and monitoring outcomes. Examples are given for applying the various steps to hypothetical problems regarding product defects, customer issues, and other scenarios.
There are a few potential issues with modeling the data this way:
1. Students are nested within classrooms. A student's outcomes may be more similar to others in their classroom compared to students in other classrooms, due to shared classroom factors. This violates the independence assumption of ordinary least squares regression.
2. Classroom-level factors like teacher quality are not included in the model but likely influence student outcomes. Failing to account for these could lead to omitted variable bias.
3. The error terms for students within the same classroom may not be independent as assumed, since classroom factors induce correlation.
To properly account for the nested data structure, we need to model the classroom as a second level in a multilevel
Process variation and continuous improvementsTarek Elneil
The document discusses various continuous improvement tools and techniques:
1) It describes process components and characteristics, process variations and causes, and process improvement methodologies.
2) It then provides a fruit pizza recipe including ingredients and directions.
3) Finally, it discusses tools for nonconformance investigation such as root cause analysis, which aims to identify and eliminate the true root causes of process failures through a systematic approach.
This document provides training on using root cause analysis to understand and solve problems. It explains that to solve problems, their root causes must be identified rather than just addressing symptoms. An example problem of a memory leak is used, where a better solution is finding and fixing the source rather than just buying more memory. Cause-effect diagrams are presented as a tool to trace problems to their root causes by asking "why" multiple times and looking for loops. Two example problems are analyzed using this approach to understand business impacts and identify unexpected underlying causes in order to propose effective countermeasures.
This is a follow up on a previous talk on hacking my energy monitor. In this talk I go into detail on how I used Machine Learning techniques in the area of Anomaly Detection to draw more value from my data collection.
Similar to LKNA 2014 Risk and Impediment Analysis and Analytics - Troy Magennis (20)
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Tastemy Pandit
Know what your zodiac sign says about your taste in food! Explore how the 12 zodiac signs influence your culinary preferences with insights from MyPandit. Dive into astrology and flavors!
SATTA MATKA SATTA FAST RESULT KALYAN TOP MATKA RESULT KALYAN SATTA MATKA FAST RESULT MILAN RATAN RAJDHANI MAIN BAZAR MATKA FAST TIPS RESULT MATKA CHART JODI CHART PANEL CHART FREE FIX GAME SATTAMATKA ! MATKA MOBI SATTA 143 spboss.in TOP NO1 RESULT FULL RATE MATKA ONLINE GAME PLAY BY APP SPBOSS
Starting a business is like embarking on an unpredictable adventure. It’s a journey filled with highs and lows, victories and defeats. But what if I told you that those setbacks and failures could be the very stepping stones that lead you to fortune? Let’s explore how resilience, adaptability, and strategic thinking can transform adversity into opportunity.
The APCO Geopolitical Radar - Q3 2024 The Global Operating Environment for Bu...APCO
The Radar reflects input from APCO’s teams located around the world. It distils a host of interconnected events and trends into insights to inform operational and strategic decisions. Issues covered in this edition include:
❼❷⓿❺❻❷❽❷❼❽ Dpboss Matka Result Satta Matka Guessing Satta Fix jodi Kalyan Final ank Satta Matka Dpbos Final ank Satta Matta Matka 143 Kalyan Matka Guessing Final Matka Final ank Today Matka 420 Satta Batta Satta 143 Kalyan Chart Main Bazar Chart vip Matka Guessing Dpboss 143 Guessing Kalyan night
Unveiling the Dynamic Personalities, Key Dates, and Horoscope Insights: Gemin...my Pandit
Explore the fascinating world of the Gemini Zodiac Sign. Discover the unique personality traits, key dates, and horoscope insights of Gemini individuals. Learn how their sociable, communicative nature and boundless curiosity make them the dynamic explorers of the zodiac. Dive into the duality of the Gemini sign and understand their intellectual and adventurous spirit.
Part 2 Deep Dive: Navigating the 2024 Slowdownjeffkluth1
Introduction
The global retail industry has weathered numerous storms, with the financial crisis of 2008 serving as a poignant reminder of the sector's resilience and adaptability. However, as we navigate the complex landscape of 2024, retailers face a unique set of challenges that demand innovative strategies and a fundamental shift in mindset. This white paper contrasts the impact of the 2008 recession on the retail sector with the current headwinds retailers are grappling with, while offering a comprehensive roadmap for success in this new paradigm.
Best Competitive Marble Pricing in Dubai - ☎ 9928909666Stone Art Hub
Stone Art Hub offers the best competitive Marble Pricing in Dubai, ensuring affordability without compromising quality. With a wide range of exquisite marble options to choose from, you can enhance your spaces with elegance and sophistication. For inquiries or orders, contact us at ☎ 9928909666. Experience luxury at unbeatable prices.
Call8328958814 satta matka Kalyan result satta guessing➑➌➋➑➒➎➑➑➊➍
Satta Matka Kalyan Main Mumbai Fastest Results
Satta Matka ❋ Sattamatka ❋ New Mumbai Ratan Satta Matka ❋ Fast Matka ❋ Milan Market ❋ Kalyan Matka Results ❋ Satta Game ❋ Matka Game ❋ Satta Matka ❋ Kalyan Satta Matka ❋ Mumbai Main ❋ Online Matka Results ❋ Satta Matka Tips ❋ Milan Chart ❋ Satta Matka Boss❋ New Star Day ❋ Satta King ❋ Live Satta Matka Results ❋ Satta Matka Company ❋ Indian Matka ❋ Satta Matka 143❋ Kalyan Night Matka..
IMPACT Silver is a pure silver zinc producer with over $260 million in revenue since 2008 and a large 100% owned 210km Mexico land package - 2024 catalysts includes new 14% grade zinc Plomosas mine and 20,000m of fully funded exploration drilling.
The Genesis of BriansClub.cm Famous Dark WEb PlatformSabaaSudozai
BriansClub.cm, a famous platform on the dark web, has become one of the most infamous carding marketplaces, specializing in the sale of stolen credit card data.
NIMA2024 | De toegevoegde waarde van DEI en ESG in campagnes | Nathalie Lam |...BBPMedia1
Nathalie zal delen hoe DEI en ESG een fundamentele rol kunnen spelen in je merkstrategie en je de juiste aansluiting kan creëren met je doelgroep. Door middel van voorbeelden en simpele handvatten toont ze hoe dit in jouw organisatie toegepast kan worden.
5. Technical Risk
Financial
Risk
Market Risk
• Real Options
• Right Staff / liquidity
• Dev Practices
• Dependencies
• Constraints
• Lean Startup
• Agile Processes
• Competitive
Awareness
• Having
funding/cash
• Having a
strategy
• Economic
prioritization
• Real Options
“Aleatory Risk”
Cannot be reduce by more info
7. Key Point
Occurrence of a risk Increases
exposure to other risks
Break the chain early
AKA: Early and meaningful
contact with enemy – RISK
(source: quote from Reinertsen, but sources from US marines?)
8.
9.
10. Correlation != Causation
We can see average flight delay
matches the shape of “Late
Aircraft,” but don’t yet know why…
11. Key Point
Serialized dependencies cascade
delays, but are not the root cause –
Why was the aircraft late?
The later you are, the later you get.
12. Four people arrange a
restaurant booking after work
Q. What is the chance they
arrive on-time to be seated?
23. If you haven’t seen an event after
testing for it n times, you can be
95% sure that its probability of
happening is less than
3/n
References: Wikipedia: Statistical Rule of Three and Thanks to John Cook: Estimating the chances of something that hasn’t happened yet,
http://www.johndcook.com/blog/2010/03/30/statistical-rule-of-three/
The Math: (1-p)n = 0.05 for p. Taking logs of both sides, n ln (1-p) = ln(0.05) ≈ -3.
Since log(1-p) is approximately -p for small values of p, we have p ≈ 3/n.
24. Statistical Rule of Three
• Example: Proofreading a
book, you find no
grammatical errors in n pages
• Error decreases as a
proportion to the number of
independent test cases
examined
• It hard to be
independent!
n percentage
20 15% (3/20)
100 3% (3/100)
200 1.5% (3/200)
500 0.6% (3/500)
1000 0.3% (3/1000)
0.00000
0.10000
0.20000
0.30000
0.40000
0.50000
0.60000
0.70000
0.80000
1
21
41
61
81
101
121
141
161
181
201
221
241
261
281
301
321
341
361
381
401
421
441
461
481
p
25. ‘s Absence of Evidence isn’t
Evidence of Absence
But, it does demonstrate the
occurrence is rare with
growing certainty
Depends on consequence….
Ps. The most common
Black Swan is project
on-time delivery!
28. “Value”
Cost of Delay
Product 1
Product 2
Product 3
Complete
Order?
3
2
1
“Time”
Remaining
Time/Effort to solve
Economic Prioritization – same time, different value
29. Product 1
Product 2
Product 3
1
2
3
Economic Prioritization – same value different time
“Value”
Cost of Delay
Complete
Order?
“Time”
Remaining
Time/Effort to solve
30. W.S.R.F. =
Prioritization Heuristic
to optimize reward
“Do Highest First”
Impact of risk
Time to resolve/mitigate
Weighted Shortest Risk First
Sum of delay time
of same risk causes
over the last 3 (?)
months
Effort estimate of
the resolution time
of risk root cause
31.
32. All Sheep in Scotland Are Black
• A psychologist, a biologist, a mathematician, and a physicist were riding
a train through the Scottish countryside. Looking out the window, they
all noticed a lone black sheep on a hill.
• The psychologist intoned, “Well, what do you know. I didn’t realize the
sheep in Scotland were black.”
• The biologist corrected him, saying, “You don’t know that all the sheep in
Scotland are black – just some of them.”
• Piping in, the mathematician retorted, “Tut, tut, tut, to be correct you
must say, ‘At least one’ sheep in Scotland is black.”
• The physicist had the last word, though, stating, “Gentlemen, all we know
with certainty based on our observations is that at least one sheep in
Scotland is black on at least one side, at least part of the time.”
• Moral: There are hard and soft sciences, and extrapolation is not always
justified.
http://creationsafaris.com/humor.htm
33. Total
Story
Lead
Time
30
days
Story / Feature Inception
5 Days
Waiting in Backlog
25 days
System Regression Testing & Staging
5 Days
Waiting for Release Window
5 Days
“Active Development”
30 days
Pre
Work
30
days
Post
Work
10
days
9 days (70 total)
approx 13%
34. THE SHAPE OF CYCLE TIME
What distribution fits cycle time data and why…
35. If we understand how cycle time is
statistically distributed, then an
initial guess of maximum allows an
inference to be made
Alternatives -
• Borrow a similar project’s data
• Borrow industry data
• Fake it until you make it… (AKA guess range)
36. Why Weibull
• Now for some Math – I know, I’m excited too!
• Simple Model
• All units of work between 1 and 3 days
• A unit of work can be a task, story, feature, project
• Base Scope of 50 units of work – Always Normal
• 5 Delays / Risks, each with
– 25% Likelihood of occurring
– 10 units of work (same as 20% scope increase each)
37. Normal, or it will
be after a few
thousand more
simulations
44. Exponential Distribution (Weibull shape = 1)
The person who gets the work can complete the work
Teams with no external dependencies
Teams doing repetitive work E.g. DevOps, Database teams,
46. Rayleigh Distribution (Weibull shape = 2)
Teams with MANY external dependencies
Teams that have many delays and re-work. E.g. Test teams
47. What Distribution To Use...
• No Data at All, or Less than < 11 Samples (why 11?)
– Uniform Range with Boundaries Guessed (safest)
– Weibull Range with Boundaries Guessed (likely)
• 11 to 30 Samples
– Uniform Range with Boundaries at 5th and 95th CI
– Weibull Range with Boundaries at 5th and 95th CI
• More than 30 Samples
– Use historical data as bootstrap reference
– Curve Fitting software
48. Probability Density Function
Histogram Weibull
x
1201101009080706050403020100
f(x)
0.28
0.24
0.2
0.16
0.12
0.08
0.04
0
Scale – How Wide in
Range. Related to the
Upper Bound. *Rough*
Guess: (High – Low) / 4
Shape – How Fat the
distribution. 1.5 is a
good starting point.
Location – The
Lower Bound
Editor's Notes
What is the chance the aircraft is late:Higher chance later in the day after n hopsHigher chance if aircraft coming from a city with bad seasonal weatherHigher chance of delay if the airport a plan is coming from isn’t a hub (staff and plan availability)
& Deaf frogs don’t jump
My name is Troy Magennis, I’ve been in software for 25 years now, from QA through to VP Architecture and Development for companies like Travelocity and Lastminute.com. Most recently I formed my own company building tools and running training on software development forecasting and risk management solutions. Feel free to take notes, but the slides and examples are available to you online. And as a special benefit for joining us today, you can download the software used throughout this session for free. Bit.ly/agilesim will take you to the right site. I wrote a book about these topics, “Forecasting and Simulating Software Development Projects” and I’d like to make sure you all got a free PDF copy of this book also. Just download it from the same location.