Here is a presentation given to the programmers at Insomniac Games on July 6, 2007. It covers two related topics. First we consider a case study of an optimization to NPC AI used on Resistance: Fall of Man. Then we discuss a broader proposal for a game-play architecture that supports and encourages higher performance code compared to the common game-object driven main loop.
The presentation is also available on Insomniac\'s public R&D website:
http://www.insomniacgames.com/tech/techpage.php
SEO has always sat at the intersection between being a science and an art. We all love to try out new ideas and try to understand what makes the search engines tick, but it can be frustrating to have to cut through the guesswork and speculation just to figure out what Google really wants from us. Even worse, we still find ourselves making SEO changes, seeing uplifts, but then not knowing which changes actually had any impact.
Fortunately, new software and better technologies now make it possible to run proper SEO-focused tests and, for the first time, actually measure the impact that each SEO change has on our site. Rob will share these techniques, discuss some of the experiments that Distilled has been running, reveal the unexpected things they’ve learned along the way, and share how you can start running experiments yourself.
An overview of gradient descent optimization algorithms Hakky St
This document provides an overview of various gradient descent optimization algorithms that are commonly used for training deep learning models. It begins with an introduction to gradient descent and its variants, including batch gradient descent, stochastic gradient descent (SGD), and mini-batch gradient descent. It then discusses challenges with these algorithms, such as choosing the learning rate. The document proceeds to explain popular optimization algorithms used to address these challenges, including momentum, Nesterov accelerated gradient, Adagrad, Adadelta, RMSprop, and Adam. It provides visualizations and intuitive explanations of how these algorithms work. Finally, it discusses strategies for parallelizing and optimizing SGD and concludes with a comparison of optimization algorithms.
[@IndeedEng] Managing Experiments and Behavior Dynamically with Proctorindeedeng
Video available at: http://youtu.be/Q1T5J0KXUwY
At this very moment, Indeed is running more than one hundred A/B experiments. In previous @IndeedEng talks, we have discussed how we use A/B testing to develop better products.
In this tech talk, software engineer Matt Schemmel and product manager Tom Bergman describe Proctor, the system we developed to define and manage all of these experiments. They explain how we use Proctor to target users using data-driven rules, adjust experiments on-the-fly, and ensure clean results for multi-variate tests. Over time, Proctor has evolved from a system designed for managing experiments to one that manages overall system behavior through dynamic "feature toggle" functionality. Matt and Tom also share lessons we have learned from years of experimenting at web scale.
Matt Schemmel is a Senior Software Engineer working primarily on our Resume products.
Tom Bergman is a Product Manager currently working on our Aggregation systems. He previously helped evolve many of Indeed's data analysis tools, and also helped us launch and grow our sites in Japan, Korea, and China.
This document discusses new directions for the khmer bioinformatics platform, including developing semi-streaming algorithms for sequence analysis using k-mers. Digital normalization is presented as an initial approach that compresses sequencing data, though it discards information. Later work introduced a two-pass semi-streaming framework using saturation detection to enable error correction and variant calling using minimal memory. Current work includes developing a pair-HMM-based graph aligner and applying it to tasks like variant calling. The khmer platform provides implementations of these streaming algorithms to enable analysis of large genomic and metagenomic datasets.
Analysis Services Best Practices From Large Deploymentsrsnarayanan
This document summarizes best practices for designing and optimizing Analysis Services cubes. It covers topics like cube design, storage and partitioning, aggregations, processing, and scalability. Specific tips include avoiding unnecessary attributes, using attribute relationships appropriately, partitioning data into slices of under 20 million rows, setting accurate aggregation usage properties, and using processing scripts for automation and control.
Are your application's tail-latencies holding it back from delivering its near-real time SLOs? Do your in-memory processing platform's long pauses only get worse with increasing heap sizes? How about those latency spikes causing variability in your end-to-end latency for your multi-tiered distributed systems?
If any of the above keep you up at night, then have no fear as Z Garbage Collector (GC) is here and is production ready in JDK 15.
In this talk, Monica Beckwith will cover the basics of Z GC and contrast it with G1 GC (the current default collector for OpenJDK JDK 11 LTS and tip).
Here is a presentation given to the programmers at Insomniac Games on July 6, 2007. It covers two related topics. First we consider a case study of an optimization to NPC AI used on Resistance: Fall of Man. Then we discuss a broader proposal for a game-play architecture that supports and encourages higher performance code compared to the common game-object driven main loop.
The presentation is also available on Insomniac\'s public R&D website:
http://www.insomniacgames.com/tech/techpage.php
SEO has always sat at the intersection between being a science and an art. We all love to try out new ideas and try to understand what makes the search engines tick, but it can be frustrating to have to cut through the guesswork and speculation just to figure out what Google really wants from us. Even worse, we still find ourselves making SEO changes, seeing uplifts, but then not knowing which changes actually had any impact.
Fortunately, new software and better technologies now make it possible to run proper SEO-focused tests and, for the first time, actually measure the impact that each SEO change has on our site. Rob will share these techniques, discuss some of the experiments that Distilled has been running, reveal the unexpected things they’ve learned along the way, and share how you can start running experiments yourself.
An overview of gradient descent optimization algorithms Hakky St
This document provides an overview of various gradient descent optimization algorithms that are commonly used for training deep learning models. It begins with an introduction to gradient descent and its variants, including batch gradient descent, stochastic gradient descent (SGD), and mini-batch gradient descent. It then discusses challenges with these algorithms, such as choosing the learning rate. The document proceeds to explain popular optimization algorithms used to address these challenges, including momentum, Nesterov accelerated gradient, Adagrad, Adadelta, RMSprop, and Adam. It provides visualizations and intuitive explanations of how these algorithms work. Finally, it discusses strategies for parallelizing and optimizing SGD and concludes with a comparison of optimization algorithms.
[@IndeedEng] Managing Experiments and Behavior Dynamically with Proctorindeedeng
Video available at: http://youtu.be/Q1T5J0KXUwY
At this very moment, Indeed is running more than one hundred A/B experiments. In previous @IndeedEng talks, we have discussed how we use A/B testing to develop better products.
In this tech talk, software engineer Matt Schemmel and product manager Tom Bergman describe Proctor, the system we developed to define and manage all of these experiments. They explain how we use Proctor to target users using data-driven rules, adjust experiments on-the-fly, and ensure clean results for multi-variate tests. Over time, Proctor has evolved from a system designed for managing experiments to one that manages overall system behavior through dynamic "feature toggle" functionality. Matt and Tom also share lessons we have learned from years of experimenting at web scale.
Matt Schemmel is a Senior Software Engineer working primarily on our Resume products.
Tom Bergman is a Product Manager currently working on our Aggregation systems. He previously helped evolve many of Indeed's data analysis tools, and also helped us launch and grow our sites in Japan, Korea, and China.
This document discusses new directions for the khmer bioinformatics platform, including developing semi-streaming algorithms for sequence analysis using k-mers. Digital normalization is presented as an initial approach that compresses sequencing data, though it discards information. Later work introduced a two-pass semi-streaming framework using saturation detection to enable error correction and variant calling using minimal memory. Current work includes developing a pair-HMM-based graph aligner and applying it to tasks like variant calling. The khmer platform provides implementations of these streaming algorithms to enable analysis of large genomic and metagenomic datasets.
Analysis Services Best Practices From Large Deploymentsrsnarayanan
This document summarizes best practices for designing and optimizing Analysis Services cubes. It covers topics like cube design, storage and partitioning, aggregations, processing, and scalability. Specific tips include avoiding unnecessary attributes, using attribute relationships appropriately, partitioning data into slices of under 20 million rows, setting accurate aggregation usage properties, and using processing scripts for automation and control.
Are your application's tail-latencies holding it back from delivering its near-real time SLOs? Do your in-memory processing platform's long pauses only get worse with increasing heap sizes? How about those latency spikes causing variability in your end-to-end latency for your multi-tiered distributed systems?
If any of the above keep you up at night, then have no fear as Z Garbage Collector (GC) is here and is production ready in JDK 15.
In this talk, Monica Beckwith will cover the basics of Z GC and contrast it with G1 GC (the current default collector for OpenJDK JDK 11 LTS and tip).
TCO19 Japan Introduction to Marathon Matchestomerun
This document discusses marathon matches on Topcoder. It describes two types of marathons - fun marathons involving combinatorial optimization problems and sponsored marathons involving real-world problems with cash prizes. An example marathon called PopulationMapping is provided, involving mapping population data on a 2D grid with the goal of finding sparsely populated areas using few queries. Typical activities for competitors during a marathon are outlined, including reading problems, developing initial solutions, improving solutions over multiple days, and optimizing solutions on the final day.
Journey of Migrating Millions of Queries on The Cloudtakezoe
This document discusses challenges in upgrading a query engine and summarizing strategies for efficiently simulating queries to test compatibility and performance. It proposes grouping queries by signature and narrowing data scans to reduce the number of queries tested. It also recommends automating result verification by generating human-readable reports and excluding uncheckable queries. Assistance tools are proposed to aid investigation of differences, which helped discover real bugs in the target version.
The document discusses theories and principles for increasing productivity and throughput (velocity). It presents queuing theory and the theory of constraints, explaining how to reduce utilization, batch size, and item size to improve flow based on queuing theory. It then provides 21 experiments teams can try based on these theories, such as reducing work in progress limits, splitting up work into smaller batches and items, and improving the identified constraint area.
Queuing Theory and the Theory of Constraints are two powerful theories that can increase your velocity. This session explains both theories in simple terms then covers how they can be applied in the real world by agile teams. 21 simple velocity increasing experiments are described that you can immediately use.
The effects of Queuing Theory impact our lives on a daily basis. Scrum uses Queuing Theory at its core and you can amplify those effects.
The Theory of Constraints can identify the one constraint that is preventing your team from increasing its velocity. It also shows us how to remove that constraint in the cheapest way possible.
Presented at Scrum Australia (@auscrum), Melbourne, 29 April 2016
1) The document describes a data mining hackathon that aimed to build predictive models to increase the subscription rate of Motley Fool visitors using demographic and weblog data.
2) The top models achieved a ROC score of 0.737 using boosted regression trees on weblog data alone, and 0.738 by also including demographic variables.
3) Key findings were that weblog data was more predictive than sparse demographic data, and that important predictor variables were pages viewed, location, and income.
The document discusses the history and features of garbage collection (GC) in Ruby. It notes that while GC has gotten a bad reputation for being slow or causing errors, it has actually been improved over time thanks to contributions from pioneering computer scientists. The document urges readers to see GC as an opportunity to strengthen their skills rather than a scapegoat for performance issues.
This document discusses managing uncertainty in value-based software engineering. It presents two goal functions - "ENERGY", which aims to reduce effort, defects, and schedule, and "Huang06", which balances being first to market against bugs. It also describes experiments that use automated search techniques to sample across the space of project options and model calibrations without explicit calibration, finding this controls estimates as well as explicit calibration.
These slides provide an overview of the basics of design of experiments. They also describe and give examples of categorical and continuous factors and responses, discrete numeric and mixture variables, and blocking factors. The slides were presented live and in recorded videos as part of the Mastering JMP webcast series. Watch the webcasts at http://www.jmp.com/mastering
An overview of gradient descent optimization algorithms.pdfvudinhphuong96
This document provides an overview of gradient descent optimization algorithms. It discusses various gradient descent variants including batch gradient descent, stochastic gradient descent (SGD), and mini-batch gradient descent. It describes the trade-offs between these methods in terms of accuracy, time, and memory usage. The document also covers challenges with mini-batch gradient descent like choosing a proper learning rate. It then discusses commonly used optimization algorithms to address these challenges, including momentum, Nesterov accelerated gradient, Adagrad, Adadelta, RMSprop, and Adam. It provides visualizations to explain how momentum and Nesterov accelerated gradient work to help accelerate SGD.
Improving throughput with the Theory of Constraints and Queuing TheoryAndrew Rusling
Practical advice on how to improve the throughput of your agile team, by using the Theory of Constraints and Queuing Theory. Shows how to apply TOC to your task board. Explains how Queuing Theory is built into Scrum and Kanban, powering you to make the most of them.
The document discusses the future of Ruby's bigdecimal library and number system. It outlines some current problems with BigDecimal including its use of global modes, lack of automatic precision tracking, and limitations on instance generation. It suggests improvements could be made to calculation speeds by implementing more advanced algorithms. Additionally, a new class is needed to represent irrational numbers as computable algorithms rather than decimal approximations.
This document summarizes research on SSD failures in a large data center over six years. It finds that SSD failure is strongly correlated with drive age, with most failures occurring in "young" drives before 1,500 program-erase cycles. Machine learning models are able to accurately predict SSD failures, with random forests performing best. The top predictive feature is drive age. While error rates increase before failures, no single error metric reliably indicates impending failure. Most failed drives remain in a failed state for months before being repaired.
This document introduces khmer, a platform for scalable sequence analysis. It discusses how khmer uses k-mers to provide implicit read alignments and assemble sequences using de Bruijn graphs. It also describes some of the challenges with k-mers, such as each sequencing error resulting in novel k-mers. The document outlines khmer's data structures and algorithms for efficiently counting k-mers and represents de Bruijn graphs. It discusses how khmer has been applied to real biological problems and highlights areas of current research using khmer, such as error correction, variant calling, and assembly-free comparisons of data sets.
Queuing Theory and the Theory of Constraints are two powerful theories that can increase your velocity. This session explains both theories in simple terms then covers how they can be applied in the real world by agile teams. 21 simple velocity increasing experiments are described that you can immediately use.
The effects of Queuing Theory impact our lives on a daily basis. Scrum uses Queuing Theory at its core and you can amplify those effects.
The Theory of Constraints can identify the one constraint that is preventing your team from increasing its velocity. It also shows us how to remove that constraint in the cheapest way possible.
Vladimir Primakov - Qa management in big agile teamsIevgenii Katsan
- Using a straightforward release pipeline with separate teams focused on new features or bug fixes to avoid context switching and overlapping work.
- Conducting cross-team planning and reviews to identify dependencies, risks, and adjust testing scope and approach accordingly.
- Establishing common processes, tools, and data across teams through alignment and documentation to facilitate collaboration.
- Ensuring infrastructure like test environments are automated, stable, and similar to production to support efficient testing across large teams.
The document discusses the anatomy of cascading failures in distributed systems. It describes common triggering conditions that can cause failures like planned changes, traffic fluctuations, resource starvation and crashes. It then explains how initial failures can cascade through load redistribution, retry amplification, latency creep and resource contention during recovery. Finally, it provides strategies for improving system resilience like robust architecture, chaos engineering, retrying policies, throttling, circuit breaking, fallbacks and choosing the right tools.
In every development process there is the question, do we invest enough on quality? Do we need to invest more? Every team knows about the dilemma of how many tests is the right amount of tests we should write. Is 80% test coverage is good enough? Maybe 90%? 100%? Should we invest more time in unit testing? Are we wasting too much time on unit-testing? Should we invest time on a faster rollback mechanism?
WIIFM
“Without data, you’re just another person with an opinion” - W. Edwards Deming
SLO Driven Development is a framework that helps the developers focus on impact and balance of every aspect of the dev process. When working currently with SLI, SLA, SLO and error budget you can learn where to invest in the development process.
Let’s talk about the importance of good SLOs and how they can help us improve our day2day
Speech of Alexey Vasiliev, Software Engineer at Railsware, at Ruby Meditation #25 Kyiv 08.12.2018
Next conference - http://www.rubymeditation.com/
In this talk, Alexey will tell about the project in which was necessary to implement A/B testing and what came out of it in result
Announcements and conference materials https://www.fb.me/RubyMeditation
News https://twitter.com/RubyMeditation
Photos https://www.instagram.com/RubyMeditation
The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation
Tuhin Mitra: How I Automate My Negative TestsAnna Royzman
This document discusses negative testing strategies for automating tests. It begins by defining negative testing as testing invalid or unexpected inputs to show how a system does not work. The document then provides examples of how writing negative tests can improve quality, coverage, stability and make applications more robust. Specific strategies are outlined, such as discussing negative acceptance criteria with users, separating tests into functional groups, and using feature flags to enable and disable features for different test types. The conclusion emphasizes changing to a more positive mindset about negative testing to achieve better outcomes.
Designing and Running Performance ExperimentsJ On The Beach
Load testing is a continuous process that involves designing realistic load tests based on real user models and data, running load tests at increasing user loads to explore the load curve, and analyzing the results in the context of production metrics to understand performance and detect saturation points. The goal is to load test applications with a purpose to maintain and improve performance, which has a significant impact on business metrics like revenue.
The document discusses improving engineering efficiency through the use of analytics. It describes how JMP software helps individuals, teams, and organizations increase engineering efficiency by enabling faster problem solving, proactive process improvement, and more productive use of time. Examples are given of companies that saw improvements such as reducing design time by over 75% and reducing data preparation work from one week to 15 minutes through the use of JMP analytics.
실험 설계의 강화 - 단순함에서 정교함으로 (Empowering Experimental Designs: From Simplicity to Sophistication)
Ryan Lekivetz (JMP), Elizabeth Claassen (JMP)
Discovery Summit Korea 2023
More Related Content
Similar to 3-1: Semiconductor Wafer Test Fail Map Clustering and Data Mining using JMP Add-In
TCO19 Japan Introduction to Marathon Matchestomerun
This document discusses marathon matches on Topcoder. It describes two types of marathons - fun marathons involving combinatorial optimization problems and sponsored marathons involving real-world problems with cash prizes. An example marathon called PopulationMapping is provided, involving mapping population data on a 2D grid with the goal of finding sparsely populated areas using few queries. Typical activities for competitors during a marathon are outlined, including reading problems, developing initial solutions, improving solutions over multiple days, and optimizing solutions on the final day.
Journey of Migrating Millions of Queries on The Cloudtakezoe
This document discusses challenges in upgrading a query engine and summarizing strategies for efficiently simulating queries to test compatibility and performance. It proposes grouping queries by signature and narrowing data scans to reduce the number of queries tested. It also recommends automating result verification by generating human-readable reports and excluding uncheckable queries. Assistance tools are proposed to aid investigation of differences, which helped discover real bugs in the target version.
The document discusses theories and principles for increasing productivity and throughput (velocity). It presents queuing theory and the theory of constraints, explaining how to reduce utilization, batch size, and item size to improve flow based on queuing theory. It then provides 21 experiments teams can try based on these theories, such as reducing work in progress limits, splitting up work into smaller batches and items, and improving the identified constraint area.
Queuing Theory and the Theory of Constraints are two powerful theories that can increase your velocity. This session explains both theories in simple terms then covers how they can be applied in the real world by agile teams. 21 simple velocity increasing experiments are described that you can immediately use.
The effects of Queuing Theory impact our lives on a daily basis. Scrum uses Queuing Theory at its core and you can amplify those effects.
The Theory of Constraints can identify the one constraint that is preventing your team from increasing its velocity. It also shows us how to remove that constraint in the cheapest way possible.
Presented at Scrum Australia (@auscrum), Melbourne, 29 April 2016
1) The document describes a data mining hackathon that aimed to build predictive models to increase the subscription rate of Motley Fool visitors using demographic and weblog data.
2) The top models achieved a ROC score of 0.737 using boosted regression trees on weblog data alone, and 0.738 by also including demographic variables.
3) Key findings were that weblog data was more predictive than sparse demographic data, and that important predictor variables were pages viewed, location, and income.
The document discusses the history and features of garbage collection (GC) in Ruby. It notes that while GC has gotten a bad reputation for being slow or causing errors, it has actually been improved over time thanks to contributions from pioneering computer scientists. The document urges readers to see GC as an opportunity to strengthen their skills rather than a scapegoat for performance issues.
This document discusses managing uncertainty in value-based software engineering. It presents two goal functions - "ENERGY", which aims to reduce effort, defects, and schedule, and "Huang06", which balances being first to market against bugs. It also describes experiments that use automated search techniques to sample across the space of project options and model calibrations without explicit calibration, finding this controls estimates as well as explicit calibration.
These slides provide an overview of the basics of design of experiments. They also describe and give examples of categorical and continuous factors and responses, discrete numeric and mixture variables, and blocking factors. The slides were presented live and in recorded videos as part of the Mastering JMP webcast series. Watch the webcasts at http://www.jmp.com/mastering
An overview of gradient descent optimization algorithms.pdfvudinhphuong96
This document provides an overview of gradient descent optimization algorithms. It discusses various gradient descent variants including batch gradient descent, stochastic gradient descent (SGD), and mini-batch gradient descent. It describes the trade-offs between these methods in terms of accuracy, time, and memory usage. The document also covers challenges with mini-batch gradient descent like choosing a proper learning rate. It then discusses commonly used optimization algorithms to address these challenges, including momentum, Nesterov accelerated gradient, Adagrad, Adadelta, RMSprop, and Adam. It provides visualizations to explain how momentum and Nesterov accelerated gradient work to help accelerate SGD.
Improving throughput with the Theory of Constraints and Queuing TheoryAndrew Rusling
Practical advice on how to improve the throughput of your agile team, by using the Theory of Constraints and Queuing Theory. Shows how to apply TOC to your task board. Explains how Queuing Theory is built into Scrum and Kanban, powering you to make the most of them.
The document discusses the future of Ruby's bigdecimal library and number system. It outlines some current problems with BigDecimal including its use of global modes, lack of automatic precision tracking, and limitations on instance generation. It suggests improvements could be made to calculation speeds by implementing more advanced algorithms. Additionally, a new class is needed to represent irrational numbers as computable algorithms rather than decimal approximations.
This document summarizes research on SSD failures in a large data center over six years. It finds that SSD failure is strongly correlated with drive age, with most failures occurring in "young" drives before 1,500 program-erase cycles. Machine learning models are able to accurately predict SSD failures, with random forests performing best. The top predictive feature is drive age. While error rates increase before failures, no single error metric reliably indicates impending failure. Most failed drives remain in a failed state for months before being repaired.
This document introduces khmer, a platform for scalable sequence analysis. It discusses how khmer uses k-mers to provide implicit read alignments and assemble sequences using de Bruijn graphs. It also describes some of the challenges with k-mers, such as each sequencing error resulting in novel k-mers. The document outlines khmer's data structures and algorithms for efficiently counting k-mers and represents de Bruijn graphs. It discusses how khmer has been applied to real biological problems and highlights areas of current research using khmer, such as error correction, variant calling, and assembly-free comparisons of data sets.
Queuing Theory and the Theory of Constraints are two powerful theories that can increase your velocity. This session explains both theories in simple terms then covers how they can be applied in the real world by agile teams. 21 simple velocity increasing experiments are described that you can immediately use.
The effects of Queuing Theory impact our lives on a daily basis. Scrum uses Queuing Theory at its core and you can amplify those effects.
The Theory of Constraints can identify the one constraint that is preventing your team from increasing its velocity. It also shows us how to remove that constraint in the cheapest way possible.
Vladimir Primakov - Qa management in big agile teamsIevgenii Katsan
- Using a straightforward release pipeline with separate teams focused on new features or bug fixes to avoid context switching and overlapping work.
- Conducting cross-team planning and reviews to identify dependencies, risks, and adjust testing scope and approach accordingly.
- Establishing common processes, tools, and data across teams through alignment and documentation to facilitate collaboration.
- Ensuring infrastructure like test environments are automated, stable, and similar to production to support efficient testing across large teams.
The document discusses the anatomy of cascading failures in distributed systems. It describes common triggering conditions that can cause failures like planned changes, traffic fluctuations, resource starvation and crashes. It then explains how initial failures can cascade through load redistribution, retry amplification, latency creep and resource contention during recovery. Finally, it provides strategies for improving system resilience like robust architecture, chaos engineering, retrying policies, throttling, circuit breaking, fallbacks and choosing the right tools.
In every development process there is the question, do we invest enough on quality? Do we need to invest more? Every team knows about the dilemma of how many tests is the right amount of tests we should write. Is 80% test coverage is good enough? Maybe 90%? 100%? Should we invest more time in unit testing? Are we wasting too much time on unit-testing? Should we invest time on a faster rollback mechanism?
WIIFM
“Without data, you’re just another person with an opinion” - W. Edwards Deming
SLO Driven Development is a framework that helps the developers focus on impact and balance of every aspect of the dev process. When working currently with SLI, SLA, SLO and error budget you can learn where to invest in the development process.
Let’s talk about the importance of good SLOs and how they can help us improve our day2day
Speech of Alexey Vasiliev, Software Engineer at Railsware, at Ruby Meditation #25 Kyiv 08.12.2018
Next conference - http://www.rubymeditation.com/
In this talk, Alexey will tell about the project in which was necessary to implement A/B testing and what came out of it in result
Announcements and conference materials https://www.fb.me/RubyMeditation
News https://twitter.com/RubyMeditation
Photos https://www.instagram.com/RubyMeditation
The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation
Tuhin Mitra: How I Automate My Negative TestsAnna Royzman
This document discusses negative testing strategies for automating tests. It begins by defining negative testing as testing invalid or unexpected inputs to show how a system does not work. The document then provides examples of how writing negative tests can improve quality, coverage, stability and make applications more robust. Specific strategies are outlined, such as discussing negative acceptance criteria with users, separating tests into functional groups, and using feature flags to enable and disable features for different test types. The conclusion emphasizes changing to a more positive mindset about negative testing to achieve better outcomes.
Designing and Running Performance ExperimentsJ On The Beach
Load testing is a continuous process that involves designing realistic load tests based on real user models and data, running load tests at increasing user loads to explore the load curve, and analyzing the results in the context of production metrics to understand performance and detect saturation points. The goal is to load test applications with a purpose to maintain and improve performance, which has a significant impact on business metrics like revenue.
Similar to 3-1: Semiconductor Wafer Test Fail Map Clustering and Data Mining using JMP Add-In (20)
The document discusses improving engineering efficiency through the use of analytics. It describes how JMP software helps individuals, teams, and organizations increase engineering efficiency by enabling faster problem solving, proactive process improvement, and more productive use of time. Examples are given of companies that saw improvements such as reducing design time by over 75% and reducing data preparation work from one week to 15 minutes through the use of JMP analytics.
실험 설계의 강화 - 단순함에서 정교함으로 (Empowering Experimental Designs: From Simplicity to Sophistication)
Ryan Lekivetz (JMP), Elizabeth Claassen (JMP)
Discovery Summit Korea 2023
The document discusses JMP and Python for statistical analysis and data science. It provides a SWOT analysis comparing the strengths, weaknesses, opportunities, and threats of each. While JMP has strengths in its user-friendly GUI and built-in statistical tools, Python has advantages in its open source nature, scalability, and versatility. The document argues that both JMP and Python are needed - JMP for common statistical analyses and visualization, and Python for tasks like data preprocessing, machine learning, and automation. It proposes training to help users integrate the two platforms.
Hi-JMP is a platform developed by SK hynix to improve data analysis efficiency and enable sharing of analysis content. It provides an app store, development console, and home interface to allow users to search, deploy, customize, and run JMP applications without installation. The platform uses dynamic UI, no-installation deployment, and a backend API to integrate various data sources and automate repetitive analysis tasks. A demo showed the app store, development console, and home features in action.
The document discusses using negative space in figure drawing to help draw the figure more easily. It then discusses using negative design space exploration in early stage naval ship design rather than positive design. This allows removing constraints to identify uncertainties and define feasible design ranges earlier to support decision making. Dynamic visualization tools are proposed to allow quicker exploration of design space, generation of more design alternatives, and obtaining insights from sensitivity analysis earlier in the design process.
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...University of Maribor
Slides from talk presenting:
Aleš Zamuda: Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapter and Networking.
Presentation at IcETRAN 2024 session:
"Inter-Society Networking Panel GRSS/MTT-S/CIS
Panel Session: Promoting Connection and Cooperation"
IEEE Slovenia GRSS
IEEE Serbia and Montenegro MTT-S
IEEE Slovenia CIS
11TH INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONIC AND COMPUTING ENGINEERING
3-6 June 2024, Niš, Serbia
International Conference on NLP, Artificial Intelligence, Machine Learning an...gerogepatton
International Conference on NLP, Artificial Intelligence, Machine Learning and Applications (NLAIM 2024) offers a premier global platform for exchanging insights and findings in the theory, methodology, and applications of NLP, Artificial Intelligence, Machine Learning, and their applications. The conference seeks substantial contributions across all key domains of NLP, Artificial Intelligence, Machine Learning, and their practical applications, aiming to foster both theoretical advancements and real-world implementations. With a focus on facilitating collaboration between researchers and practitioners from academia and industry, the conference serves as a nexus for sharing the latest developments in the field.
Harnessing WebAssembly for Real-time Stateless Streaming PipelinesChristina Lin
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMSIJNSA Journal
The smart irrigation system represents an innovative approach to optimize water usage in agricultural and landscaping practices. The integration of cutting-edge technologies, including sensors, actuators, and data analysis, empowers this system to provide accurate monitoring and control of irrigation processes by leveraging real-time environmental conditions. The main objective of a smart irrigation system is to optimize water efficiency, minimize expenses, and foster the adoption of sustainable water management methods. This paper conducts a systematic risk assessment by exploring the key components/assets and their functionalities in the smart irrigation system. The crucial role of sensors in gathering data on soil moisture, weather patterns, and plant well-being is emphasized in this system. These sensors enable intelligent decision-making in irrigation scheduling and water distribution, leading to enhanced water efficiency and sustainable water management practices. Actuators enable automated control of irrigation devices, ensuring precise and targeted water delivery to plants. Additionally, the paper addresses the potential threat and vulnerabilities associated with smart irrigation systems. It discusses limitations of the system, such as power constraints and computational capabilities, and calculates the potential security risks. The paper suggests possible risk treatment methods for effective secure system operation. In conclusion, the paper emphasizes the significant benefits of implementing smart irrigation systems, including improved water conservation, increased crop yield, and reduced environmental impact. Additionally, based on the security analysis conducted, the paper recommends the implementation of countermeasures and security approaches to address vulnerabilities and ensure the integrity and reliability of the system. By incorporating these measures, smart irrigation technology can revolutionize water management practices in agriculture, promoting sustainability, resource efficiency, and safeguarding against potential security threats.
Embedded machine learning-based road conditions and driving behavior monitoringIJECEIAES
Car accident rates have increased in recent years, resulting in losses in human lives, properties, and other financial costs. An embedded machine learning-based system is developed to address this critical issue. The system can monitor road conditions, detect driving patterns, and identify aggressive driving behaviors. The system is based on neural networks trained on a comprehensive dataset of driving events, driving styles, and road conditions. The system effectively detects potential risks and helps mitigate the frequency and impact of accidents. The primary goal is to ensure the safety of drivers and vehicles. Collecting data involved gathering information on three key road events: normal street and normal drive, speed bumps, circular yellow speed bumps, and three aggressive driving actions: sudden start, sudden stop, and sudden entry. The gathered data is processed and analyzed using a machine learning system designed for limited power and memory devices. The developed system resulted in 91.9% accuracy, 93.6% precision, and 92% recall. The achieved inference time on an Arduino Nano 33 BLE Sense with a 32-bit CPU running at 64 MHz is 34 ms and requires 2.6 kB peak RAM and 139.9 kB program flash memory, making it suitable for resource-constrained embedded systems.
Batteries -Introduction – Types of Batteries – discharging and charging of battery - characteristics of battery –battery rating- various tests on battery- – Primary battery: silver button cell- Secondary battery :Ni-Cd battery-modern battery: lithium ion battery-maintenance of batteries-choices of batteries for electric vehicle applications.
Fuel Cells: Introduction- importance and classification of fuel cells - description, principle, components, applications of fuel cells: H2-O2 fuel cell, alkaline fuel cell, molten carbonate fuel cell and direct methanol fuel cells.
A review on techniques and modelling methodologies used for checking electrom...nooriasukmaningtyas
The proper function of the integrated circuit (IC) in an inhibiting electromagnetic environment has always been a serious concern throughout the decades of revolution in the world of electronics, from disjunct devices to today’s integrated circuit technology, where billions of transistors are combined on a single chip. The automotive industry and smart vehicles in particular, are confronting design issues such as being prone to electromagnetic interference (EMI). Electronic control devices calculate incorrect outputs because of EMI and sensors give misleading values which can prove fatal in case of automotives. In this paper, the authors have non exhaustively tried to review research work concerned with the investigation of EMI in ICs and prediction of this EMI using various modelling methodologies and measurement setups.
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressionsVictor Morales
K8sGPT is a tool that analyzes and diagnoses Kubernetes clusters. This presentation was used to share the requirements and dependencies to deploy K8sGPT in a local environment.
Introduction- e - waste – definition - sources of e-waste– hazardous substances in e-waste - effects of e-waste on environment and human health- need for e-waste management– e-waste handling rules - waste minimization techniques for managing e-waste – recycling of e-waste - disposal treatment methods of e- waste – mechanism of extraction of precious metal from leaching solution-global Scenario of E-waste – E-waste in India- case studies.
3. ✓ Semiconductor Wafer Test Failure Map Clustering and
Data Mining using JMP Add-In
1. Semiconductor Process
2. Good, Bad Grouping in conventional way
3. User Friendly Interface
4. Data Mining Process & Result
5. Example
6. Summary
4. Over 99% of the defects in wafer test are within fabrication
Wafer Test
Failure
5. Good, Bad grouping in Convention way (1)
Failure
Rate
(%)
Wafer Test Time
>10.0%
↑BAD
↓GOOD
<1.00%
✓ Final Data Mining
→ Good Gr. vs Bad Gr.
✓ Problem
→ Garbage in, Garbage out
Various Maps @Bad Gr.
Blue = Fail
Orange = Pass
This is the type
of wafer map
engineers want
to analyze.
* Goal
Find the real cause
✓ .
✓ .
✓ .
✓ .
✓ .
but,
Garbage in, Garbage out
There are various types
of failure map including
the map we are finding
→ Garbage in
6. Good, Bad grouping in Convention way (2)
A large loss of time = Inefficient
✓ Advantage
: an accurate classification
✓ Disadvantage
: Time Consumption
✓ Engineer finds and classifies map.
We suggest the solutions with this
problems in this presentation.
7. User Friendly Interface
* Clustering
-. sda symbol
-. EPM
-. RPM
-. WT Item (Fail Bit Count)
*sda = statistical defect analysis
8. User Friendly Interface
Set period
Select Fab.
Select product
Select defect
Select Failure
to analyze
Clustering
Mining
* Clustering
-. sda symbol **
-. EPM
-. RPM **
-. WT Item (Fail Bit Count)
**sda = statistical defect analysis
**RPM = Reliability Parameter Monitoring
* Clustering selection → Set period → Select Fab. → Select product → Cluster Analysis Click!! → Select defect category → Select Failure Maps to analyze
Bad
Good Good Exclude Exclude
Exclude Exclude
9. User Friendly Interface
Bad
Good Good Exclude Exclude
Exclude Exclude
Bad Gr.
Lot & Wafer List ✓ Good, Bad Portion
Lot and Wafer
Lot and Wafer
✓ Clustering Group Portion
Clustering
Group
Portion
Good,
Bad
Portion
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
Fail
Rate
✓ Good, Bad group fail rate comparison
10. Classification using Hierarchical Clustering
Set period
Select Fab.
Select product
Select defect
Select Fail
to analyze
Good Good Exclude Exclude Exclude
Exclude
Exclude Bad
Bad
Bad
Bad
Bad Bad Bad
Good
✓ Clustering Group
: Engineers can select certain Area of a Wafer and
classify that area as a good group or bad group
Good Gr. Bad Gr.
VS
✓ Select Conditions ✓ Final Mining Group
* Good, Bad and Exclude Multiple choices possible
✓ Designating the good or bad groups
directly by looking at the clustered map
group
✓ Mining with final Good, Bad Groups
✓ Classifying failure maps in accurate and
fast.
➔ Dramatically improve engineer time loss
11. ✓ Mining (Inline Legend)
Select Oper. Select Legend.
Data Mining Process Interface
* Select Inline → Select Start Data, End Data → Select Oper. → Select Legend. → Click “연계분석”
12. Data Mining Result Interface
Rank Summary by
Algorithm Mining Result
Cause Process
Ranking
Various mining algorithms
OOOO Step Legend
OOOO Step Legend
OOOO Step Legend
* Algorithms
-. Decision Tree
-. Randomforest
-. ANOVA
13. ✓ Mining (EPM)
Data Mining Process Interface
aaa para.
aaa
para.
* Select EPM → Select Start Data, End Data → Click “연계분석”
14. Clustering & Data Mining Example (1)
✓ Select Conditions
Good Good Exclude Bad Bad
Bad
✓ Clustering Result ✓ Mining Result
Cause Process Rank Summary by Algorithm
OOOO Step Legend
OOOO Step Legend
OOOO Step Legend
OOOO Step Legend
OOOO Step Legend
OOOO Step Legend
A
EQ.
B
EQ.
C
EQ.
D
EQ.
D
EQ.
C
EQ.
B
EQ.
A
EQ.
OOOO Step Process Run Time
✓ Result
Click!!
Focused on specific EQ (D EQ) and process run time.
→ These could be most likely the causes.
15. Clustering & Data Mining Example (2)
✓ Mining Result (Data Visualization : Parallel Plot)
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
OOO
Step
By
Process
Step
Legend
Good group scattered.
Bad group scattered.
The path of Bad group overlapped with one equipment. →This could be most likely the cause.
a EQ
b EQ
c EQ
d EQ
* Association Analysis is also possible.
Data Visualization Analysis
16. Clustering & Data Mining Example (3)
○ Delayed step process detection between Good and Bad group using multivariate.
Dual axis occurrence
Good
Bad
Shift
Bad Gr.
Good Gr.
A Step
Process run time
B Step
Process run time
C Step
Process run time
D Step
Process run time
Process
Flow
Process
Flow
Good
Gr.
Bad
Gr.
+1 Day +1 Day +1 Day
+1 Day +8 Day +1 Day
↙Cause of defect
A Step
Process run time
B Step
Process run time
C Step
Process run time
D Step
Process run time
A
Step
Process
run
time
B Step
Process run time
A
Step
Process
run
time
C Step
Process run time
D Step
Process run time
A
Step
Process
run
time
A
Step
Process
run
time
A
Step
Process
run
time
B Step
Process run time
B Step
Process run time
Process run time
Process
run
time
○ Delay step detection between Good and Bad group using multivariate.
If there is a problem with
delay in B step
Data Visualization Analysis
17. WF Count Human [min] AI [min]
100 6 0.03
1000 60 0.16
3000 180 5
Clustering Process Time [ Human vs AI ]
Human
[min]
AI
[min]
AI
180min
5min
Human
Wafer Count
✓ Dramatically reduces time consumption for engineer
18. Summary
1. Classifying failure maps in faster and more accurate way
2. User Friendly Interface.
3. Reducing engineer’s time spends on repetitive work tasks dramatically
4. Anyone can use JMP Add-In.