Delta debugging is a technique for isolating the cause of failures in programs. It works by systematically removing parts of changes made between a version where the program worked and a version where it fails. The algorithm efficiently determines the minimal set of changes responsible for the failure. The technique was able to isolate a single change causing a failure in GDB from 178,000 changed lines within a few hours. Automating debugging through techniques like delta debugging can help developers quickly find and fix bugs.
The document discusses various approaches to debugging software systems in a structured manner. It outlines steps like understanding the system architecture and components, using compiler and linker options for instrumentation, designing test cases to uncover bugs, following debugging rules like making failures reproducible and isolating changes. The document also mentions tools that can help with debugging like static checkers, code instrumentation tools, and maintaining a set of regression tests.
This document provides an overview of debugging techniques and best practices. It discusses what debugging is, common debugging rules and methods, as well as tools and techniques for preventing bugs. Key points covered include understanding the system, making failures reproducible, dividing problems, changing one thing at a time, and keeping an audit trail. The document also mentions code reviews, assertions, and defensive programming as ways to prevent bugs.
The document discusses various software testing techniques including black box testing, white box testing, and grey box testing. It provides details on specific techniques such as equivalence partitioning, boundary value analysis, statement coverage, condition coverage, function coverage, and cyclomatic complexity. The objective is to understand these techniques so they can be used effectively to test applications and find defects.
The document discusses debugging processes. It begins by defining debugging as finding and correcting software errors uncovered during testing. The debugging process involves executing test cases, assessing results for discrepancies between expected and actual performance, and attempting to find the cause through iterative testing and corrections. There are generally three debugging approaches: brute force using extensive logging, backtracking by tracing code backwards from symptoms, and cause elimination using binary partitioning to isolate potential error causes.
This document provides an overview of software testing concepts and definitions. It discusses key topics such as software quality, testing methods like static and dynamic testing, testing levels from unit to acceptance testing, and testing types including functional, non-functional, regression and security testing. The document is intended as an introduction to software testing principles and terminology.
Language-Based Testing and Debugging
Andreas Zeller, CISPA Helmholtz Center for Information Security
Joint work with Dominic Steinhöfel and the CISPA team
If one knows the input language of a software system, one can use it for generating test inputs or for determining input patterns that cause specific behavior. But how does one specify an input language such that all properties of inputs are covered? In this talk, I introduce the ISLa specification language, combining grammars and constraints over nonterminals to capture even the most complex input languages. ISLa serves as a fuzzer, allowing to test programs with a large variety of inputs systematically, but also as a parser, decomposing inputs and outputs into individual elements. This opens up new directions for debugging, determining the exact conditions under which a program fails: "The program fails if the <check-temperature-option> is given and <temperature> < 0 holds." Includes live demos!
Andreas Zeller is faculty at the CISPA Helmholtz Center for Information Security and professor for Software Engineering at Saarland University. His research on automated debugging, mining software archives, specification mining, and security testing has proven highly influential. Zeller is one of the few researchers to have received two ERC Advanced Grants, most recently for his S3 project. Zeller is an ACM Fellow and holds an ACM SIGSOFT Outstanding Research Award.
As a young aspiring scientist, social media is one of the outlets to disseminate your work and connect to the community. This talk gives hints on the benefits and risks of science on social media. Talk at the ICSE 2022 New Faculty Symposium.
Andreas Zeller's keynote at the 1st Intl Fuzzing workshop 2022 at NDSS: https://fuzzingworkshop.github.io/program.html
Do you fuzz your own program, or do you fuzz someone else's program? The answer to this question has vast consequences on your view on fuzzing. Fuzzing someone else's program is the typical adverse "security tester" perspective, where you want your fuzzer to be as automatic and versatile as possible. Fuzzing your own code, however, is more like a traditional tester perspective, where you may assume some knowledge about the program and its context, but may also want to _exploit_ this knowledge - say, to direct the fuzzer to critical locations.
In this talk, I detail these differences in perspectives and assumptions, and highlight their consequences for fuzzer design and research. I also highlight cultural differences in the research communities, and what happens if you submit a paper to the wrong community. I close with an outlook into our newest frameworks, set to reconcile these perspectives by giving users unprecedented control over fuzzing, yet staying fully automatic if need be.
Bio: Andreas Zeller is faculty at the CISPA Helmholtz Center for Information Security and a professor for Software Engineering at Saarland University, both in Saarbrücken, Germany. His research on automated debugging, mining software archives, specification mining, and security testing has won several awards for its impact in academia and industry. Zeller is an ACM Fellow, an IFIP Fellow, an ERC Advanced Grant Awardee, and holds an ACM SIGSOFT Outstanding Research Award.
The document discusses various approaches to debugging software systems in a structured manner. It outlines steps like understanding the system architecture and components, using compiler and linker options for instrumentation, designing test cases to uncover bugs, following debugging rules like making failures reproducible and isolating changes. The document also mentions tools that can help with debugging like static checkers, code instrumentation tools, and maintaining a set of regression tests.
This document provides an overview of debugging techniques and best practices. It discusses what debugging is, common debugging rules and methods, as well as tools and techniques for preventing bugs. Key points covered include understanding the system, making failures reproducible, dividing problems, changing one thing at a time, and keeping an audit trail. The document also mentions code reviews, assertions, and defensive programming as ways to prevent bugs.
The document discusses various software testing techniques including black box testing, white box testing, and grey box testing. It provides details on specific techniques such as equivalence partitioning, boundary value analysis, statement coverage, condition coverage, function coverage, and cyclomatic complexity. The objective is to understand these techniques so they can be used effectively to test applications and find defects.
The document discusses debugging processes. It begins by defining debugging as finding and correcting software errors uncovered during testing. The debugging process involves executing test cases, assessing results for discrepancies between expected and actual performance, and attempting to find the cause through iterative testing and corrections. There are generally three debugging approaches: brute force using extensive logging, backtracking by tracing code backwards from symptoms, and cause elimination using binary partitioning to isolate potential error causes.
This document provides an overview of software testing concepts and definitions. It discusses key topics such as software quality, testing methods like static and dynamic testing, testing levels from unit to acceptance testing, and testing types including functional, non-functional, regression and security testing. The document is intended as an introduction to software testing principles and terminology.
Language-Based Testing and Debugging
Andreas Zeller, CISPA Helmholtz Center for Information Security
Joint work with Dominic Steinhöfel and the CISPA team
If one knows the input language of a software system, one can use it for generating test inputs or for determining input patterns that cause specific behavior. But how does one specify an input language such that all properties of inputs are covered? In this talk, I introduce the ISLa specification language, combining grammars and constraints over nonterminals to capture even the most complex input languages. ISLa serves as a fuzzer, allowing to test programs with a large variety of inputs systematically, but also as a parser, decomposing inputs and outputs into individual elements. This opens up new directions for debugging, determining the exact conditions under which a program fails: "The program fails if the <check-temperature-option> is given and <temperature> < 0 holds." Includes live demos!
Andreas Zeller is faculty at the CISPA Helmholtz Center for Information Security and professor for Software Engineering at Saarland University. His research on automated debugging, mining software archives, specification mining, and security testing has proven highly influential. Zeller is one of the few researchers to have received two ERC Advanced Grants, most recently for his S3 project. Zeller is an ACM Fellow and holds an ACM SIGSOFT Outstanding Research Award.
As a young aspiring scientist, social media is one of the outlets to disseminate your work and connect to the community. This talk gives hints on the benefits and risks of science on social media. Talk at the ICSE 2022 New Faculty Symposium.
Andreas Zeller's keynote at the 1st Intl Fuzzing workshop 2022 at NDSS: https://fuzzingworkshop.github.io/program.html
Do you fuzz your own program, or do you fuzz someone else's program? The answer to this question has vast consequences on your view on fuzzing. Fuzzing someone else's program is the typical adverse "security tester" perspective, where you want your fuzzer to be as automatic and versatile as possible. Fuzzing your own code, however, is more like a traditional tester perspective, where you may assume some knowledge about the program and its context, but may also want to _exploit_ this knowledge - say, to direct the fuzzer to critical locations.
In this talk, I detail these differences in perspectives and assumptions, and highlight their consequences for fuzzer design and research. I also highlight cultural differences in the research communities, and what happens if you submit a paper to the wrong community. I close with an outlook into our newest frameworks, set to reconcile these perspectives by giving users unprecedented control over fuzzing, yet staying fully automatic if need be.
Bio: Andreas Zeller is faculty at the CISPA Helmholtz Center for Information Security and a professor for Software Engineering at Saarland University, both in Saarbrücken, Germany. His research on automated debugging, mining software archives, specification mining, and security testing has won several awards for its impact in academia and industry. Zeller is an ACM Fellow, an IFIP Fellow, an ERC Advanced Grant Awardee, and holds an ACM SIGSOFT Outstanding Research Award.
Illustrated Code: Building Software in a Literate Way
Andreas Zeller, CISPA Helmholtz Center for Information Security
Notebooks – rich, interactive documents that join together code, documentation, and outputs – are all the rage with data scientists. But can they be used for actual software development? In this talk, I share experiences from authoring two interactive textbooks – fuzzingbook.org and debuggingbook.org – and show how notebooks not only serve for exploring and explaining code and data, but also how they can be used as software modules, integrating self-checking documentation, tests, and tutorials all in one place. The resulting software focuses on the essential, is well-documented, highly maintainable, easily extensible, and has a much higher shelf life than the "duct tape and wire” prototypes frequently found in research and beyond.
At my talk "On Impact in Software Engineering Research", I present a number of lessons from (and for!) high-impact research:
* Work on a real problem
* Assume as little as possible
* Keep things simple
* Have a sound model
* Keep on learning
* Keep on moving
* Build prototypes
Video at https://youtu.be/md4Fp3Pro0o
Andreas Zeller is faculty at the CISPA Helmholtz Center for Information Security and professor for Software Engineering at Saarland University, both in Saarbrücken, Germany. His research on automated debugging, mining software archives, specification mining, and security testing has won several awards for its impact in academia and industry. Zeller is an ACM Fellow, an IFIP Fellow, an ERC Advanced Grant Awardee, and holds an ACM SIGSOFT Outstanding Research Award.
How do you make your research impactful? How do you get towards your book, your tool, your algorithm? Andreas Zeller shares lessons learned in his career.
Software-Tests automatisch erzeugen: Frische Ansätze für Forschung, Praxis und Lehre
Andreas Zeller, CISPA Helmholtz-Zentrum für Informationssicherheit, Saarbrücken
Automatisch erzeugte Softwaretests können mit wenig menschlichem Aufwand viele Fehler finden. In diesem Vortrag stelle ich aktuelle Techniken zur Testerzeugung vor, die für ein gegebenes Programm vollautomatisch dessen Eingabesprache ableiten und aus den so entstehenden Grammatiken große Mengen gültiger Testeingaben ableiten. Unser grammatikbasierter LangFuzz-Testgenerator hat so in den JavaScript-Interpretern von Firefox, Chrome und Edge tausende Fehler gefunden. Die Techniken sind in dem interaktiven Buch “Generating Software Tests” (www.fuzzingbook.org) zusammengefasst. In einer Mischung von Text und Programmcode können Leser im Browser direkt mit den Programmen experimentieren und ihren Code live ergänzen und erweitern.
Andreas Zeller ist Forscher am CISPA Helmholtz-Zentrum für Informationssicherheit und Professor für Softwaretechnik an der Universität des Saarlandes, beide in Saarbrücken. Seine Forschung beschäftigt sich mit der Analyse großer Software-Systeme und ihre Entwicklungsgeschichte. In 2010 wurde Zeller zum Fellow der ACM ernannt für seine Beiträge zur automatischen Fehlersuche und der Analyse von Software-Archiven, für die er 2018 als erster Deutscher den Outstanding Research Award der ACM SIGSOFT erhielt. Seine aktuellen Projekte kombinieren Testerzeugung und Specification Mining, gefördert durch den Europäischen Forschungsrat (ERC).
- Andreas Zeller gave a talk on impact in software engineering research.
- He discussed different perspectives on what constitutes impactful research, including intellectual challenge, elegance and reusability, usefulness, and innovation.
- Zeller described his own path to impact, starting with his PhD work on configuration management using feature logic, through developing the DDD debugger and introducing delta debugging. His work then expanded to mining software repositories and more recently mining app descriptions and behaviors.
What does it take to have high impact in software engineering research? Andreas Zeller, a "high impact" SE researcher, shares his personal story and perspective.
The document provides 12 tips for preparing a successful grant proposal for the European Research Council (ERC). The tips include understanding the ERC process and guidelines; starting the proposal process many months before the deadline; reserving several weeks solely for writing; getting feedback from multiple reviewers outside your specialty; leveraging local expertise in EU funding; emphasizing your unique achievements and qualifications; proposing a high-risk, high-reward project with a clear title and structure; getting straight to the point without jargon; polishing the proposal extensively; and recognizing that following the tips does not guarantee success but prevents misunderstandings. The overall aim is to craft a proposal that clearly communicates your message to reviewers in a manner that stands out against competing
You are a young researcher on your first independent position. What can you do to get your research work funded? How do you frame your work, find the right partners, address the funding body?
Slides from Andreas Zeller's presentation at the New Faculty Symposium at ICSE 2017, Buenos Aires, Argentina.
Models—abstract and simple descriptions of some artifact—are the backbone of all software engineering activities. While writing models is hard, existing code can serve as a source for abstract descriptions of how software behaves. To infer correct usage, code analysis needs usage examples, though; the more, the better.
We have built a lightweight parser that efficiently extracts API usage models from source code—models that can then be used to detect anomalies. Applied on the 200 mil- lion lines of code of the Gentoo Linux distribution, we would extract more than 15 million API constraints. On the web site checkmycode.org, anyone can check his/her code against the “wisdom of Linux”.
The document discusses using mutation testing to evaluate test quality by seeding programs with faults and seeing if tests can find them. It outlines challenges with efficiency and determining equivalent mutants. It proposes using dynamic invariants learned through execution to characterize the impact of mutations and identify equivalent mutants, aiming to make mutation testing more practical.
I need to change some piece of code. What do I need to consider? Is there anything else I need to change? We show what to learn from software repositories.
Jeder Programmierer kennt die Situation: Ein Programm läuft nicht so, wie es soll. Ich stelle Techniken vor, die automatisch
(a) die Ursachen eines Fehlverhaltens finden - indem wir genau die Aspekte isolieren, die das Zustandekommen eines Fehlers verursachen;
(b) Programmfehler finden - indem wir aus dem Code "normale" Anweisungsfolgen lernen und nach Abweichungen suchen; und
(c) vorhersagen, wo in Zukunft Fehler auftreten werden - indem wir maschinell lernen, welche Code- und Prozesseigenschaften bisher mit Fehlern korrelierten.
Fallstudien an echten Programmen mit echten Fehlern, von AspectJ über Firefox zu Windows demonstrieren die Praxistauglichkeit der vorgestellten Verfahren.
Andreas Zeller ist Professor für Softwaretechnik an der Universität des Saarlandes in Saarbrücken. Sein Forschungsgebiet ist die Analyse großer Software-Systeme und deren Fehler. Sein Buch "Why Programs Fail - A Guide to Systematic Debugging" wurde 2006 mit dem Jolt Software Development Productivity Award ausgezeichnet.
Vortrag der GI Regionalgruppe Rhein-Neckar 2008-06-19
The document discusses various myths and misconceptions in software engineering. It summarizes research that analyzed large codebases to determine factors correlated with defects. Metrics like lines of code and complexity metrics correlated with defects, but the relationship depended on the project. Defect-prone modules could be predicted by combining metrics. Other factors like developer experience, prior defects, language features, and dependencies also influenced bugs. While some properties were project-specific, others like the role of requirements, design, coding and testing were universal defect sources.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
More Related Content
More from CISPA Helmholtz Center for Information Security
Illustrated Code: Building Software in a Literate Way
Andreas Zeller, CISPA Helmholtz Center for Information Security
Notebooks – rich, interactive documents that join together code, documentation, and outputs – are all the rage with data scientists. But can they be used for actual software development? In this talk, I share experiences from authoring two interactive textbooks – fuzzingbook.org and debuggingbook.org – and show how notebooks not only serve for exploring and explaining code and data, but also how they can be used as software modules, integrating self-checking documentation, tests, and tutorials all in one place. The resulting software focuses on the essential, is well-documented, highly maintainable, easily extensible, and has a much higher shelf life than the "duct tape and wire” prototypes frequently found in research and beyond.
At my talk "On Impact in Software Engineering Research", I present a number of lessons from (and for!) high-impact research:
* Work on a real problem
* Assume as little as possible
* Keep things simple
* Have a sound model
* Keep on learning
* Keep on moving
* Build prototypes
Video at https://youtu.be/md4Fp3Pro0o
Andreas Zeller is faculty at the CISPA Helmholtz Center for Information Security and professor for Software Engineering at Saarland University, both in Saarbrücken, Germany. His research on automated debugging, mining software archives, specification mining, and security testing has won several awards for its impact in academia and industry. Zeller is an ACM Fellow, an IFIP Fellow, an ERC Advanced Grant Awardee, and holds an ACM SIGSOFT Outstanding Research Award.
How do you make your research impactful? How do you get towards your book, your tool, your algorithm? Andreas Zeller shares lessons learned in his career.
Software-Tests automatisch erzeugen: Frische Ansätze für Forschung, Praxis und Lehre
Andreas Zeller, CISPA Helmholtz-Zentrum für Informationssicherheit, Saarbrücken
Automatisch erzeugte Softwaretests können mit wenig menschlichem Aufwand viele Fehler finden. In diesem Vortrag stelle ich aktuelle Techniken zur Testerzeugung vor, die für ein gegebenes Programm vollautomatisch dessen Eingabesprache ableiten und aus den so entstehenden Grammatiken große Mengen gültiger Testeingaben ableiten. Unser grammatikbasierter LangFuzz-Testgenerator hat so in den JavaScript-Interpretern von Firefox, Chrome und Edge tausende Fehler gefunden. Die Techniken sind in dem interaktiven Buch “Generating Software Tests” (www.fuzzingbook.org) zusammengefasst. In einer Mischung von Text und Programmcode können Leser im Browser direkt mit den Programmen experimentieren und ihren Code live ergänzen und erweitern.
Andreas Zeller ist Forscher am CISPA Helmholtz-Zentrum für Informationssicherheit und Professor für Softwaretechnik an der Universität des Saarlandes, beide in Saarbrücken. Seine Forschung beschäftigt sich mit der Analyse großer Software-Systeme und ihre Entwicklungsgeschichte. In 2010 wurde Zeller zum Fellow der ACM ernannt für seine Beiträge zur automatischen Fehlersuche und der Analyse von Software-Archiven, für die er 2018 als erster Deutscher den Outstanding Research Award der ACM SIGSOFT erhielt. Seine aktuellen Projekte kombinieren Testerzeugung und Specification Mining, gefördert durch den Europäischen Forschungsrat (ERC).
- Andreas Zeller gave a talk on impact in software engineering research.
- He discussed different perspectives on what constitutes impactful research, including intellectual challenge, elegance and reusability, usefulness, and innovation.
- Zeller described his own path to impact, starting with his PhD work on configuration management using feature logic, through developing the DDD debugger and introducing delta debugging. His work then expanded to mining software repositories and more recently mining app descriptions and behaviors.
What does it take to have high impact in software engineering research? Andreas Zeller, a "high impact" SE researcher, shares his personal story and perspective.
The document provides 12 tips for preparing a successful grant proposal for the European Research Council (ERC). The tips include understanding the ERC process and guidelines; starting the proposal process many months before the deadline; reserving several weeks solely for writing; getting feedback from multiple reviewers outside your specialty; leveraging local expertise in EU funding; emphasizing your unique achievements and qualifications; proposing a high-risk, high-reward project with a clear title and structure; getting straight to the point without jargon; polishing the proposal extensively; and recognizing that following the tips does not guarantee success but prevents misunderstandings. The overall aim is to craft a proposal that clearly communicates your message to reviewers in a manner that stands out against competing
You are a young researcher on your first independent position. What can you do to get your research work funded? How do you frame your work, find the right partners, address the funding body?
Slides from Andreas Zeller's presentation at the New Faculty Symposium at ICSE 2017, Buenos Aires, Argentina.
Models—abstract and simple descriptions of some artifact—are the backbone of all software engineering activities. While writing models is hard, existing code can serve as a source for abstract descriptions of how software behaves. To infer correct usage, code analysis needs usage examples, though; the more, the better.
We have built a lightweight parser that efficiently extracts API usage models from source code—models that can then be used to detect anomalies. Applied on the 200 mil- lion lines of code of the Gentoo Linux distribution, we would extract more than 15 million API constraints. On the web site checkmycode.org, anyone can check his/her code against the “wisdom of Linux”.
The document discusses using mutation testing to evaluate test quality by seeding programs with faults and seeing if tests can find them. It outlines challenges with efficiency and determining equivalent mutants. It proposes using dynamic invariants learned through execution to characterize the impact of mutations and identify equivalent mutants, aiming to make mutation testing more practical.
I need to change some piece of code. What do I need to consider? Is there anything else I need to change? We show what to learn from software repositories.
Jeder Programmierer kennt die Situation: Ein Programm läuft nicht so, wie es soll. Ich stelle Techniken vor, die automatisch
(a) die Ursachen eines Fehlverhaltens finden - indem wir genau die Aspekte isolieren, die das Zustandekommen eines Fehlers verursachen;
(b) Programmfehler finden - indem wir aus dem Code "normale" Anweisungsfolgen lernen und nach Abweichungen suchen; und
(c) vorhersagen, wo in Zukunft Fehler auftreten werden - indem wir maschinell lernen, welche Code- und Prozesseigenschaften bisher mit Fehlern korrelierten.
Fallstudien an echten Programmen mit echten Fehlern, von AspectJ über Firefox zu Windows demonstrieren die Praxistauglichkeit der vorgestellten Verfahren.
Andreas Zeller ist Professor für Softwaretechnik an der Universität des Saarlandes in Saarbrücken. Sein Forschungsgebiet ist die Analyse großer Software-Systeme und deren Fehler. Sein Buch "Why Programs Fail - A Guide to Systematic Debugging" wurde 2006 mit dem Jolt Software Development Productivity Award ausgezeichnet.
Vortrag der GI Regionalgruppe Rhein-Neckar 2008-06-19
The document discusses various myths and misconceptions in software engineering. It summarizes research that analyzed large codebases to determine factors correlated with defects. Metrics like lines of code and complexity metrics correlated with defects, but the relationship depended on the project. Defect-prone modules could be predicted by combining metrics. Other factors like developer experience, prior defects, language features, and dependencies also influenced bugs. While some properties were project-specific, others like the role of requirements, design, coding and testing were universal defect sources.
More from CISPA Helmholtz Center for Information Security (16)
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
21. Vom Fachbereich f¨ r Mathematik und Informatik
u
der Technischen Universit¨ t Braunschweig
a
genehmigte Dissertation
zur Erlangung des Grades eines
Doktor-Ingenieurs (Dr.-Ing.)
Andreas Zeller
Configuration Management with Version Sets
A Unified Software Versioning Model
and its Applications
1. April 1997
1. Referent: Prof. Dr. Gregor Snelting
2. Referent: Prof. Dr. Walter F. Tichy
Eingereicht am: 1. November 1996
22.
23. Bug Reports
From: Brian Kahne <bkahne@ibmoto.com>
To: DDD Bug Report Address <bug-ddd@gnu.org>
Subject: Problem with DDD and GDB 4.17
When using DDD with GDB 4.16, the run command
correctly uses any prior command-line arguments, or
the value of "set args". However, when I switched to
GDB 4.17, this no longer worked: If I entered a run
command in the console window, the prior
command-line options would be lost. [...]
44. Bug Reports
From: Brian Kahne <bkahne@ibmoto.com>
To: DDD Bug Report Address <bug-ddd@gnu.org>
Subject: Problem with DDD and GDB 4.17
When using DDD with GDB 4.16, the run command
correctly uses any prior command-line arguments, or
the value of "set args". However, when I switched to
GDB 4.17, this no longer worked: If I entered a run
command in the console window, the prior
command-line options would be lost. [...]
55. Bisection
✔ ✔ ✔✘ ✘
Yesterday Today
Failure Cause
56. What was Changed
$ diff -r gdb-4.16 gdb-4.17
diff -r gdb-4.16/COPYING gdb-4.17/COPYING
5c5
< 675 Mass Ave, Cambridge, MA 02139, USA
---
> 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
282c282
< Appendix: How to Apply These Terms to Your New Programs
---
> How to Apply These Terms to Your New Programs
…and so on for 178,200 lines (8,721 locations)
93. Isolating Changes
Delta Debugging Log
100000
GDB with ddmin algorithm
10000 ... with dd algorithm
... plus scope information
Changes left
1000
100
10
1
0 50 100 150 200 250 300
Tests executed
• Result after 98 tests (= 1 hour)
94. The Failure Cause
diff -r gdb-4.16/gdb/infcmd.c gdb-4.17/gdb/infcmd.c
1239c1278
< "Set arguments to give program being debugged when it is
started.n
---
> "Set argument list to give program being debugged when
it is started.n
• Documentation becomes GDB output
• DDD expects Arguments,
but GDB outputs Argument list
95. Andreas Zeller · TU Braunschweig
Gestern lief mein Programm.
Heute nicht mehr. Warum?
Fehlersuche mit Delta Debugging
Andreas Zeller
Mittagsseminar, TU Braunschweig, 16. M¨rz 1998
a
0
96. Yesterday, my program worked.
Today, it does not. Why?
Andreas Zeller
Universit¨ t Passau
a
Lehrstuhl f¨ r Software-Systeme
u
Innstraße 33, D-94032 Passau, Germany
zeller@acm.org
Abstract. Imagine some program and a number of changes. If none of these
changes is applied (“yesterday”), the program works. If all changes are applied
(“today”), the program does not work. Which change is responsible for the fail-
ure? We present an efficient algorithm that determines the minimal set of failure-
inducing changes. Our delta debugging prototype tracked down a single failure-
inducing change from 178,000 changed GDB lines within a few hours.
1 A True Story
The GDB people have done it again. The new release 4.17 of the GNU debugger [6]
149. Contracts
set_hour (h: INTEGER) is
-- Set the hour from `h'
require
sane_h: 0 <= h and h <= 23
ensure
hour_set: hour = h
minute_unchanged: minutes = old minutes
second_unchanged: seconds = old seconds
160. Scienti c Method
Problem Report
Code
Hypothesis Prediction
Run
More Runs
161. Scienti c Method
Problem Report
Code
Hypothesis Prediction Experiment
Run
More Runs
162. Scienti c Method
Problem Report
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
More Runs
163. Scienti c Method
Problem Report Hypothesis is supported:
re ne hypothesis
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
More Runs
164. Scienti c Method
Problem Report Hypothesis is supported:
re ne hypothesis
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
Hypothesis is rejected:
More Runs create new hypothesis
165. Scienti c Method
Problem Report Hypothesis is supported:
re ne hypothesis
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
Hypothesis is rejected:
More Runs create new hypothesis
Diagnosis
169. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment
Observation
Conclusion
170. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment Observe a[0] at Line 37.
Observation
Conclusion
171. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment Observe a[0] at Line 37.
Observation a[0] = 0 holds as predicted.
Conclusion
172. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment Observe a[0] at Line 37.
Observation a[0] = 0 holds as predicted.
Conclusion Hypothesis is confirmed.
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
And this is me in 1997
I was not singing at all at the time &#x2013;&#xA0;I was doing my PhD&#x2026;
with this man, my advisor Gregor Snelting.
we would usually find a cool formalism first, and then look at applications.
Say: Girard&#x2019;s linear logic
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
we had slicing, algorithmic debugging &#x2013;&#xA0;that&#x2019;s it!
Sometimes life gives you a blank sheet of paper, and it&#x2019;s up to you to fill it.
In my case, it was: bugs bugs bugs
In my case, it was: bugs bugs bugs
In my case, it was: bugs bugs bugs
In my case, it was: bugs bugs bugs
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
But: bisection doesn&#x2019;t cut it!
But how do you get to this minimal difference? I tried quite a number of approaches, including genetic programming and search-based methods such as simulated annealing. Eventually, I had to come up with my own.
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
1998 talk
Pic of 1999 paper. Note that I submitted this 1 year after the first talk. That&#x2019;s because I spent all this time simplifying it.
That&#x2019;s because I spent one year making it as simple as possible, but no simpler. And this worked quite well.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
a few &#x201C;simple&#x201D; ideas have obtained great traction
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
Today, delta debugging is a core part of git, mercurial, and other version control tools. (which is how I contributed to version control after all)
entry on delta debugging, s/w architecture, more
entry on delta debugging, s/w architecture, more
So, that&#x2019;s what we have right now: tools tools tools
So, that&#x2019;s what we have right now: tools tools tools
So, that&#x2019;s what we have right now: tools tools tools
what do we still need to do?
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
We lack good benchmarks.
http://www.jjchandler.com/tombstone/download.php
We cannot tell bugs from features.
And by the way, this implies that interactive debuggers are the wrong tools.
Again, remember the long nights.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
So maybe it&#x2019;s not bugs bugs bugs
So maybe it&#x2019;s not bugs bugs bugs
So maybe it&#x2019;s not bugs bugs bugs
or tools tools tools
or tools tools tools
or tools tools tools
but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;
let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;
let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;