invited talk presented for the Distinguished Lecturer Series of the Department of Computer Science at the University of Illinois at Chicago, 10 April 2014
invited talk presented for the Distinguished Speaker Series of the Institute for Software Research (ISR) at the University of California, Irvine, 5 April 2013
invited talk presented for the Distinguished Speaker Series of the Institute for Software Research (ISR) at the University of California, Irvine, 5 April 2013
Automatically Discovering, Reporting and Reproducing Android Application CrashesKevin Moran
Mobile developers face unique challenges when detecting and reporting crashes in apps due to their prevailing GUI event-driven nature and additional sources of inputs (e.g., sensor readings). To support developers in these tasks, we introduce a novel, automated approach called CRASHSCOPE. This tool explores a given Android app using systematic input generation, according to several strategies informed by static and dynamic analyses, with the intrinsic goal of triggering crashes. When a crash is detected, CRASHSCOPE generates an augmented crash report containing screenshots, detailed crash reproduction steps, the captured exception stack trace, and a fully replayable script that automatically reproduces the crash on a target device(s).
We evaluated CRASHSCOPE’s effectiveness in discovering crashes as compared to five state-of-the-art Android input generation tools on 61 applications. The results demonstrate that CRASHSCOPE performs about as well as current tools for detecting crashes and provides more detailed fault information. Additionally, in a study analyzing eight real-world Android app crashes, we found that CRASHSCOPE’s reports are easily readable and allow for reliable reproduction of crashes by presenting more explicit information than human written reports.
Interpretability and Reproducibility in Production Machine Learning Applicat...Swaminathan Sundararaman
The past decade has seen tremendous growth in production deployments of machine learning algorithms across a range of applications such as targeted advertising, self driving cars, speech translation, medical diagnosis etc [1]. In these contexts, models make key decisions such as predicting the likelihood of a person committing a future crime, trustworthiness for a loan approval, medical diagnosis etc [2]. Presence of bias based on gender, geographical location, race etc., and their consequent negative impact, have been uncovered in several of these deployments [3], [4]. Industries and governments are reacting, enacting regulations requiring that decisions made by machine learning models be Interpretable/Explainable [5].
Explainability across the full range of ML and DL algorithms is an unsolved research problem, with many innovations over the last several years and entire conferences devoted to the topic. However, even simple explainability solutions that are considered established in development (training environments) run into additional difficulties when put into live production.
Our design pattern uses a well known technique for explainability - the Canary model (sometimes called Surrogate model) [6,7]. In this approach, a classically non-explainable technique, such as a Neural Network, is paired with an explainable model (that approximates the predictions of the non-explainable technique) such as a Decision Tree. As long as predictions match - the Canary model’s behavior can be used to provide a human understandable reasoning for the prediction.
Dependability Benchmarking by Injecting Software BugsRoberto Natella
Benchmarks are an established practice for performance evaluation in the computer industry since decades. Examples of successful benchmarking initiatives are the TPC (Transaction Processing Performance Council) and the SPEC (Standard Performance Evaluation Corporation). More recently, the research community developed the notion of dependability benchmarking, which evaluates the quality of service (throughput, availability, etc.) of competing products in the presence of faults, by using fault injection. The idea of dependability benchmarking has been applied in several domains including transaction processing, telecom, automotive, etc.
Given that software faults (bugs) are a major cause of failures, it becomes important to assess dependability against these faults. However, emulating software faults in a controlled fault injection experiment is a difficult problem, since bugs originate from human error. This presentation discusses about the open challenges and the recent advances in the field of emulating software bugs in a representative way.
AN ALTERNATIVE APPROACH FOR SELECTION OF PSEUDO RANDOM NUMBERS FOR ONLINE EXA...cscpconf
Fast and accurate selection of random pattern is needed for many scientific and commercial applications. One of the major applications is Online Examination system. In this paper, a sophisticated approach has been developed for the selection of uniform pseudo random pattern for Online Examination System. Three random integer generators have been compared for this
purpose. Most commonly used procedural language based pseudo random number; PHP random generator and atmospheric noise based true random number generator have been considered for easy generation of random patterns. The test result shows a varying degree of improvement in the quality of randomness of the generated patterns. The randomness quality of the generated pseudo random pattern has been assured by diehard test suite. An experimental
outcome for our recommended approach signifies that our approach selects a quality set of random pattern for Online Examination System
H2O.ai basic components and model deployment pipeline presented. Benchmark for scalability, speed and accuracy of machine learning libraries for classification presented from https://github.com/szilard/benchm-ml.
Time Series Anomaly Detection with .net and AzureMarco Parenzan
If you have any device or source that generates values over time (also a log from a service), you want to determine if in a time frame, the time serie is correct or you can detect some anomalies. What can you do as a developer (not a Data Scientist) with .NET o Azure? Let's see how in this session.
In this analysis, we establish an economic and mathematical (queueing theory) framework to calculate how much it would cost to build the dev/test lab of your dreams. One in which developers never wait for testing resources, and every commit is tested on replicas of the production environment. Then, we contrast several different options to get there and evaluate all of them from an economic perspective.
After the presentation, in order to illustrate that this is not only theory, I also gave a quick demonstration of how we at Ravello use our own technology to develop our own application. Each engineer can spin up as many instances of the production replica app as needed on demand for dev/test. We showed how we have integrated Ravello with Jenkins so that on every commit, we spin up the production replica application and run integration tests in parallel.
If you have any questions, feel free to reach out. We are more than happy to discuss how this may be relevant to your development process. www.ravellosystems.com
Known Unknowns: Testing in the Presence of Uncertainty (talk at ACM SIGSOFT F...David Rosenblum
talk presented in the Visions & Challenges Track of the ACM SIGSOFT 22nd International Symposium on the Foundations of Software Engineering (FSE 2014), Hong Kong, 20 November 2014; the paper won 2nd Prize in the track
Career Management (invited talk at ICSE 2014 NFRS)David Rosenblum
invited talk presented at the ICSE 2014 New Faculty and Researcher Symposium (NFRS), Hyderabad, India, 3 June 2014
More Related Content
Similar to Jogging While Driving, and Other Software Engineering Research Problems (invited talk for UIC Computer Science Distinguished Lecturer Series)
Automatically Discovering, Reporting and Reproducing Android Application CrashesKevin Moran
Mobile developers face unique challenges when detecting and reporting crashes in apps due to their prevailing GUI event-driven nature and additional sources of inputs (e.g., sensor readings). To support developers in these tasks, we introduce a novel, automated approach called CRASHSCOPE. This tool explores a given Android app using systematic input generation, according to several strategies informed by static and dynamic analyses, with the intrinsic goal of triggering crashes. When a crash is detected, CRASHSCOPE generates an augmented crash report containing screenshots, detailed crash reproduction steps, the captured exception stack trace, and a fully replayable script that automatically reproduces the crash on a target device(s).
We evaluated CRASHSCOPE’s effectiveness in discovering crashes as compared to five state-of-the-art Android input generation tools on 61 applications. The results demonstrate that CRASHSCOPE performs about as well as current tools for detecting crashes and provides more detailed fault information. Additionally, in a study analyzing eight real-world Android app crashes, we found that CRASHSCOPE’s reports are easily readable and allow for reliable reproduction of crashes by presenting more explicit information than human written reports.
Interpretability and Reproducibility in Production Machine Learning Applicat...Swaminathan Sundararaman
The past decade has seen tremendous growth in production deployments of machine learning algorithms across a range of applications such as targeted advertising, self driving cars, speech translation, medical diagnosis etc [1]. In these contexts, models make key decisions such as predicting the likelihood of a person committing a future crime, trustworthiness for a loan approval, medical diagnosis etc [2]. Presence of bias based on gender, geographical location, race etc., and their consequent negative impact, have been uncovered in several of these deployments [3], [4]. Industries and governments are reacting, enacting regulations requiring that decisions made by machine learning models be Interpretable/Explainable [5].
Explainability across the full range of ML and DL algorithms is an unsolved research problem, with many innovations over the last several years and entire conferences devoted to the topic. However, even simple explainability solutions that are considered established in development (training environments) run into additional difficulties when put into live production.
Our design pattern uses a well known technique for explainability - the Canary model (sometimes called Surrogate model) [6,7]. In this approach, a classically non-explainable technique, such as a Neural Network, is paired with an explainable model (that approximates the predictions of the non-explainable technique) such as a Decision Tree. As long as predictions match - the Canary model’s behavior can be used to provide a human understandable reasoning for the prediction.
Dependability Benchmarking by Injecting Software BugsRoberto Natella
Benchmarks are an established practice for performance evaluation in the computer industry since decades. Examples of successful benchmarking initiatives are the TPC (Transaction Processing Performance Council) and the SPEC (Standard Performance Evaluation Corporation). More recently, the research community developed the notion of dependability benchmarking, which evaluates the quality of service (throughput, availability, etc.) of competing products in the presence of faults, by using fault injection. The idea of dependability benchmarking has been applied in several domains including transaction processing, telecom, automotive, etc.
Given that software faults (bugs) are a major cause of failures, it becomes important to assess dependability against these faults. However, emulating software faults in a controlled fault injection experiment is a difficult problem, since bugs originate from human error. This presentation discusses about the open challenges and the recent advances in the field of emulating software bugs in a representative way.
AN ALTERNATIVE APPROACH FOR SELECTION OF PSEUDO RANDOM NUMBERS FOR ONLINE EXA...cscpconf
Fast and accurate selection of random pattern is needed for many scientific and commercial applications. One of the major applications is Online Examination system. In this paper, a sophisticated approach has been developed for the selection of uniform pseudo random pattern for Online Examination System. Three random integer generators have been compared for this
purpose. Most commonly used procedural language based pseudo random number; PHP random generator and atmospheric noise based true random number generator have been considered for easy generation of random patterns. The test result shows a varying degree of improvement in the quality of randomness of the generated patterns. The randomness quality of the generated pseudo random pattern has been assured by diehard test suite. An experimental
outcome for our recommended approach signifies that our approach selects a quality set of random pattern for Online Examination System
H2O.ai basic components and model deployment pipeline presented. Benchmark for scalability, speed and accuracy of machine learning libraries for classification presented from https://github.com/szilard/benchm-ml.
Time Series Anomaly Detection with .net and AzureMarco Parenzan
If you have any device or source that generates values over time (also a log from a service), you want to determine if in a time frame, the time serie is correct or you can detect some anomalies. What can you do as a developer (not a Data Scientist) with .NET o Azure? Let's see how in this session.
In this analysis, we establish an economic and mathematical (queueing theory) framework to calculate how much it would cost to build the dev/test lab of your dreams. One in which developers never wait for testing resources, and every commit is tested on replicas of the production environment. Then, we contrast several different options to get there and evaluate all of them from an economic perspective.
After the presentation, in order to illustrate that this is not only theory, I also gave a quick demonstration of how we at Ravello use our own technology to develop our own application. Each engineer can spin up as many instances of the production replica app as needed on demand for dev/test. We showed how we have integrated Ravello with Jenkins so that on every commit, we spin up the production replica application and run integration tests in parallel.
If you have any questions, feel free to reach out. We are more than happy to discuss how this may be relevant to your development process. www.ravellosystems.com
Similar to Jogging While Driving, and Other Software Engineering Research Problems (invited talk for UIC Computer Science Distinguished Lecturer Series) (20)
Known Unknowns: Testing in the Presence of Uncertainty (talk at ACM SIGSOFT F...David Rosenblum
talk presented in the Visions & Challenges Track of the ACM SIGSOFT 22nd International Symposium on the Foundations of Software Engineering (FSE 2014), Hong Kong, 20 November 2014; the paper won 2nd Prize in the track
SIGSOFT Impact Award: Reflections and Prospects (invited talk at SIGSOFT FSE ...David Rosenblum
Invited talk with Alexander L. Wolf upon receiving the first ACM SIGSOFT Impact Paper Award, at the 16th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (ACM SIGSOFT FSE), 13 November 2008.
Assertions a Decade Later (invited talk at ICSE 2002)David Rosenblum
Invited talk upon receiving the 2002 ICSE Most Influential Paper Award for ICSE 1992, at the 24th International Conference on Software Engineering (ICSE 2002), 22 May 2002.
Welocme to ViralQR, your best QR code generator.ViralQR
Welcome to ViralQR, your best QR code generator available on the market!
At ViralQR, we design static and dynamic QR codes. Our mission is to make business operations easier and customer engagement more powerful through the use of QR technology. Be it a small-scale business or a huge enterprise, our easy-to-use platform provides multiple choices that can be tailored according to your company's branding and marketing strategies.
Our Vision
We are here to make the process of creating QR codes easy and smooth, thus enhancing customer interaction and making business more fluid. We very strongly believe in the ability of QR codes to change the world for businesses in their interaction with customers and are set on making that technology accessible and usable far and wide.
Our Achievements
Ever since its inception, we have successfully served many clients by offering QR codes in their marketing, service delivery, and collection of feedback across various industries. Our platform has been recognized for its ease of use and amazing features, which helped a business to make QR codes.
Our Services
At ViralQR, here is a comprehensive suite of services that caters to your very needs:
Static QR Codes: Create free static QR codes. These QR codes are able to store significant information such as URLs, vCards, plain text, emails and SMS, Wi-Fi credentials, and Bitcoin addresses.
Dynamic QR codes: These also have all the advanced features but are subscription-based. They can directly link to PDF files, images, micro-landing pages, social accounts, review forms, business pages, and applications. In addition, they can be branded with CTAs, frames, patterns, colors, and logos to enhance your branding.
Pricing and Packages
Additionally, there is a 14-day free offer to ViralQR, which is an exceptional opportunity for new users to take a feel of this platform. One can easily subscribe from there and experience the full dynamic of using QR codes. The subscription plans are not only meant for business; they are priced very flexibly so that literally every business could afford to benefit from our service.
Why choose us?
ViralQR will provide services for marketing, advertising, catering, retail, and the like. The QR codes can be posted on fliers, packaging, merchandise, and banners, as well as to substitute for cash and cards in a restaurant or coffee shop. With QR codes integrated into your business, improve customer engagement and streamline operations.
Comprehensive Analytics
Subscribers of ViralQR receive detailed analytics and tracking tools in light of having a view of the core values of QR code performance. Our analytics dashboard shows aggregate views and unique views, as well as detailed information about each impression, including time, device, browser, and estimated location by city and country.
So, thank you for choosing ViralQR; we have an offer of nothing but the best in terms of QR code services to meet business diversity!
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
The Art of the Pitch: WordPress Relationships and Sales
Jogging While Driving, and Other Software Engineering Research Problems (invited talk for UIC Computer Science Distinguished Lecturer Series)
1. Jogging While Driving!
and Other Software Engineering
Research Problems
David S. Rosenblum!
Dean, School of Computing!
National University of Singapore
8. NUS
School of Computing
✓Ranked #1 in Asia, #9 in the world [QS World University Rankings by Subject]!
✓2 Departments: Computer Science and Information Systems!
✓111 Academic Staff (tenure-track & teaching track)!
✓115 Research Staff!
✓1800 Undergraduate Students!
✓180 Masters Students!
✓350 PhD Students!
✓S$25 million operating budget!
✓S$10 million+ in research income per annum
10. Certainty in
Software Engineering
Engineering of software is centered around
simplistic,“yes/no” characterizations of artifacts
Program is correct/incorrect
Program execution finished/crashed
Compilation completed/aborted
Test suite succeeded/failed
Specification is satisfied/violated
20. Adaptation in CAAAs
Physical Context
Sensed Context
Inferred Context
Presumed Context
Environment
Context!
Manager
Application
Adaptation!
Manager
Middleware
M. Sama, D.S. Rosenblum, Z.Wang and S. Elbaum,“Multi-Layer Faults in the Architectures of Mobile,
Context-Aware Adaptive Applications”, Journal of Systems and Software,Vol. 83, Issue 6, Jun. 2010, pp. 906–914.
21. Adaptation in CAAAs
Physical Context
Sensed Context
Inferred Context
Presumed Context
Environment
Context!
Manager
Application
Adaptation!
Manager
Middleware
Rule
Engine
M. Sama, D.S. Rosenblum, Z.Wang and S. Elbaum,“Multi-Layer Faults in the Architectures of Mobile,
Context-Aware Adaptive Applications”, Journal of Systems and Software,Vol. 83, Issue 6, Jun. 2010, pp. 906–914.
22. Adaptation in CAAAs
Physical Context
Sensed Context
Inferred Context
Presumed Context
Environment
Context!
Manager
Application
Adaptation!
Manager
Middleware
3rd-Party
Libraries
Rule
Engine
M. Sama, D.S. Rosenblum, Z.Wang and S. Elbaum,“Multi-Layer Faults in the Architectures of Mobile,
Context-Aware Adaptive Applications”, Journal of Systems and Software,Vol. 83, Issue 6, Jun. 2010, pp. 906–914.
23. Approach
1.Derive Adaptation Finite-State Machine
(A-FSM) from rule logic!
2.Explore state space of A-FSM to discover
all potential faults!
✓Enumerative algorithms!
✓Symbolic algorithms!
3.(Confirm existence of discovered faults)
M. Sama, S. Elbaum, F. Raimondi and D.S. Rosenblum,“Context-Aware Adaptive Applications: Fault Patterns and Their
Automated Identification”, IEEETransactions on Software Engineering,Vol. 36, No. 5, Sep./Oct. 2010, pp. 644-661.
29. PhoneAdapter A-FSM
checking location implies GPS is on!
locations are mutually exclusive!
speeds monotonically increase!
a meeting’s end time is later than its start time
Global constraints:
ActivateMeeting DeactivateMeeting
Office
Driving!
Fast
Meeting
Driving
Sync
General
Home
Outdoor
Jogging
40. PhoneAdapter Results
Hazards: Enumerative
n PhoneAdapter
aptation Races and Cycles Context Hazards
signments Race Cycle Paths Hold Activ. Prior.
3968 45 13 14085 0 11 3182
3968 135 23 161 0 0 52
3072 97 19 2 0 0 0
2560 36 13 16 2 2 4
3072 58 19 2 0 0 0
2816 76 19 104 8 0 13
2848 29 1 82634 1828 368 2164
2048 32 1 0 0 0 0
1024 27 5 2 2 0 0
ned a formal model of a key complex behavioral char-
eristic, namely adaptation, of an increasingly large and
Table 2: Faults
State Vars. Nondet. Adaptation Dead Pred
Assignments Faults Assignments
General 7 128 37 128
Outdoor 5 32 3 17
Jogging 2 4 0 1
Driving 3 8 0 7
DrivingFast 2 4 0 2
Home 4 16 0 9
O ce 7 128 1 65
Meeting 1 2 0 2
Sync 2 4 0 1
6.4 Detecting Context Hazards
This class of faults corresponds to sequences of asynchr
41. CAAAs
Summary
✓Rule-based CAAAs can be extremely fault-
prone, even with a small set of rules!
✓The model checking algorithms find many
actual faults, with different tradeoffs!
✓Some alternative to rule-based adaptation
may be preferable
46. P≥0.95 [ ]
Probabilistic
Model Checking
Model
Checker
✓
✕
State Machine!
Model
Temporal
Properties
Results
Counterexample!
Trace
System
Requirements
0.4
0.6
Probabilistic
Probabilistic
! ¬p → ◊q( )∧"( )
47. P=? [ ]
Probabilistic
Model Checking
Model
Checker
✓
✕
State Machine!
Model
Temporal
Properties
Results
Counterexample!
Trace
System
Requirements
0.4
0.6
Quantitative Results
0.9732Probabilistic
Probabilistic
! ¬p → ◊q( )∧"( )
48. Example
Die Tossing Simulated by Coin Flipping
Knuth-Yao algorithm,
from the PRISM group
(Kwiatkowska et al.)
0
3
2
1
6
4
5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
49. Example
Die Tossing Simulated by Coin Flipping
Knuth-Yao algorithm,
from the PRISM group
(Kwiatkowska et al.)
The behavior is governed by a!
theoretical probability distribution
0
3
2
1
6
4
5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
50. P≥0.95 [ ]
Probabilistic
Model Checking
Model
Checker
✓
State Machine!
Model
Temporal
Properties
Results
Counterexample!
Trace
System
Requirements
0.4
0.6
Quantitative Results
0.9732Probabilistic
Probabilistic
! ¬p → ◊q( )∧"( )
51. P≥0.95 [ ]
Probabilistic
Model Checking
Model
Checker
✓
State Machine!
Model
Temporal
Properties
Results
Counterexample!
Trace
System
Requirements
Quantitative Results
0.9732Probabilistic
Probabilistic
0.41
0.59
! ¬p → ◊q( )∧"( )
52. P≥0.95 [ ]
Probabilistic
Model Checking
Model
Checker
✕
State Machine!
Model
Temporal
Properties
Results
Counterexample!
Trace
System
Requirements
Quantitative Results
Probabilistic
Probabilistic
0.41
0.59
0.6211
! ¬p → ◊q( )∧"( )
53. Example!
Zeroconf Protocol
s1s0 s2 s3
q
1
1
{ok} {error}
{start} s4
s5
s6
s7
s8
1
1-q
1-p
1-p
1-p
1-p
p p p
p
1
from the PRISM group
(Kwiatkowska et al.)
54. Example!
Zeroconf Protocol
s1s0 s2 s3
q
1
1
{ok} {error}
{start} s4
s5
s6
s7
s8
1
1-q
1-p
1-p
1-p
1-p
p p p
p
1
The behavior is governed by an!
empirically estimated probability distribution
from the PRISM group
(Kwiatkowska et al.)
packet-loss rate
55. Perturbed Probabilistic
Systems
• Starting Points!
✓Discrete-Time Markov Chains (DTMCs)!
✓… with one or more probability parameters!
✓… verified against reachability properties:!
!
✓… and (more recently) LTL properties
S? ∪ S!
Guoxin Su and David S. Rosenblum,“Asymptotic Bounds for QuantitativeVerification of
Perturbed Probabilistic Systems”, Proc. ICFEM 2013!
!
Guoxin Su and David S. Rosenblum,“Perturbation Analysis of Stochastic Systems with
Empirical Distribution Parameters”, Proc. ICSE 2014
56. Parametric
Markov Chains
• A distribution parameter in a DTMC is represented as a
vector x of parameters xi!
• The norm of total variance represents the amount of
perturbation:!
!
• The parameter is allowed a “sufficiently small”
perturbation with respect to ideal reference values r:!
!
• Can generalize to multiple parameters
v = vi∑
x − r ≤ Δ
57. Perturbation Bounds
• Perturbation Function!
!
where A is the transition probability sub-matrix for S?
and b is the vector of one-step probabilities from S? to S!
!
• Condition Numbers: [ICFEM 2013]!
!
• Quadratic Bounds: [ICSE 2014]!
ρ x( )= ι? i A x
i
i b x( )− Ai
i b( )( )i=0
∞
∑
κ = lim
δ→0
sup
ρ(x − r)
δ
: x − r ≤ δ,δ > 0
⎧
⎨
⎩
⎫
⎬
⎭
f −
(δ )− inf ρ(x − r) + f +
(δ )− supρ(x − r) = o(δ 2
)
59. Additional Aspects
• Models
✓Markov Decision Processes (MDPs)!
✓Continuous-Time Markov Chains (CMTCs)!
• Verification
✓PCTL Model Checking!
with singularities due to nested P[ ] operators!
✓Reward Properties!
✓Alternative Norms and Bounds!
Kullback-Leibler Divergence!
✓Parameters as random variables
60. Other Forms of
Uncertainty
“There are known knowns; there are things we know
we know. We also know there are known unknowns;
that is to say, we know there are some things we do
not know. But there are also unknown unknowns –
the ones we don’t know we don’t know.”!
!
— Donald Rumsfeld
61. Uncertainty in Testing
1982: Elaine Weyuker: Non-Testable Programs!
- Impossible/too costly to efficiently check results!
- Example: mathematical software!
2010: David Garlan: Intrinsic Uncertainty!
- Systems embody intrinsic uncertainty/imprecision!
- Cannot easily distinguish bugs from “features”!
- Example: ubiquitous computing
69. Sources of
Uncertainty
✓Output: results, characteristics of results!
✓Sensors: redundancy, reliability, resolution!
✓Context: sensing, inferring, fusing!
✓Machine learning: imprecision, user-specificity
These create significant challenges for
software engineering research and practice!
70. Conclusion
✓Software engineering (certainly) suffers
from excessive certainty!
✓A probabilistic mindset offers some insight!
✓But significant challenges remain for
probabilistic verification!
✓And other forms of uncertainty remain a
challenge to address
71. Jogging While Driving!
and Other Software Engineering
Research Problems
David S. Rosenblum!
Dean, School of Computing!
National University of Singapore