Evaluating long-term predictive power of standard reliability growth models o...RAKESH RANA
Evaluating long-term predictive power of standard reliability growth models on automotive systems
Presented at:
24th annual International Symposium on Software Reliability Engineering (ISSRE-2013), Nov-2013, Pasadena, USA
Get full text of publication at:
http://rakeshrana.website/index.php/work/publications/
Crossing the Analytics Chasm and Getting the Models You Developed DeployedRobert Grossman
There are two cultures in data science and analytics - those that develop analytic models and those that deploy analytic models into operational systems. In this talk, we review the life cycle of analytic models and provide an overview of some of the approaches that have been developed for managing analytic models and workflows and for deploying them, including using analytic engines and analytic containers . We give a quick overview of languages for analytic models (PMML) and analytic workflows (PFA). We also describe the emerging discipline of AnalyticOps that has borrowed some of the techniques of DevOps.
Evaluating SRGMs for Automotive Software ProjectRAKESH RANA
Evaluation of standard reliability growth models in the context of automotive software systems
Presented at:
PROFES conferences, the 14th International Conference of Product Focused Software Development and Process Improvement, in Paphos, Cyprus, 12-14 June 2013.
Updated slides for my talk at the CHAQ meeting in Antwerp. I also added slides on some of my experiences on performing empirical studies with open source and industrial software systems.
Keynote presentation from ECBS conference. The talk is about how to use machine learning and AI in improving software engineering. Experiences from our project in Software Center (www.software-center.se).
In this paper we present a quality model for spreadsheets, based on the ISO/IEC 9126 standard that defines a generic quality model for software. To each of the software characteristics defined in the ISO/IEC 9126, we associate an equivalent spreadsheet characteristic. Then, we propose a set of spreadsheet specific metrics to assess the quality of a spreadsheet in each of the defined characteristics. In order to obtain the normal distribution of expected values for a spreadsheet in each of the metrics that we propose, we have executed them against all spreadsheets in the large and widely used EUSES spreadsheet corpus. Then, we quantify each characteristic of our quality model after computing the values of our metrics, and we define quality scores for the different ranges of values. Finally, to automate the atribution of a quality score to a given spreadsheet, according to our quality model, we have integrated the computation of the metrics it includes in both a batch and a web-based tool.
Descriptionsordernametypeformatvallabvarlab1location_idint%8.0gNumeric identifier of export source location2partner_idint%8.0gNumeric identifier of export destination location3yearint%8.0gYear4location_codestr3%9sThe source of the export goods (NZL) 3 character code.5location_name_short_enstr11%11sDescriptive name of country/region6partner_codestr3%9sThe destination of the export goods 3 character code7partner_name_short_enstr44%44sDescriptive name of country/region8hs_product_codestr2%9sProduct code identifier9export_valuedouble%10.0g(sum) export_value10comlang_offbyte%8.0g1 for common official of primary language11comlang_ethnobyte%8.0g1 if a language is spoken by at least 9% of the population in both countries12colonybyte%8.0g1 for pairs ever in colonial relationship13distfloat%9.0gsimple distance (most populated cities, km)14distcapfloat%9.0gsimple distance between capitals (capitals, km)15distwdouble%9.0gweighted distance (pop-wt, km)16pop_odouble%9.0gPopulation, total in mn17pop_ddouble%9.0gPopulation, total in mn18gdp_odouble%9.0gGDP (current US$)19gdp_ddouble%9.0gGDP (current US$)20gdpcap_ofloat%9.0gGDP per cap (current US$)21gdpcap_dfloat%9.0gGDP per cap (current US$)22area_olong%12.0gArea in sq. kms23area_dlong%12.0gArea in sq. kms24comcurbyte%8.0g1=Common currency25comreligfloat%9.0g1=Common religion26gatt_obyte%8.0g1=Origin is GATT/WTO member27gatt_dbyte%8.0g1=Destination is GATT/WTO member28pta_bbbyte%9.0g1=Non-reciprocal PTA ; 2=PTA (Source: Baier & Bergstrand, 2009)29fta_wtobyte%8.0g1=RTA (Source: WTO, 2015)30fta_bbbyte%8.0g1=FTA;2=Cust. Union;3=Common Market;4=Economic union (Baier & Bergstrand, 2009)31eu_dbyte%8.0g1=Destination is a EU member32product_idint%8.0gProduct id code33hs_product_name_short_enstr48%48sEnglish description of product category
SEES0095
Advanced Quantitative
Methods
2019/2020
Project Description and Q & A
Svetlana Makarova room 525 (16 Taviton)
s.makaro[email protected]
2 SEES0095 AQM Project Description and Q & A
Brief Extraction from the
SEESG46 Advanced Quantitative Methods Course Description
The course will be assessed through 100% coursework assignment. The assignment will be in the
form of a project. For your project are you encouraged to use data related to emerging markets
and transition economies – for example, on Demographic and Health indicators, voting patterns
and determinants, regional GDP, Foreign Direct Investment, Growth, inflation, labour markets
etc – and carry out an appropriate statistical analysis. Your results should be written up as a
research project incorporating an introduction, a contextual or literature background, a
methodology, data description and interpretation, results and conclusions. The practical
exercises throughout the course will prepare you for this assignment. You will be expected to
use Stata as the only computational tool. No other statistical packages will be allowed. The
assignment should be no more than 5,000 ...
Evaluating long-term predictive power of standard reliability growth models o...RAKESH RANA
Evaluating long-term predictive power of standard reliability growth models on automotive systems
Presented at:
24th annual International Symposium on Software Reliability Engineering (ISSRE-2013), Nov-2013, Pasadena, USA
Get full text of publication at:
http://rakeshrana.website/index.php/work/publications/
Crossing the Analytics Chasm and Getting the Models You Developed DeployedRobert Grossman
There are two cultures in data science and analytics - those that develop analytic models and those that deploy analytic models into operational systems. In this talk, we review the life cycle of analytic models and provide an overview of some of the approaches that have been developed for managing analytic models and workflows and for deploying them, including using analytic engines and analytic containers . We give a quick overview of languages for analytic models (PMML) and analytic workflows (PFA). We also describe the emerging discipline of AnalyticOps that has borrowed some of the techniques of DevOps.
Evaluating SRGMs for Automotive Software ProjectRAKESH RANA
Evaluation of standard reliability growth models in the context of automotive software systems
Presented at:
PROFES conferences, the 14th International Conference of Product Focused Software Development and Process Improvement, in Paphos, Cyprus, 12-14 June 2013.
Updated slides for my talk at the CHAQ meeting in Antwerp. I also added slides on some of my experiences on performing empirical studies with open source and industrial software systems.
Keynote presentation from ECBS conference. The talk is about how to use machine learning and AI in improving software engineering. Experiences from our project in Software Center (www.software-center.se).
In this paper we present a quality model for spreadsheets, based on the ISO/IEC 9126 standard that defines a generic quality model for software. To each of the software characteristics defined in the ISO/IEC 9126, we associate an equivalent spreadsheet characteristic. Then, we propose a set of spreadsheet specific metrics to assess the quality of a spreadsheet in each of the defined characteristics. In order to obtain the normal distribution of expected values for a spreadsheet in each of the metrics that we propose, we have executed them against all spreadsheets in the large and widely used EUSES spreadsheet corpus. Then, we quantify each characteristic of our quality model after computing the values of our metrics, and we define quality scores for the different ranges of values. Finally, to automate the atribution of a quality score to a given spreadsheet, according to our quality model, we have integrated the computation of the metrics it includes in both a batch and a web-based tool.
Descriptionsordernametypeformatvallabvarlab1location_idint%8.0gNumeric identifier of export source location2partner_idint%8.0gNumeric identifier of export destination location3yearint%8.0gYear4location_codestr3%9sThe source of the export goods (NZL) 3 character code.5location_name_short_enstr11%11sDescriptive name of country/region6partner_codestr3%9sThe destination of the export goods 3 character code7partner_name_short_enstr44%44sDescriptive name of country/region8hs_product_codestr2%9sProduct code identifier9export_valuedouble%10.0g(sum) export_value10comlang_offbyte%8.0g1 for common official of primary language11comlang_ethnobyte%8.0g1 if a language is spoken by at least 9% of the population in both countries12colonybyte%8.0g1 for pairs ever in colonial relationship13distfloat%9.0gsimple distance (most populated cities, km)14distcapfloat%9.0gsimple distance between capitals (capitals, km)15distwdouble%9.0gweighted distance (pop-wt, km)16pop_odouble%9.0gPopulation, total in mn17pop_ddouble%9.0gPopulation, total in mn18gdp_odouble%9.0gGDP (current US$)19gdp_ddouble%9.0gGDP (current US$)20gdpcap_ofloat%9.0gGDP per cap (current US$)21gdpcap_dfloat%9.0gGDP per cap (current US$)22area_olong%12.0gArea in sq. kms23area_dlong%12.0gArea in sq. kms24comcurbyte%8.0g1=Common currency25comreligfloat%9.0g1=Common religion26gatt_obyte%8.0g1=Origin is GATT/WTO member27gatt_dbyte%8.0g1=Destination is GATT/WTO member28pta_bbbyte%9.0g1=Non-reciprocal PTA ; 2=PTA (Source: Baier & Bergstrand, 2009)29fta_wtobyte%8.0g1=RTA (Source: WTO, 2015)30fta_bbbyte%8.0g1=FTA;2=Cust. Union;3=Common Market;4=Economic union (Baier & Bergstrand, 2009)31eu_dbyte%8.0g1=Destination is a EU member32product_idint%8.0gProduct id code33hs_product_name_short_enstr48%48sEnglish description of product category
SEES0095
Advanced Quantitative
Methods
2019/2020
Project Description and Q & A
Svetlana Makarova room 525 (16 Taviton)
s.makaro[email protected]
2 SEES0095 AQM Project Description and Q & A
Brief Extraction from the
SEESG46 Advanced Quantitative Methods Course Description
The course will be assessed through 100% coursework assignment. The assignment will be in the
form of a project. For your project are you encouraged to use data related to emerging markets
and transition economies – for example, on Demographic and Health indicators, voting patterns
and determinants, regional GDP, Foreign Direct Investment, Growth, inflation, labour markets
etc – and carry out an appropriate statistical analysis. Your results should be written up as a
research project incorporating an introduction, a contextual or literature background, a
methodology, data description and interpretation, results and conclusions. The practical
exercises throughout the course will prepare you for this assignment. You will be expected to
use Stata as the only computational tool. No other statistical packages will be allowed. The
assignment should be no more than 5,000 ...
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
This presentation includes basic of PCOS their pathology and treatment and also Ayurveda correlation of PCOS and Ayurvedic line of treatment mentioned in classics.
Thinking of getting a dog? Be aware that breeds like Pit Bulls, Rottweilers, and German Shepherds can be loyal and dangerous. Proper training and socialization are crucial to preventing aggressive behaviors. Ensure safety by understanding their needs and always supervising interactions. Stay safe, and enjoy your furry friends!
The simplified electron and muon model, Oscillating Spacetime: The Foundation...RitikBhardwaj56
Discover the Simplified Electron and Muon Model: A New Wave-Based Approach to Understanding Particles delves into a groundbreaking theory that presents electrons and muons as rotating soliton waves within oscillating spacetime. Geared towards students, researchers, and science buffs, this book breaks down complex ideas into simple explanations. It covers topics such as electron waves, temporal dynamics, and the implications of this model on particle physics. With clear illustrations and easy-to-follow explanations, readers will gain a new outlook on the universe's fundamental nature.
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
This slide is special for master students (MIBS & MIFB) in UUM. Also useful for readers who are interested in the topic of contemporary Islamic banking.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
Normal Labour/ Stages of Labour/ Mechanism of LabourWasim Ak
Normal labor is also termed spontaneous labor, defined as the natural physiological process through which the fetus, placenta, and membranes are expelled from the uterus through the birth canal at term (37 to 42 weeks
Safalta Digital marketing institute in Noida, provide complete applications that encompass a huge range of virtual advertising and marketing additives, which includes search engine optimization, virtual communication advertising, pay-per-click on marketing, content material advertising, internet analytics, and greater. These university courses are designed for students who possess a comprehensive understanding of virtual marketing strategies and attributes.Safalta Digital Marketing Institute in Noida is a first choice for young individuals or students who are looking to start their careers in the field of digital advertising. The institute gives specialized courses designed and certification.
for beginners, providing thorough training in areas such as SEO, digital communication marketing, and PPC training in Noida. After finishing the program, students receive the certifications recognised by top different universitie, setting a strong foundation for a successful career in digital marketing.
An Empirical Study of Reliability Growth of Open versus Closed Source Software through Software Reliability Growth Models
1. An Empirical Study of Reliability Growth of Open versus
Closed Source Software through Software Reliability
Growth Models
Najeeb Ullah , Maurizio Morisio
POLITECNICO DI TORINO
1
3. Motivation
• Open Source Software (OSS) have attracted
significant attention in recent years
• Uses of Open Source Software (OSS) are increasing
and many commercial products use OSS in various
fields
• “Forrester” empirical research highlighted that many
European software companies have clear strategy for
OSS adoption but fear and questions on adoption
• Quality and hence reliability of OSS is under focus
3
4. Background
• In literature software reliability growth models
(SRGM) have been widely used for reliability
characterization of closed source software (CSS)
• Many studies available about applicability of SRGMs
for reliability characterization of OSS with unclear
results
• Studied reliability growth of OSS versus CSS through
SRGMs
4
5. Software reliability growth model: SRGMS
• Reliability:
– Probability of failure free operation of software for specified
period of time in specified environment (IEEE Glossary)
• Reliability Model:
– Mathematical expression that specifies the general form of
the software failure process as a function of factors such as
fault introduction, fault removal, and the operational
environment. (IEEE Glossary)
– Divided into many types
• SRGM are classified into Concave and S-Shaped
Concave Shaped Model S-Shaped Mode
6. Goals And Research Questions
• Study and compare reliability growth of OSS with
CSS projects
• Determine a good model in terms of fitting and
prediction for OSS and CSS projects
– RQ1: Do SRGM models fit well the failure occurrence
pattern of OSS and CSS projects?
– RQ2: Which SRGM models are good predictors for OSS
and CSS?
6
7. Methodology and Models Evaluation Metrics (RQ1)
• Non Linear Regression used for Models fitting.
• Goodness of Fit (i.e. R2 ):
– indicates how successful the fit is.
– R2 takes a value between 0 and 1, inclusive.
– The closer the R2 value is to one, the better the fit.
– Rank models for each data set on R2 value
– Selected Threshold (i.e. Categorization) : R2 > 0.90
7
8. Methodology and Models Evaluation Metrics (RQ2)
• Models are fitted to two-third data points of each data
set and predict the remaining.
• Theil’s Statistics (i.e. TS: Accuracy)
– Average deviation percentage over all data points.
– The closer TS is to zero, the better the prediction accuracy of the
model.
– Rank models for each data set on TS value.
– Selected Threshold (i.e. Categorization) : TS < 10%
• Predictive Relative Error (i.e. PRE: Correctness)
– Ratio between the error difference (actual versus predicted) and
the predicted number of defects
– Rank models for each data set on PRE
– Selected threshold (i.e. Categorization) : 10% of total no of
actual defects
8
9. Selected SRGM Models
• Eight Models have been selected for this study due to
their prevalence among SRM.
Model Type
Musa Okumoto Concave
Inflection S-Shaped S-Shaped
Goel Okumoto Concave
Delayed S-Shaped S-Shaped
Logistic S-Shaped
Yamada Exponential Concave
Gompertz S-Shaped
Generalized Goel Concave
9
10. Data Collection (Industrial datasets)
• Searched papers on IEEE Explorer, ACM Digital
Library and in four journals, i.e. Journal of Information
and Software Technology, the Journal of System and
Software, IEEE software and IEEE Transaction on
Reliability.
• Strings used:
– Software failure rate, Software failure intensity, Software
failure Data sets, Software failure rate and
Reliability, Software failure intensity and Reliability.
• 2000 Papers, Relevant 14 papers, contained failure
data sets on 28 projects (22 from CSS, 6 from OSS) .
10
11. Data Collection (OSS datasets)
11
Projects Number of releases
C++ Standard Library 8
JUDDI 4
GNOME 3
APACHE 3
• Defects data downloaded from Apache repositories
using JIRA
13. Models Ranking Results (RQ1) on R2
CSS DS OSS DS
Model Fitted DS No of DS having
Highest R2 Value
Fitted DS No of DS having
Highest R2 Value
Musa 22/22 5/22 18/18 0/18
Inflection 22/22 4/22 18/18 5/18
Goel 21/22 0/22 18/18 0/18
Delayed 20/22 2/22 16/18 1/18
Logistic 20/22 3/22 15/18 1/18
Yamada 18/22 0/22 18/18 0/18
Gompertz 14/22 4/22 17/18 8/18
Generalized 10/22 4/22 16/18 3/18
13
14. Models Prediction Results (RQ2) On TS & PRE
Box Plots of Prediction Correctness
(PRE) values
Box Plots of Prediction Accuracy
(TS) values
15. Models Ranking Results (RQ2)
CSS DS OSS DS
Model No of DS
having lowest
TS Value
No of DS having
Lowest PRE
Value
No of DS
having lowest
TS Value
No of DS having
Lowest PRE
Value
Musa 10/22 10/22 4/18 2/18
Inflection 2/22 4/22 3/18 3/18
Goel 3/22 3/22 1/18 3/18
Delayed 2/22 2/22 4/18 3/18
Logistic 4/22 2/22 5/18 3/18
Yamada 2/22 3/22 1/18 3/18
Gompertz 3/22 2/22 6/18 6/18
Generalized 1/22 1/22 3/18 2/18
15
16. Threat to Validity
• No hypothesis test in methodology used for
answering RQs
• The choice of thresholds is not grounded in the
literature.
• Provide ranking of the models to identify the best
model for each type of dataset and metric.
16
17. Conclusions
• Results
– All selected SRGM fit to defect data of the OSS projects in
the same manner as that of CSS. OSS reliability grows
similarly to that of CSS.
– Musa Okumoto and Inflection S-Shaped perform well for
CSS, while for OSS Inflection S-Shaped and Gompertz are
good performers.
– SRGM can be used for reliability characterization of OSS
projects
• Observations
– Best models in OSS are different from the best in industrial
datasets
17
18. Questions ?
An Empirical Study of Reliability Growth of Open versus
Closed Source Software through Software Reliability
Growth Models
Najeeb Ullah , Maurizio Morisio
name.surname@polito.it
18