2018 Distinguished Speaker, the UC Irvine Institute for Software Research (ISR) Distinguished Speaker Series 2018-2019. "Intelligent Software Engineering: Synergy between AI and Software Engineering" http://isr.uci.edu/content/isr-distinguished-speaker-series-2018-2019
MSRA 2018: Intelligent Software Engineering: Synergy between AI and Software ...Tao Xie
Invited Talk at the 2018 Computing in the 21st Century Conference & Asia Faculty Summit on MSRA’s 20th Anniversary https://www.microsoft.com/en-us/research/event/computing-in-the-21st-century-conference-asia-faculty-summit-on-msras-20th-anniversary/#!agenda
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...Tao Xie
2018 Keynote Speaker, Symposium on Dependable Software Engineering - Theories, Tools and Applications (SETTA 2018). "Intelligent Software Engineering: Synergy between AI and Software Engineering" http://confesta2018.csp.escience.cn/dct/page/65581
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...ACM Chicago
Join us as Tao Xie, Professor and Willett Faculty Scholar in the Department of Computer Science at the University of Illinois at Urbana-Champaign and ACM Distinguished Speaker, talks about Intelligent Software Engineering: Synergy between AI and Software Engineering. This is a joint meeting hosted by Chicago Chapter ACM / Loyola University Computer Science Department.
MSRA 2018: Intelligent Software Engineering: Synergy between AI and Software ...Tao Xie
Invited Talk at the 2018 Computing in the 21st Century Conference & Asia Faculty Summit on MSRA’s 20th Anniversary https://www.microsoft.com/en-us/research/event/computing-in-the-21st-century-conference-asia-faculty-summit-on-msras-20th-anniversary/#!agenda
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...Tao Xie
2018 Keynote Speaker, Symposium on Dependable Software Engineering - Theories, Tools and Applications (SETTA 2018). "Intelligent Software Engineering: Synergy between AI and Software Engineering" http://confesta2018.csp.escience.cn/dct/page/65581
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...ACM Chicago
Join us as Tao Xie, Professor and Willett Faculty Scholar in the Department of Computer Science at the University of Illinois at Urbana-Champaign and ACM Distinguished Speaker, talks about Intelligent Software Engineering: Synergy between AI and Software Engineering. This is a joint meeting hosted by Chicago Chapter ACM / Loyola University Computer Science Department.
Keynote presentation from ECBS conference. The talk is about how to use machine learning and AI in improving software engineering. Experiences from our project in Software Center (www.software-center.se).
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019Patrizio Pelliccione
ML and AI are increasingly dominating the high-tech industry. Organizations and technology companies are leveraging their big data to create new products or improve their processes to reach the next level in their market. However, ML and AI are not a silver bullet and Software 2.0 is not the end of software developers or software engineering.
In this talk I will argument on how software engineering can help ML and AI to become the key technology for (autonomous) systems of the near future. Software engineering best practices and achievements reached in the last decades might help, e.g., (i) democratising the use of ML/AI, (ii) composing, reusing, chaining ML/AI models to solve more complex problems, and (iii) supporting for reasoning about correctness, repeatability, explainability, traceability, fairness, ethics, while building an ML/AI pipeline.
The complexity of current software-based systems has led the software engineering community to look for inspiration in diverse related fields (e.g., robotics, artificial intelligence) as well as other areas (e.g., biology) to find new ways of designing and managing systems and services.
Publish or Perish: Questioning the Impact of Our Research on the Software Dev...Margaret-Anne Storey
A Video for this talk can be found here: https://www.youtube.com/watch?v=DvRdBb9TEUI
Abstract: How often do we pause to consider how we, as a community, decide which developer problems we address, or how well we are doing at evaluating our solutions within real development contexts? Many of our research contributions in software engineering can be considered as purely technical. Yet somewhere, at some time, a software developer may be impacted by our research. In this talk, I invite the community to question the impact of our research on software developer productivity. To guide the discussion, I first paint a picture of the modern-day developer and the challenges they experience. I then present 4+1 views of software engineering research --- views that concern research context, method choice, research paradigms, theoretical knowledge and real-world impact. I demonstrate how these views can be used to design, communicate and distinguish individual studies, but also how they can be used to compose a critical perspective of our research at a community level. To conclude, I propose structural changes to our collective research and publishing activities --- changes to provoke a more expeditious consideration of the many challenges facing today's software developer.
(Thanks to Brynn Hawker for slide design and proposed new badges. brynn@hawker.me)
In 2003 Dave et al. have coined the term “opinion mining” to refer to “processing a set of search results for a given item, generating a list of product attributes (quality, features, etc.) and aggregating opinions about each of them (poor, mixed, good)”. Nine years later, in 2012 Brooks and Swigger have applied sentiment analysis in the context of software engineering. Today another nine years have passed and it is time to look back: what have we achieved as a research community and where should we go next?
To answer this question we conducted a systematic literature review involving 185 papers. Based on the literature review we present 1) well-defined categories of opinion mining-related software development activities, 2) available opinion mining approaches, whether they are evaluated when adopted in other studies, and how their performance is compared, 3) available datasets for performance evaluation and tool customization, and 4) concerns or limitations SE researchers might need to take into account when applying/customizing these opinion mining techniques. The results of our study serve as references to choose suitable opinion mining tools for SE tasks, and provide critical insights for the further development of opinion mining techniques in the SE domain.
This work has been done together with Bin Lin, Gabriele Bavota and Michele Lanza from Università della Svizzera italiana, Switzerland, Nathan Cassee from Eindhoven University of Technology, The Netherlands and Nicole Novielli from University of Bari, Italy.
Implications of Open Source Software Use (or Let's Talk Open Source)Gail Murphy
A talk given to the UBC Computer Science Alumni group discussing a number of implications of the use of open source as part of the global software supply chain.
Analyzing Big Data's Weakest Link (hint: it might be you)HPCC Systems
Tim Menzies, NC State University, presents at the 2015 HPCC Systems Engineering Summit Community Day.
For Big Data applications, there is a lack of any gold standards for "good analysis" or methods to assess our certification programs. Hence, we are still in the dark about whether or not our human analysts are making the best use possible of the tools of Big Data. While much progress has been made in the systems aspects of Big Data, certain critical human-centered aspects remain an open issue. Regardless of the sophistication of the analysis tools and environment, all that architecture can still be used incorrectly by users. If this issue was confined to a small number of inexperienced users, then it could be addressed via process improvements such as better training. But is it? What do we know about our analysts? Where are the studies that mine the people doing the data miners?
This presentation offers some preliminary results on tools that combine ECL with other methods that recognize the code generated by experienced or inexperienced developers. While the results are preliminary, they do raise the possibility that we can better characterize what it means to be experienced (or inexperienced) at Big Data applications.
Keynote presentation from ECBS conference. The talk is about how to use machine learning and AI in improving software engineering. Experiences from our project in Software Center (www.software-center.se).
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019Patrizio Pelliccione
ML and AI are increasingly dominating the high-tech industry. Organizations and technology companies are leveraging their big data to create new products or improve their processes to reach the next level in their market. However, ML and AI are not a silver bullet and Software 2.0 is not the end of software developers or software engineering.
In this talk I will argument on how software engineering can help ML and AI to become the key technology for (autonomous) systems of the near future. Software engineering best practices and achievements reached in the last decades might help, e.g., (i) democratising the use of ML/AI, (ii) composing, reusing, chaining ML/AI models to solve more complex problems, and (iii) supporting for reasoning about correctness, repeatability, explainability, traceability, fairness, ethics, while building an ML/AI pipeline.
The complexity of current software-based systems has led the software engineering community to look for inspiration in diverse related fields (e.g., robotics, artificial intelligence) as well as other areas (e.g., biology) to find new ways of designing and managing systems and services.
Publish or Perish: Questioning the Impact of Our Research on the Software Dev...Margaret-Anne Storey
A Video for this talk can be found here: https://www.youtube.com/watch?v=DvRdBb9TEUI
Abstract: How often do we pause to consider how we, as a community, decide which developer problems we address, or how well we are doing at evaluating our solutions within real development contexts? Many of our research contributions in software engineering can be considered as purely technical. Yet somewhere, at some time, a software developer may be impacted by our research. In this talk, I invite the community to question the impact of our research on software developer productivity. To guide the discussion, I first paint a picture of the modern-day developer and the challenges they experience. I then present 4+1 views of software engineering research --- views that concern research context, method choice, research paradigms, theoretical knowledge and real-world impact. I demonstrate how these views can be used to design, communicate and distinguish individual studies, but also how they can be used to compose a critical perspective of our research at a community level. To conclude, I propose structural changes to our collective research and publishing activities --- changes to provoke a more expeditious consideration of the many challenges facing today's software developer.
(Thanks to Brynn Hawker for slide design and proposed new badges. brynn@hawker.me)
In 2003 Dave et al. have coined the term “opinion mining” to refer to “processing a set of search results for a given item, generating a list of product attributes (quality, features, etc.) and aggregating opinions about each of them (poor, mixed, good)”. Nine years later, in 2012 Brooks and Swigger have applied sentiment analysis in the context of software engineering. Today another nine years have passed and it is time to look back: what have we achieved as a research community and where should we go next?
To answer this question we conducted a systematic literature review involving 185 papers. Based on the literature review we present 1) well-defined categories of opinion mining-related software development activities, 2) available opinion mining approaches, whether they are evaluated when adopted in other studies, and how their performance is compared, 3) available datasets for performance evaluation and tool customization, and 4) concerns or limitations SE researchers might need to take into account when applying/customizing these opinion mining techniques. The results of our study serve as references to choose suitable opinion mining tools for SE tasks, and provide critical insights for the further development of opinion mining techniques in the SE domain.
This work has been done together with Bin Lin, Gabriele Bavota and Michele Lanza from Università della Svizzera italiana, Switzerland, Nathan Cassee from Eindhoven University of Technology, The Netherlands and Nicole Novielli from University of Bari, Italy.
Implications of Open Source Software Use (or Let's Talk Open Source)Gail Murphy
A talk given to the UBC Computer Science Alumni group discussing a number of implications of the use of open source as part of the global software supply chain.
Analyzing Big Data's Weakest Link (hint: it might be you)HPCC Systems
Tim Menzies, NC State University, presents at the 2015 HPCC Systems Engineering Summit Community Day.
For Big Data applications, there is a lack of any gold standards for "good analysis" or methods to assess our certification programs. Hence, we are still in the dark about whether or not our human analysts are making the best use possible of the tools of Big Data. While much progress has been made in the systems aspects of Big Data, certain critical human-centered aspects remain an open issue. Regardless of the sophistication of the analysis tools and environment, all that architecture can still be used incorrectly by users. If this issue was confined to a small number of inexperienced users, then it could be addressed via process improvements such as better training. But is it? What do we know about our analysts? Where are the studies that mine the people doing the data miners?
This presentation offers some preliminary results on tools that combine ECL with other methods that recognize the code generated by experienced or inexperienced developers. While the results are preliminary, they do raise the possibility that we can better characterize what it means to be experienced (or inexperienced) at Big Data applications.
Synergy of Human and Artificial Intelligence in Software EngineeringTao Xie
Keynote Talk by Tao Xie at International NSF sponsored Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE 2013) http://promisedata.org/raise/2013/
Recovery of Traceability Links and Behavior Models for Software Maintenance,...Hironori Washizaki
Hironori Washizaki, Recovery of Traceability Links and Behavior Models for Software Maintenance, Keynote, 2016 International Conference for Top and Emerging Computer Scientists (IC-TECS 2016), Taipei, Dec 24, 2016.
SBQS 2013 Keynote: Cooperative Testing and AnalysisTao Xie
SBQS 2013 Keynote: Cooperative Testing and Analysis: Human-Tool, Tool-Tool, and Human-Human Cooperations to Get Work Done http://sbqs.dcc.ufba.br/view/palestrantes.php
MSR 2022 Foundational Contribution Award Talk: Software Analytics: Reflection...Tao Xie
MSR 2022 Foundational Contribution Award Talk on "Software Analytics: Reflection and Path Forward" by Dongmei Zhang and Tao Xie
https://conf.researchr.org/info/msr-2022/awards
Diversity and Computing/Engineering: Perspectives from AlliesTao Xie
Slides from the invited talk given on Feb 13, 2019 being part of a diversity and inclusion week - Infusion 2019. Infusion is a diversity focused week for the Illinois College of Engineering, hosted by the Dean's Student Advisory Committee of Engineering Council. This invited talk was co-hosted by the NSBE - UIUC chapter.
Transferring Software Testing Tools to PracticeTao Xie
ACM SIGSOFT Webinar co-presented by Nikolai Tillmann (Microsoft), Judith Bishop (Microsoft Research), Pratap Lakshman (Microsoft), Tao Xie (University of Illinois at Urbana-Champaign) http://www.sigsoft.org/resources/webinars.html
Transferring Software Testing and Analytics Tools to PracticeTao Xie
Keynote Talk in the Workshop on Testing: Academia-Industry Collaboration, Practice and Research Techniques (TAIC PART 2016) http://www2016.taicpart.org/
Towards Mining Software Repositories Research that MattersTao Xie
Towards Mining Software Repositories Research that Matters. Talk slides at Next Generation of Mining Software Repositories '14 (Pre-FSE 2014 Event), Nov 15–16. HKUST, Hong Kong http://ng2014.msrworld.org/
Teaching and Learning Programming and Software Engineering via Interactive Ga...Tao Xie
Pex4Fun (http://www.pex4fun.com/) is a web-based educational gaming environment for teaching and learning programming and software engineering. Pex4Fun can be used to teach and learn programming and software engineering at many levels, from high school all the way through graduate courses. With Pex4Fun, a student edits code in any browser – with Intellisense – and Pex4Fun executes it and analyzes it in the cloud. Pex4Fun connects teachers, curriculum authors, and students in a unique social experience, tracking and streaming progress updates in real time. In particular, Pex4Fun finds interesting and unexpected input values (with Pex, an advanced test-generation tool) that help students understand what their code is actually doing. The real fun starts with coding duels where a student writes code to implement a teacher's secret specification (in the form of sample-solution code not visible to the student). Pex4Fun finds any discrepancies in behavior between the student’s code and the secret specification. Such discrepancies are given as feedback to the student to guide how to fix the student’s code to match the behavior of the secret specification. In early 2014, Code Hunt (https://www.codehunt.com/) has been released as a redesign of Pex4Fun as game.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Corporate Management | Session 3 of 3 | Tendenci AMS
Intelligent Software Engineering: Synergy between AI and Software Engineering (UCI ISR 2018)
1. Intelligent Software Engineering:
Synergy between AI and Software
Engineering
Tao Xie
University of Illinois at Urbana-Champaign
taoxie@illinois.edu
http://taoxie.cs.illinois.edu/
5. Dynamic Symbolic Execution
Code to generate inputs for:
Constraints to solve
a!=null
a!=null &&
a.Length>0
a!=null &&
a.Length>0 &&
a[0]==1234567890
void CoverMe(int[] a)
{
if (a == null) return;
if (a.Length > 0)
if (a[0] == 1234567890)
throw new Exception("bug");
}
Observed constraints
a==null
a!=null &&
!(a.Length>0)
a!=null &&
a.Length>0 &&
a[0]!=1234567890
a!=null &&
a.Length>0 &&
a[0]==1234567890
Data
null
{}
{0}
{123…}a==null
a.Length>0
a[0]==123…
T
TF
T
F
F
Execute&MonitorSolve
Choose next path
Done: There is no path left.
Negated condition
[DART: Godefroid et al. PLDI’05]
Z3Constraint solver
has decision procedures for
- Arrays
- Linear integer arithmetic
- Bitvector arithmetic
- Floating-point arithmetic
- …
6. Past: Automated Software Testing
• 10 years of collaboration with Microsoft Research on Pex
[ASE’14 Ex]
• .NET Test Generation Tool based on Dynamic Symbolic Execution
• Tackle challenges of
• Path explosion via fitness function [DSN’09]
• Method sequence explosion via program synthesis [OOPSLA’11]
• …
• Shipped in Visual Studio 2015/2017 Enterprise Edition
• As IntelliTest
Tillmann, de Halleux, Xie. Transferring an Automated Test Generation Tool to Practice: From Pex to Fakes and Code
Digger. ASE’14 Experience Papers http://taoxie.cs.illinois.edu/publications/ase14-pexexperiences.pdf
7. Past: Android App Testing
• 2 years of collaboration with Tencent Inc. WeChat testing team
• Guided Random Test Generation Tool improved over Google Monkey
• Resulting tool deployed in daily WeChat testing practice
• WeChat = WhatsApp + Facebook + Instagram + PayPal + Uber …
• #monthly active users: 1 billion @2018 March
• Daily#: dozens of billion messages sent, hundreds of million photos uploaded,
hundreds of million payment transactions executed
• First studies on testing industrial Android apps
[FSE’16IN][ICSE’17SEIP]
• Beyond open source Android apps
focused by academia
WeChat
9. Android Test Generation Tools: A Retrospective
Monkey
Official
Blind random
9
2008 2017
Stoat
FSE ’17
Model-based
evolutionary
2012
GUIRipper
ASE ‘12
Model-based
2013 2015 2016... ..
A3E
OOPSLA ‘13
Systematic
Dynodroid
FSE ‘13
Guided random
SwiftHand
OOPSLA ‘13
Model-based
DroidBot
ICSE-C ‘17
Model-based
ACTEve
FSE ‘12
Concolic
WCTester
FSE-Ind ‘16
Guided random
Sapienz
ISSTA ‘16
Evolutionary
Study by
Choudhary et
al.
ASE ‘15
How do these tools perform on industrial apps
that people actually use everyday?
10. Android Test Generation Tools: Existing Evaluations
10
2017
Stoat
FSE ’17
Model-based
evolutionary
2012
GUIRipper
ASE ‘12
Model-based
2013 2015 2016..
A3E
OOPSLA ‘13
Systematic
Dynodroid
FSE ‘13
Guided random
SwiftHand
OOPSLA ‘13
Model-based
DroidBot
ICSE-C ‘17
Model-based
ACTEve
FSE ‘12
Concolic
WCTester
FSE-Ind ‘16
Guided random
Sapienz
ISSTA ‘16
Evolutionary
Study by
Choudhary et
al.
ASE ‘15
Industrial apps
not involved
Industrial apps
limitedly involved
Single case study
only
There is no comprehensive comparison among existing
tools over industrial apps.
Does a newly proposed tool really outperform existing
tools (especially Monkey) on industrial apps?
Wang, Li, Yang, Cao, Zhang, Deng, Xie. An Empirical Study of Android Test Generation Tools in Industrial Cases. ASE’18.
http://taoxie.cs.illinois.edu/publications/ase18-androidtest.pdf
11. Next: Intelligent Software Testing
• Learning from others working on the same things
• Our work on mining API usage method sequences to test the API
[ESEC/FSE’09: MSeqGen]
• Visser et al. Green: Reducing, reusing and recycling constraints in program analysis.
[FSE’12]
• Learning from others working on similar things
• Jia et al. Enhancing reuse of constraint solutions to improve symbolic execution.
[ISSTA’15]
• Aquino et al. Heuristically Matching Solution Spaces of Arithmetic Formulas to
Efficiently Reuse Solutions. [ICSE’17]
[Jia et al. ISSTA’15]
Continuous Learning
12. Software Analytics
Software analytics is to enable software practitioners
to perform data exploration and analysis in order to
obtain insightful and actionable information for data-
driven tasks around software and services.
Dongmei Zhang, Yingnong Dang, Jian-Guang Lou, Shi Han, Haidong Zhang, and Tao Xie. Software Analytics as
a Learning Case in Practice: Approaches and Experiences. In MALETS 2011
http://research.microsoft.com/en-us/groups/sa/malets11-analytics.pdf
13. Software Analytics
Software analytics is to enable software practitioners
to perform data exploration and analysis in order to
obtain insightful and actionable information for data-
driven tasks around software and services.
Dongmei Zhang, Yingnong Dang, Jian-Guang Lou, Shi Han, Haidong Zhang, and Tao Xie. Software Analytics as
a Learning Case in Practice: Approaches and Experiences. In MALETS 2011
http://research.microsoft.com/en-us/groups/sa/malets11-analytics.pdf
14. Data sources
Runtime traces
Program logs
System events
Perf counters
…
Usage log
User surveys
Online forum posts
Blog & Twitter
…
Source code
Bug history
Check-in history
Test cases
Eye tracking
MRI/EMG
…
16. Past: Software Analytics
• StackMine [ICSE’12, IEEESoft’13]: performance debugging in the large
• Data Source: Performance call stack traces from Windows end users
• Analytics Output: Ranked clusters of call stack traces based on shared patterns
• Impact: Deployed/used in daily practice of Windows Performance Analysis team
• XIAO [ACSAC’12, ICSE’17 SEIP]: code-clone detection and search
• Data Source: Source code repos (+ given code segment optionally)
• Analytics Output: Code clones
• Impact: Shipped in Visual Studio 2012; deployed/used in daily practice of
Microsoft Security Response Center
Internet
@Microsoft Research Asia
17. Past: Software Analytics
• Service Analysis Studio [ASE’13-EX]: service incident management
• Data Source: Transaction logs, system metrics, past incident reports
• Analytics Output: Healing suggestions/likely root causes of the given incident
• Impact: Deployed and used by an important Microsoft service (hundreds of
millions of users) for incident management
@Microsoft Research Asia
18. Open Source Microservice Benchmark System
TrainTicket
70+ microservices, including 41 business ones, 30 infrastructure ones (message middleware service,
distributed cache services, database services), totally 300K LOC
Git Repo:https://github.com/microcosmx/train_ticket
• Include Java、Python、Go、
Node.js
• Use asynchronous communication
and queue
• Substantial test cases including
100+ unit and integration tests
• Visualization tools for runtime
monitoring and management
Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Chenjie Xu, Chao Ji, and Wenyun Zhao. Poster: Benchmarking Microservice Systems for Software
Engineering Research. ICSE 2018 Posters. http://taoxie.cs.illinois.edu/publications/icse18poster-microservices.pdf
Fudan、UIUC、SUTD Collaborative Research
21. Translation of NL to Regular Expressions/SQL
● Program Aliasing: a semantically equivalent program may have
many syntactically different forms
NL Sentences
22. NL Regex: sequence-to-sequence model
● Encoder/Decoder: 2 layers stacked LSTM architectures [Locascio et al. EMNLP’16]
23. Training Objective: Maximum Likelihood Estimation (MLE)
Maximizing Semantic Correctness
● Standard seq-to-seq maximizes likelihood mapping NL to ground truth
● MLE penalizes syntactically different but semantically equivalent regex
● Reward : semantic correctness
● Alternative objective: Maximize the expected
Leveraging the REINFORCE technique of policy gradient [William’92] to maximize Expected Semantic Correctness
Zhong, Guo, Yang, Peng, Xie, Lou, Liu, Zhang. SemRegex: A Semantics-Based Approach for Generating Regular Expressions
from Natural Language Specifications. EMNLP’18. http://taoxie.cs.illinois.edu/publications/emnlp18-semregex.pdf
24. Measurements of Semantic Correctness
([ABab]&[A-Z]).*X● Minimal DFAs
● Test Cases (pos/neg string examples)
25. Evaluation Results of NLRegex Approaches
Zhong, Guo, Yang, Peng, Xie, Lou, Liu, Zhang. SemRegex: A Semantics-Based Approach for Generating Regular Expressions
from Natural Language Specifications. EMNLP’18. http://taoxie.cs.illinois.edu/publications/emnlp18-semregex.pdf
DFA-equivalence Accuracy
27. Intelligent SE:Tech Transfer
http://www.diffblue.com/
Oxford University spin-off,
Daniel Kroening et al.
Peking University spin-
off,
Ge Li et al.
https://www.codota.com/
Technion spin-off,
Eran Yahav et al.
Technical University Munich spin-off,
Benedikt Hauptmann et al.
https://www.qualicen.de/en/
http://www.aixcoder.com/
MaJiCKe
UCL spin-off, Mark Harman et al.
Acquired by Facebook
http://www.engineering.ucl.ac.uk/news/bug-finding-majicke-finds-home-facebook/
https://www.deepcode.ai/
ETH Zurich spin-off,
Martin Vechev et al.
28. Quite Many Recent Papers in AI/ML for SE
https://ml4code.github.io/
• 2018 (35)
• 2017 (34)
• 2016 (25)
• 2015 (25)
• 2014 (14)
• 2013 (9)
• 2012 (1)
• 2009 (1)
• 2007 (1)https://arxiv.org/abs/1709.06182
31. Self-Driving Tesla Involved in Fatal Crash (2016 June 30)
http://www.nytimes.com/2016/07/01/business/self-driving-tesla-fatal-crash-investigation.html
“A Tesla car in autopilot crashed into a trailer
because the autopilot system failed to recognize
the trailer as an obstacle due to its “white color
against a brightly lit sky” and the “high ride
height”
http://www.cs.columbia.edu/~suman/docs/deepxplore.pdf
33. Microsoft's Teen Chatbot Tay
Turned into Genocidal Racist (2016 March 23/24)
http://www.businessinsider.com/ai-expert-explains-why-microsofts-tay-chatbot-is-so-racist-2016-3
"There are a number of precautionary
steps they [Microsoft] could have taken.
It wouldn't have been too hard to create
a blacklist of terms; or narrow the scope
of replies. They could also have simply
manually moderated Tay for the first few
days, even if that had meant slower
responses."
“businesses and other AI developers will
need to give more thought to the
protocols they design for testing and
training AIs like Tay.”
34. Amazon’s AI Tool Discriminating Against Women
https://www.businessinsider.com/amazon-built-ai-to-hire-people-discriminated-against-women-2018-10
35. Amazon’s AI Tool Discriminating Against Women
https://www.businessinsider.com/amazon-built-ai-to-hire-people-discriminated-against-women-2018-10
36. Adversarial Machine Learning/Testing
● Adversarial testing [Szegedy et al. ICLR’14]: find corner-case inputs imperceptible
to human but induce errors
36
School bus OstrichCarefully crafted noise
Pei et al. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. SOSP 2017. Slide adapted from SOSP’17 slides
37. DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Systematic testing of Deep Neural Nets (DNNs)
• Neuron coverage: testing coverage metric for
deep nerual net
• Automated: cross-check multiple DNNs
• Realistic: physically realizable transformations
(e.g., lighting)
• Effective:
• 15 State-of-the-art DNNs on 5 large datasets (ImageNet,
Self-driving cars, PDF/Android malware)
• Numerous corner-case errors
• 50% more neuron coverage than existing testing
No accident
Darker: Accident
DeepXplore
37
Pei et al. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. SOSP 2017. Slide adapted from SOSP’17 slides
38. Example Detected Erroneous Behaviors
Turn rightGo straight
38
Go straight Turn left
Pei et al. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. SOSP 2017.
Tian et al. DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars. ICSE 2018. Slide adapted from SOSP’17 slides
39. Example Detected Erroneous Behaviors
Turn rightGo straight
39
Go straight Turn left
Pei et al. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. SOSP 2017.
Tian et al. DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars. ICSE 2018. Slide adapted from SOSP’17 slides
Lu et al. NO Need to Worry about Adversarial Examples in
Object Detection in Autonomous Vehicles. CVPR’17.
https://arxiv.org/abs/1707.03501“
“
40. Neural Machine Translation
Screen snapshot captured on April 5, 2018
• Overall better than statistical machine
translation
• Worse controllability
• Existing translation quality assurance
Need reference translation,not
applicable online
Cannot precisely locate problem
types and
41. Translation Quality Assurance
● Key idea:black-box algorithms specialized for common problems
○ No need for reference translation; need only the original sentence and generated
translation
○ Precise problem localization
● Common problems
○ Under-translation
○ Over-translation
Tencent、UIUC Collaborative Work
Zheng, Wang, Liu, Zhang, Zeng, Deng, Yang, He, Xie. Testing Untestable Neural Machine Translation: An Industrial Case. arXiv:1807.02340, July 2018.
42. Industry Impact
● Adopted to improve WeChat translation service (over 1 billion users,
online serving 12 million translation tasks)
○ Offline monitoring (regression testing)
○ Online monitoring (real time selection of best model)
● Large scale test data for translation
○ ~130K English/180K Chinese words/phrases
○ Detect numerous problems in Google Translate and YouDao
BLEU Score Improvement %Problems Reduction
Problem Cases in Other Translation Services
Tencent、UIUC Collaborative Work
Zheng, Wang, Liu, Zhang, Zeng, Deng, Yang, He, Xie.
Testing Untestable Neural Machine Translation: An
Industrial Case. arXiv:1807.02340, July 2018.
43. Quite Many Recent Papers in SE for AI/ML
• Ma et al. MODE: Automated Neural Network Model Debugging via State Differential Analysis and Input
Selection. ESEC/FSE’18
• Sun et al. Concolic Testing for Deep Neural Networks. ASE’18
• Udeshi et al. Automated Directed Fairness Testing. ASE’18
• Ma et al. DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. ASE’18
• Zhang et al. DeepRoad: GAN-based Metamorphic Testing and Input Validation Framework for Autonomous
Driving Systems. ASE’18
• Dwarakanath et al. Identifying Implementation Bugs in Machine Learning based Image Classifiers using
Metamorphic Testing. ISSTA’18
• Zhang et al. An Empirical Study on TensorFlow Program Bugs. ISSTA’18
• Tian et al. DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars. ICSE’18
• Abdessalem et al. Testing Vision-Based Control Systems Using Learnable Evolutionary Algorithms. ICSE’18
• Odena, Goodfellow. TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing.
arXiv:1807.10875. 2018.
• …
45. (SE AI) Practice Impact
Problem
Domain
Solution
Domain
Practice
Intelligent Software Engineering
Intelligence Software Engineering
46. 46
Thank You!
Q & A
This work was supported in part by NSF under grants no. CNS-1513939, CNS-1564274, CCF-1816615, and a grant from
the ZJUI Research Program.