HereWeCode 2022: Dalhousie University
•Download as PPTX, PDF•
0 likes•169 views
RAISE Lab at Dalhousie University aims to develop tools and technologies for intelligent automation in software engineering. An overview is presented by Dr. Masud Rahman, Assistant Professor, Faculty of Computer Science, Dalhousie University, Canada.
More Related Content
Similar to HereWeCode 2022: Dalhousie University
Similar to HereWeCode 2022: Dalhousie University (20)
More from Masud Rahman
More from Masud Rahman (20)
Recently uploaded
Recently uploaded (20)
HereWeCode 2022: Dalhousie University
- 1. Software Engineering Automated Software Debugging Automated Code Search Automated Code Review Masud Rahman, PhD Assistant Professor Faculty of Computer Science Office: 218, Goldberg Building Dalhousie University, Canada masud.rahman@dal.ca Interested in more details? please visit: https://web.cs.dal.ca/~masud/raise
- 2. Masud Rahman: Academic Journey 2 Khulna University, Bangladesh (2005—2009) University of Saskatchewan, Canada (2012—2019) Polytechnique Montreal, Canada (2019--2020) Dalhousie University, Canada (2020--2022)
- 3. Real Life Software Bugs & Failures $1.7 trillion/year (Global, 2017) Software bug is a fault/error/flaw in the program that causes the program to behave unexpectedly ---Wikipedia
- 4. A Tale of Software Bugs & Features! Find the bug Understand the bug Repair the bug/faulty code Find the right code for a feature Quality control of code-level changes 1 2 3 4
- 5. Bug Report & Bug Localization Bug Localization
- 6. 6 Stack traces 500 keywords Example 1: Find the bug in software code i entry j entry Ci Mi Cj Mj Static Static Hierarchical
- 7. Search Keyword Selection from Trace Graph 7 ) ( ) 1 0 ( | ) ( | ) ( ) 1 ( ) ( i v In j j j i v Out v S v S Ci Cj Mk Mn Cp PageRank Algorithm (Google) ESEC/FSE 2018
- 8. Find the bug using Information Retrieval JDIValue, toString, execute, EvaluationThread, run, NullPointerException able cast null Keyword selection 127 Words 1
- 9. Explain the bug (a.k.a., faulty software code) • Rule-based explanation • Not accurate • Cryptic, hard to understand
- 10. Example 2: Explain a bug with GitHub Bug-fix pull requests 10 Faulty code Commit messages
- 11. Explain a bug with regular texts Abstract Syntax Tree (AST) Convert input dense tensor Deep learning Message
- 12. 12 BufferedImage Grayscale ImageEdit ColorConvertOp File Transparency ColorSpace BufferedImageOp Graphics ImageEffects Convert image to gray scale without losing transparency Example 3: Find the right software code
- 13. Candidate API Collection from Stack Overflow 13 PageRank TF-IDF IndexColorModel ColorSpaceType BufferedImageOp Gray ImageEffects JPEGResize Color IOException Graphics ColorConvertOp Relevant Q&As Code Elements Candidate API List Candidate API List Convert image to Grayscale …. ICSME 2018
- 14. Relevant API Selection with Borda Count 14 BORDA Count: A>B if ∑rank(A) > ∑rank(B) Borda Score Candidate APIs by PageRank Candidate APIs by TF-IDF B: Donald A: Joe
- 15. Relevant API Selection with Word Embedding 15 Semantic Proximity: A>B if proximity(Q,A) > proximity(Q,B) Semantic Proximity Score Query Candidate API List 1.4M
- 16. Impact of NLP2API on Search Query 16 Convert image to gray scale without losing transparency 115 BufferedImage Grayscale ImageEdit ColorConvertOp File Transparency ColorSpace BufferedImageOp Graphics ImageEffects 02 Convert image to gray scale without losing transparency
- 17. Masud Rahman, PhD Assistant Professor Faculty of Computer Science Office: 218, Goldberg CS Building Dalhousie University, Canada masud.rahman@dal.ca Interested in more details? please visit: https://web.cs.dal.ca/~masud/raise
- 18. Relevant API Selection with Word Embedding 18 Semantic Proximity: A>B if proximity(Q,A) > proximity(Q,B) Semantic Proximity Score Query Candidate API List 1.4M
- 19. Query Expansion with Relevant API Classes 19 Borda Score Semantic Proximity Score Initial Query Expanded Query Ranked API Classes
Editor's Notes
- This is my academic journey. I completed my undergrad from Khulna University back in 2009. Then I came to Canada back in 2012 for my graduate studies. I completed my Masters and PhD from University of Saskatchewan. Then in 2019, I moved to Polytechnique Montreal as a postdoctoral fellow.
- This is an example bug report! It talks about a software bug! Now, the bug is hidden somewhere in the software code. So, the process of finding that buggy code is called bug localization.
- This is an example for noisy bug report. It contains stack trace information. It contains five hundred keywords. That means too many signals. It is hard to separate the signals from the noise. That means, the bug report does not work well as a search query. So, how to solve this problem? How can we make a query and find out the bug?
- Once we have the graph, we use a graph-based algorithm called PageRank for keyword selection. Now, this is a recursive algorithm, and its a bit complex. But I will try to explain it with this diagram. So, the algorithm is based on voting mechanism. That is, if a node gets enough votes from other important nodes, that this node is likely to be important. So, why this guy is bigger and laughing? Because, he is being voted by other important nodes. So, this is a recursive process. Once the computation is over, we get a ranked list of nodes. That means, from the trace graph, we get a ranked list of method and class names. But how does it help? Lets see.
- Now, let me show you how Information Retrieval-based bug localization works. This is an example bug report, and we want to find out this buggy code from the codebase. Now, what the existing approaches were doing? They tried to consider this whole bug report as a query Then they submit this query to a code search engine which is sitting on top of our codebase. When I say search engine, I mean the local search engines like Lucene. Now this ad hoc query returns the buggy code at 53rd position. That means the developer needs to check 52 non-buggy code before reaching the buggy code, which is time-consuming and not good for developer productivity. However, if we look closely, we see that this bug report contains 127 words. If we can carefully choose these keywords and use them as a query, we can retrieve the buggy code at the topmost position, which is exactly what we want, right? Now obviously, identifying these keywords is extremely challenging, which makes it our first research problem. So, we reduce the bug localization problem into a keyword selection problem ☺
- Developers search for software bugs and features within a local codebase. However, searching within a local codebase might not be enough. They need to search on the web. Study shows that they spend 20% of their time for code search on the web. So, lets say the developer is implementing a software feature and he/she needs the code that can convert an image to gray scale without losing the transparency Now as a standard practice, the develop makes this natural language query, and submits the query to GitHub, the largest code repository on the web. Now, GitHub provides this result. But as you see, it does not look very relevant. The developer is looking for something like this. If we look carefully, we see that it is pretty hard to retrieve this code with this query. Because, there is not enough keyword matching. But if we can replace this natural language query with these relevant API classes, then we can get lots of keyword matching and we can easily retrieve this code. But as you can imagine, transforming the NL query into these relevant API classes could be very challenging! So, this makes it our third research problem. And once again, Stack Overflow is our friend in this grand challenge.
- First, we submit the query to Stack Overflow, which returns a list of relevant Q&A threads. What is a Q&A thread? Well, here is an example. This is a question, and this is the answer. We also see that the answer contain several program elements such as API classes. So, what we do? We capture these program elements using regular expressions. Then we use two keyword selection algorithms to make two list of candidate API classes. But then what? We use two more items to detect the most relevant API classes from these lists. The essence of Borda count is -- If API A is more frequent than API B in the relevant Q & A threads from Stack Overflow, A is more appropriate than B. So, it’s a kind of likelihood of A over B for the target query. For the second metric, we preprocess Stack Overflow corpus, develop a Skip-gram model using FastText, an improved version of Word2Vec. Then we determine, how close an API is to the given query keywords within the semantic space. So, we A is more semantically close to query Q than B, then A is more appropriate than B for the query. So, we then combine these two metrics for each candidate API class, do the ranking, and return the Top-K classes as our reformulation terms.
- Since we have two candidate lists, we use Borda count to find the most relevant API classes. Now, this is how it works? If Bernie wins according to multiple polls, he is a better candidate. Similarly if API A ranks higher in multiple ranked list than API B, then API A is more relevant in our problem context. So, yes, this is how, we get the Borda score for all the candidates.
- We also use another way to find out the most relevant API classes, this is called semantic proximity. For doing that, first, we collect 1.4 million Q&A threads from Stack Overflow and make a corpus. We preprocess them and feed them to FastText. Now, FastText is a neural text classifier tool that is basically a 3-layer neural network. What it does is, it transforms the corpus into a semantic space. Now, what is a semantic space? Now, this is an example of semantic space for food names. Here we see that Ramen is closer to Spaghetti than burger. Well, if you tasted these foods, then it makes sense, right? Well, similarly, we create semantic space for API classes and keywords and determine their semantic proximity. Finally, we get the semantic proximity score for each API class.
- So, here is the result. If we use only the natural language query, we can retrieve this code example at bottom of the list. But when we use the query our tool, it returns the same relevant code at the 2nd position, which is really interesting. More extensive experiments could be found in the paper.
- We also use another way to find out the most relevant API classes, this is called semantic proximity. For doing that, first, we collect 1.4 million Q&A threads from Stack Overflow and make a corpus. We preprocess them and feed them to FastText. Now, FastText is a neural text classifier tool that is basically a 3-layer neural network. What it does is, it transforms the corpus into a semantic space. Now, what is a semantic space? Now, this is an example of semantic space for food names. Here we see that Ramen is closer to Spaghetti than burger. Well, if you tasted these foods, then it makes sense, right? Well, similarly, we create semantic space for API classes and keywords and determine their semantic proximity. Finally, we get the semantic proximity score for each API class.
- Now, we have two scores for each class. What we do? We combine them, rank them, and then collect top few API classes. Then we add them to the natural language query to get the expanded search query.