Analytics for Software
Development
RayBuse•AnalyticsforSoftwareDevelopment
Internship Timeline
1 2 3 4 5 6 7 8 9 10 11
June
12
July AugustMay
First Day
May 2...
RayBuse•AnalyticsforSoftwareDevelopment
You are a manager at Microsoft …
RayBuse•AnalyticsforSoftwareDevelopment
Your responsibilities
• Defines the schedule commitments for the product group
• D...
RayBuse•AnalyticsforSoftwareDevelopment
Your responsibilities
RayBuse•AnalyticsforSoftwareDevelopment
Fortunately …
Telemetry
Architecture
Complexity
Bug Reports
Documentation
Dependen...
RayBuse•AnalyticsforSoftwareDevelopment
Unfortunately …
Telemetry
Architecture
Complexity
Bug Reports
Documentation
Depend...
RayBuse•AnalyticsforSoftwareDevelopment
Which metrics are important?
RayBuse•AnalyticsforSoftwareDevelopment
Which metrics are important?
RayBuse•AnalyticsforSoftwareDevelopment
Which artifacts should be measured?
RayBuse•AnalyticsforSoftwareDevelopment
The Challenge
RayBuse•AnalyticsforSoftwareDevelopment
Analytics
Measurements
Metrics
Exploratory Analysis
Quantitative Analysis
Qualitat...
RayBuse•AnalyticsforSoftwareDevelopment
Analytics
What Happened?
What is happening
now?
What will happen?
How and why did ...
RayBuse•AnalyticsforSoftwareDevelopment
What Happened?
What is happening
now?
What will happen?
How and why did it
happen?...
RayBuse•AnalyticsforSoftwareDevelopment
A basis for making decisions
RayBuse•AnalyticsforSoftwareDevelopment
Decision Scenarios
RayBuse•AnalyticsforSoftwareDevelopment
The difficulty of interpreting data is a
significant barrier to the use of
analyti...
RayBuse•AnalyticsforSoftwareDevelopment
Software Analytics Theory
RayBuse•AnalyticsforSoftwareDevelopment
Exploration
Search for important factors in data.
Analysis
Answer specific questio...
RayBuse•AnalyticsforSoftwareDevelopment
Description Insight Relevant Techniques
Summarization Search for important or unus...
RayBuse•AnalyticsforSoftwareDevelopment
DEMO
RayBuse•AnalyticsforSoftwareDevelopment
Prototype Tool
RayBuse•AnalyticsforSoftwareDevelopment
Surprise Analysis
Distribution of changes
(context)
Sample of changes
RayBuse•AnalyticsforSoftwareDevelopment
Surprise Analysis
Distribution of changes
(context)
Sample of changes
RayBuse•AnalyticsforSoftwareDevelopment
Surprise Analysis
𝑍(𝑥) =
1
𝜎 2𝜋
𝑒−(𝑥−𝜇)2 2𝜎2
Type equation here.
𝑆𝑢𝑟𝑝𝑟𝑖𝑠𝑒 𝑥 =
|𝑍 𝑥...
RayBuse•AnalyticsforSoftwareDevelopment
Surprise Analysis
Context Sample
RayBuse•AnalyticsforSoftwareDevelopment
Surprise Analysis
Context Sample
RayBuse•AnalyticsforSoftwareDevelopment
Software Analysts
RayBuse•AnalyticsforSoftwareDevelopment
In Summary…
Survey Theory Tool
RayBuse•AnalyticsforSoftwareDevelopment
Future Work
Survey Theory
Other
Theories
Other
Tools
More
Features
Tool
RayBuse•AnalyticsforSoftwareDevelopment
In my other life …
Upcoming SlideShare
Loading in...5
×

MSR End of Internship Talk

1,274

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,274
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MSR End of Internship Talk

  1. 1. Analytics for Software Development
  2. 2. RayBuse•AnalyticsforSoftwareDevelopment Internship Timeline 1 2 3 4 5 6 7 8 9 10 11 June 12 July AugustMay First Day May 24 FoSE Paper June 14 Survey ICSE Deadline Aug 20 Last Day Aug 20 Prototype Analysis / Theory This Talk Aug 17
  3. 3. RayBuse•AnalyticsforSoftwareDevelopment You are a manager at Microsoft …
  4. 4. RayBuse•AnalyticsforSoftwareDevelopment Your responsibilities • Defines the schedule commitments for the product group • Delivers the product on time against milestones • Develops effective metrics for the product • Approves group goals and appropriately funds projects • From improving the team's engineering system to improving the group's engineering system • Prevents obstacles and analyzes where and why they occur in order to prevent them • Ensures accountability across the group • Ensures alignment between senior management and product group commitments • Takes responsibility for decisions and uses appropriate decision-making techniques to drive effective decision making in the group • Ensures the group understands its connection to customer/partner value • Ensures effective customer/partner feedback channels and mechanisms are in place and regularly utilized across teams • Ensures the group is appropriately aligned with other product teams to achieve results • Proactively identifies problems across disciplines and improves cross-discipline processes • Effectively takes advantage of his or her personal network of key contacts across disciplines to gather feedback and improve the product • Ensures an appropriate organization is in place and resources are optimized for success • Proactively optimizes the group's recruiting processes and practices • Defines performance standards for the group • Conducts effective performance reviews of all team members, including managers • Has succession plans in place for top talent including own successor • Coaches and motivates managers • Provides critical input into the discipline's vision and strategy • Prioritizes resource needs for the group and executes against prioritized requirements • Effectively implements change aligned with business objectives across the group • Ensures that the product group thoroughly understands the customer's business, including current and future needs
  5. 5. RayBuse•AnalyticsforSoftwareDevelopment Your responsibilities
  6. 6. RayBuse•AnalyticsforSoftwareDevelopment Fortunately … Telemetry Architecture Complexity Bug Reports Documentation Dependencies Expertise Code ClonesVelocity Defect Density Readability Churn Ownership Change Type Test Coverage Failure Models
  7. 7. RayBuse•AnalyticsforSoftwareDevelopment Unfortunately … Telemetry Architecture Complexity Bug Reports Documentation Dependencies Expertise Code ClonesVelocity Defect Density Readability Churn Ownership Change Type Test Coverage Failure Models
  8. 8. RayBuse•AnalyticsforSoftwareDevelopment Which metrics are important?
  9. 9. RayBuse•AnalyticsforSoftwareDevelopment Which metrics are important?
  10. 10. RayBuse•AnalyticsforSoftwareDevelopment Which artifacts should be measured?
  11. 11. RayBuse•AnalyticsforSoftwareDevelopment The Challenge
  12. 12. RayBuse•AnalyticsforSoftwareDevelopment Analytics Measurements Metrics Exploratory Analysis Quantitative Analysis Qualitative Analysis Experiments Insights Insights Why? Measurements What? How much? What if? Goal
  13. 13. RayBuse•AnalyticsforSoftwareDevelopment Analytics What Happened? What is happening now? What will happen? How and why did it happen? What is the next best action? What is the best/ worst that can happen? Past Present Future Information Insight
  14. 14. RayBuse•AnalyticsforSoftwareDevelopment What Happened? What is happening now? What will happen? How and why did it happen? What is the next best action? What is the best/ worst that can happen? Past Present Future Information Insight Difficulty 0.43 Importance 0.98 Difficulty 0.51 Importance 0.88 Difficulty 1 Importance 0.84 Difficulty 0.8 Importance 1 Difficulty 0.84 Importance 0.9 Difficulty 0.88 Importance 0.8 MoreDifficult More DifficultMore Important
  15. 15. RayBuse•AnalyticsforSoftwareDevelopment A basis for making decisions
  16. 16. RayBuse•AnalyticsforSoftwareDevelopment Decision Scenarios
  17. 17. RayBuse•AnalyticsforSoftwareDevelopment The difficulty of interpreting data is a significant barrier to the use of analytics today. 0 5 10 15 20 25 30 Strongly Agree Agree Neutral Disagree Strongly Disagree Managers Developers
  18. 18. RayBuse•AnalyticsforSoftwareDevelopment Software Analytics Theory
  19. 19. RayBuse•AnalyticsforSoftwareDevelopment Exploration Search for important factors in data. Analysis Answer specific questions. Experimentation Compare to alternate scenarios. Past Summarization Trends Modeling Present Alerts (& Correlations) Overlays Benchmarking Future Forecasting Goals Simulation Software Analytics Theory
  20. 20. RayBuse•AnalyticsforSoftwareDevelopment Description Insight Relevant Techniques Summarization Search for important or unusual factors to associated with a time range. Characterize events, understand why they happened. Topic analysis, NLP Alerts (& Correlations) Continuous search for unusual changes or relationships in variables Notice important events. Statistics, Repeated measures Forecasting Search for and predict unusual events in the future based on current trends. Anticipate events. Extrapolation, Statistics Trends How is an artifact changing? Understand the direction of the project. Regression analysis Overlays What artifacts account for current activity? Understand the relationships between artifacts. Cluster analysis, repository mining Goals How are features/artifacts changing in the context of completion or some other goal? Assistance for planning Root-cause analysis Modeling Compares the abstract history of similar artifacts. Identify important factors in history. Learn from previous projects. Machine learning Benchmarking Identify vectors of similarity/difference across artifacts. Assistance for resource allocation and many other decisions Statistics Simulation Simulate changes based on other artifact models. Assistance for general decisions What-if? analysis
  21. 21. RayBuse•AnalyticsforSoftwareDevelopment DEMO
  22. 22. RayBuse•AnalyticsforSoftwareDevelopment Prototype Tool
  23. 23. RayBuse•AnalyticsforSoftwareDevelopment Surprise Analysis Distribution of changes (context) Sample of changes
  24. 24. RayBuse•AnalyticsforSoftwareDevelopment Surprise Analysis Distribution of changes (context) Sample of changes
  25. 25. RayBuse•AnalyticsforSoftwareDevelopment Surprise Analysis 𝑍(𝑥) = 1 𝜎 2𝜋 𝑒−(𝑥−𝜇)2 2𝜎2 Type equation here. 𝑆𝑢𝑟𝑝𝑟𝑖𝑠𝑒 𝑥 = |𝑍 𝑥 − 𝑍 𝜇 | 𝑍(𝜇) normalized
  26. 26. RayBuse•AnalyticsforSoftwareDevelopment Surprise Analysis Context Sample
  27. 27. RayBuse•AnalyticsforSoftwareDevelopment Surprise Analysis Context Sample
  28. 28. RayBuse•AnalyticsforSoftwareDevelopment Software Analysts
  29. 29. RayBuse•AnalyticsforSoftwareDevelopment In Summary… Survey Theory Tool
  30. 30. RayBuse•AnalyticsforSoftwareDevelopment Future Work Survey Theory Other Theories Other Tools More Features Tool
  31. 31. RayBuse•AnalyticsforSoftwareDevelopment In my other life …

×