Using *Grimoire to analyze OpenStack                                     Jesus M. Gonzalez-Barahona                       ...
c 2012, 2013 Bitergia            Some rights reserved. This presentation is distributed under the    “Attribution-ShareAli...
We’ve done the Grizzly companies analysis                            http://blog.bitergia.com/2013/04/04/           compan...
We’re deploying the OpenStack activity dashboard                                            http://activity.openstack.org/...
We’re deploying the OpenStack activity dashboard (2)This is a (preliminary) preview!!!                            http://a...
Measuring OpenStack activity Information about code, community, development                                    for OpenSta...
Measuring OpenStack activity (why?)Why?                Open development: transparency     Complex information: knowledge e...
Data has to be extracted, minedData lives in repositoriesusually not designed to release it easily:                       ...
The MetricsGrimoire approachSet of tools specialized in retrieving information fromdifferent kinds of repositories. Among t...
vizGrimoire: Milking the databasesOnce information is ready for querying:       it   can     be     queried directly in th...
vizGrimoireR: statistics, chartsR package specialized in managing MetricsGrimoireinformationConnects directly to the datab...
vizGrimoireJS: visualizationJavaScript library producing visualizationsRetrieves JSON files and produces:    live charts: e...
How the OpenStack dashboard is created      Run MetricsGrimoire (CVSAnalY, Bicho, MLStats) on      repositories      Examp...
Future features: tracking other parameters                 0.99 (black) / 0.95 (green) / 0.5 (red) / 0.25 (blue)          ...
Future features: tracking other parameters (2)Time-to-close (quantiles over time)                                         ...
Future features: tracking other parameters (3)Demographics (attraction rate, retention rate per generation)               ...
In summary...       OpenStack repositories have a wealth of information       We all can do (and many do) our own analysis...
Bitergia: an spin-offStarted operations in July 2012Builds on the experience of LibreSoft R&D groupOffering professional pro...
CreditsThanks go to...       Many LibreSoft developers who developed MetricsGrimoire       The (small) community now maint...
This is the end, my friend     Please, provide *any* feedback     [I would love to know what interested you the most]     ...
Upcoming SlideShare
Loading in...5
×

Gonzalez barahona community_board_metrics_0415162

369

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
369
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Gonzalez barahona community_board_metrics_0415162

  1. 1. Using *Grimoire to analyze OpenStack Jesus M. Gonzalez-Barahona jgb@bitergia.com http://identi.ca/jgbarah http://twitter.com/jgbarah Bitergia GSyC/LibreSoft (Universidad Rey Juan Carlos) OpenStack Summit, Portland, April 15th, 2013 OpenStack Summit Portland 2013 1/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  2. 2. c 2012, 2013 Bitergia Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 3.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/3.0/ OpenStack Summit Portland 2013 2/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  3. 3. We’ve done the Grizzly companies analysis http://blog.bitergia.com/2013/04/04/ companies-contributing-to-openstack-grizzly-analysis/ OpenStack Summit Portland 2013 3/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  4. 4. We’re deploying the OpenStack activity dashboard http://activity.openstack.org/dash/ OpenStack Summit Portland 2013 4/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  5. 5. We’re deploying the OpenStack activity dashboard (2)This is a (preliminary) preview!!! http://activity.openstack.org/dash/dashboard/ OpenStack Summit Portland 2013 5/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  6. 6. Measuring OpenStack activity Information about code, community, development for OpenStack can be retrieved, organized, analyzedLet’s do it! OpenStack Summit Portland 2013 6/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  7. 7. Measuring OpenStack activity (why?)Why? Open development: transparency Complex information: knowledge extraction Community-based decisions: objective information Decision tracking: parameters to decide ... OpenStack Summit Portland 2013 7/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  8. 8. Data has to be extracted, minedData lives in repositoriesusually not designed to release it easily: tools are needed to retrieve and extractData includes many complexities and details tools are needed to assist in mining, analysisAnalyze free software with free software! OpenStack Summit Portland 2013 8/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  9. 9. The MetricsGrimoire approachSet of tools specialized in retrieving information fromdifferent kinds of repositories. Among them: CVSAnalY: source code management (CVS, Subversion, git, etc.) Bicho: issue tracking systems (Bugzilla, Jira, SourceForge, Allura, Launchpad, Google Code, etc.) & code revision systems (Gerrit) MLStats: mailing lists (mbox files, Mailman archives, etc.)Store all the information in SQL databases http://metricsgrimoire.github.com OpenStack Summit Portland 2013 9/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  10. 10. vizGrimoire: Milking the databasesOnce information is ready for querying: it can be queried directly in the database it can be analyzed from R, Python, etc. it can be filtered, manually inspected, improved it can be combined, cross-analyzed it can be visualizedSet of tools to simplify & automate all of this https://vizgrimoire.github.com OpenStack Summit Portland 2013 10 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  11. 11. vizGrimoireR: statistics, chartsR package specialized in managing MetricsGrimoireinformationConnects directly to the database and: gets the information from it filters & massages it does statistical analysis on it produces charts and WebGL 3D graphs produces JSON files to export to other tools...and lets you unleash all the potential of R OpenStack Summit Portland 2013 11 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  12. 12. vizGrimoireJS: visualizationJavaScript library producing visualizationsRetrieves JSON files and produces: live charts: evolution, pies, bars, etc. tables and text comparative charts actionable dashboards soon to support links to information in forgeIntegration with HTML5 applications OpenStack Summit Portland 2013 12 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  13. 13. How the OpenStack dashboard is created Run MetricsGrimoire (CVSAnalY, Bicho, MLStats) on repositories Example: determining which git repositories to mine Produce queries specific for OpenStack Example: condition for deciding who closes a ticket Run customized Python & R scripts to produce JSON files Example: produce results per OpenStack subproject Customize vizGrimoireJS to display charts Example: remove bots from top tables Export the result via HTTP We still have performance issues...DoGonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack OpenStack Summit Portland 2013 20 /Jesus all of this continuously 13
  14. 14. Future features: tracking other parameters 0.99 (black) / 0.95 (green) / 0.5 (red) / 0.25 (blue) 6 5 4 3 2 2000 2002 2004 2006 2008 2010 2012 Time Time-to-close (quantiles over time): Time in minutes, log 10 scale OpenStack Summit Portland 2013 14 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  15. 15. Future features: tracking other parameters (2)Time-to-close (quantiles over time) OpenStack Summit Portland 2013 15 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  16. 16. Future features: tracking other parameters (3)Demographics (attraction rate, retention rate per generation) OpenStack Summit Portland 2013 16 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  17. 17. In summary... OpenStack repositories have a wealth of information We all can do (and many do) our own analysis Free software to analyze free software development Let’s define common formats to interface to different tools We can incrementally develop a powerful platform What would you like to know about OpenStack? OpenStack Summit Portland 2013 17 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  18. 18. Bitergia: an spin-offStarted operations in July 2012Builds on the experience of LibreSoft R&D groupOffering professional products and servicesFocused on: Metrics about software development (including community metrics) Specialized support for development forges (including metrics for projects) http://bitergia.com OpenStack Summit Portland 2013 18 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  19. 19. CreditsThanks go to... Many LibreSoft developers who developed MetricsGrimoire The (small) community now maintaining MetricsGrimoire Some Bitergia developers producing vizGrimoire The (future) community maintaining vizGrimoire The many free software developers who produced all the software on which these tools rely You, who are producing OpenStack http://libresoft.es http://bitergia.com OpenStack Summit Portland 2013 19 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  20. 20. This is the end, my friend Please, provide *any* feedback [I would love to know what interested you the most] [...and the least] Final note: Let’s find ways to understand OpenStack a bit better OpenStack Summit Portland 2013 20 /Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStack 20
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×