Using *Grimoire to Analyze
Upcoming SlideShare
Loading in...5
×
 

Using *Grimoire to Analyze

on

  • 567 views

 

Statistics

Views

Total Views
567
Views on SlideShare
567
Embed Views
0

Actions

Likes
0
Downloads
13
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Using *Grimoire to Analyze Using *Grimoire to Analyze Presentation Transcript

  • Using *Grimoire to analyze OpenStackJesus M. Gonzalez-Barahonajgb@bitergia.comhttp://identi.ca/jgbarah http://twitter.com/jgbarahBitergiaGSyC/LibreSoft (Universidad Rey Juan Carlos)OpenStack Summit, Portland, April 15th, 2013Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 1 /20
  • c 2012, 2013 BitergiaSome rights reserved. This presentation is distributed under the“Attribution-ShareAlike 3.0” license, by Creative Commons, available athttp://creativecommons.org/licenses/by-sa/3.0/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 2 /20
  • We’ve done the Grizzly companies analysishttp://blog.bitergia.com/2013/04/04/companies-contributing-to-openstack-grizzly-analysis/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 3 /20
  • We’re deploying the OpenStack activity dashboardhttp://activity.openstack.org/dash/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 4 /20
  • We’re deploying the OpenStack activity dashboard (2)This is a (preliminary) preview!!!http://activity.openstack.org/dash/dashboard/Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 5 /20
  • Measuring OpenStack activityInformation about code, community, developmentfor OpenStackcan be retrieved, organized, analyzedLet’s do it!Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 6 /20
  • Measuring OpenStack activity (why?)Why?Open development: transparencyComplex information: knowledge extractionCommunity-based decisions: objective informationDecision tracking: parameters to decide...Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 7 /20
  • Data has to be extracted, minedData lives in repositoriesusually not designed to release it easily:tools are needed to retrieve and extractData includes many complexities and detailstools are needed to assist in mining, analysisAnalyze free software with free software!Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 8 /20
  • The MetricsGrimoire approachSet of tools specialized in retrieving information fromdifferent kinds of repositories. Among them:CVSAnalY: source code management(CVS, Subversion, git, etc.)Bicho: issue tracking systems(Bugzilla, Jira, SourceForge, Allura, Launchpad,Google Code, etc.)& code revision systems (Gerrit)MLStats: mailing lists(mbox files, Mailman archives, etc.)Store all the information in SQL databaseshttp://metricsgrimoire.github.comJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 9 /20
  • vizGrimoire: Milking the databasesOnce information is ready for querying:it can be queried directly in the databaseit can be analyzed from R, Python, etc.it can be filtered, manually inspected, improvedit can be combined, cross-analyzedit can be visualizedSet of tools to simplify & automate all of thishttps://vizgrimoire.github.comJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 10 /20
  • vizGrimoireR: statistics, chartsR package specialized in managing MetricsGrimoireinformationConnects directly to the database and:gets the information from itfilters & massages itdoes statistical analysis on itproduces charts and WebGL 3D graphsproduces JSON files to export to other tools...and lets you unleash all the potential of RJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 11 /20
  • vizGrimoireJS: visualizationJavaScript library producing visualizationsRetrieves JSON files and produces:live charts: evolution, pies, bars, etc.tables and textcomparative chartsactionable dashboardssoon to support links to information in forgeIntegration with HTML5 applicationsJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 12 /20
  • How the OpenStack dashboard is createdRun MetricsGrimoire (CVSAnalY, Bicho, MLStats) onrepositoriesExample: determining which git repositories to mineProduce queries specific for OpenStackExample: condition for deciding who closes a ticketRun customized Python & R scripts to produce JSONfiles Example: produce results per OpenStacksubprojectCustomize vizGrimoireJS to display charts Example:remove bots from top tablesExport the result via HTTP We still have performanceissues...Do all of this continuouslyJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 13 /20
  • Future features: tracking other parametersTime0.99(black)/0.95(green)/0.5(red)/0.25(blue)2000 2002 2004 2006 2008 2010 201223456Time-to-close (quantiles over time): Time in minutes, log 10 scaleJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 14 /20
  • Future features: tracking other parameters (2)Time-to-close (quantiles over time)Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 15 /20
  • Future features: tracking other parameters (3)Demographics (attraction rate, retention rate per generation)Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 16 /20
  • In summary...OpenStack repositories have a wealth of informationWe all can do (and many do) our own analysisFree software to analyze free software developmentLet’s define common formats to interface to differenttoolsWe can incrementally develop a powerful platformWhat would you like to know about OpenStack?Jesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 17 /20
  • Bitergia: an spin-offStarted operations in July 2012Builds on the experience of LibreSoft R&D groupOffering professional products and servicesFocused on:Metrics about software development(including community metrics)Specialized support for development forges(including metrics for projects)http://bitergia.comJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 18 /20
  • CreditsThanks go to...Many LibreSoft developers who developed MetricsGrimoireThe (small) community now maintaining MetricsGrimoireSome Bitergia developers producing vizGrimoireThe (future) community maintaining vizGrimoireThe many free software developers who produced all the software onwhich these tools relyYou, who are producing OpenStackhttp://libresoft.eshttp://bitergia.comJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 19 /20
  • This is the end, my friendPlease, provide *any* feedback[I would love to know what interested you the most][...and the least]Final note:Let’s find ways to understand OpenStack a bit betterJesus Gonzalez-Barahona (Bitergia) Using *Grimoire to analyze OpenStackOpenStack Summit Portland 2013 20 /20