Download to read offline
![/run
$ docker run -p 127.0.0.1:5601:5601
-v $(pwd)/credentials.cfg:/mordred-override.cfg
-t grimoirelab/full
Credentials file:
[github]
api-token = XXX](https://image.slidesharecdn.com/softdiveintogrimoirelab-180305160956/75/Soft-Dive-Into-GrimoireLab-Twitter-OSS-workshop-18-2048.jpg)
![/perceval
From command line:
$ perceval [-c <file>] [-g] <backend> [<args>] |--help |
--version
In your Python code:
…
from perceval.backends.core.<backend> import <Backend>
…
backend_repo = <Backend>(<params>)
for item in backend_repo.fetch():
…](https://image.slidesharecdn.com/softdiveintogrimoirelab-180305160956/75/Soft-Dive-Into-GrimoireLab-Twitter-OSS-workshop-27-2048.jpg)
![/visualization (grimoirelab) $ kidash --help
usage: usage: kidash [options]
Import or Export a Kibana Dashboard
optional arguments:
-h, --help show this help message and exit
-e ELASTIC_URL, --elastic_url ELASTIC_URL
Host with elastic search(default:
http://127.0.0.1:9200)
--elastic_url-enrich ELASTIC_URL_ENRICH
Host with elastic search and enriched indexes
--dashboard DASHBOARD
Kibana dashboard id to export
--export EXPORT_FILE file with the dashboard exported
--import IMPORT_FILE file with the dashboard to be imported
--kibana KIBANA_INDEX
Kibana index name (.kibana default)
--list list available dashboards
-g, --debug
--data-sources DATA_SOURCES [DATA_SOURCES ...]
Data sources to be included
--add-vis-studies Include visualizations for studies](https://image.slidesharecdn.com/softdiveintogrimoirelab-180305160956/75/Soft-Dive-Into-GrimoireLab-Twitter-OSS-workshop-38-2048.jpg)
Uploaded byManrique Lopez
Soft Dive Into GrimoireLab. Twitter OSS workshop
The document discusses GrimoireLab, an open source software project for software development analytics. It provides an overview of GrimoireLab's history and main components for data gathering (Perceval), enrichment (ELK, SortingHat), and visualization (Kibiter, Sigils, Kidash). It also describes how to run GrimoireLab locally and explores some of its future directions. The presentation aims to provide community managers with tools for data science and visualization using GrimoireLab.
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Soft Dive Into GrimoireLab. Twitter OSS workshop
- 1. Deep Soft DiveInto GrimoireLab Manrique López @bitergia jsmanrique at bitergia.com https://speakerdeck.com/bitergia Twitter OSS internal workshop, Twitter Inc. offices, March 2018
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18. /run $ docker run-p 127.0.0.1:5601:5601 -v $(pwd)/credentials.cfg:/mordred-override.cfg -t grimoirelab/full Credentials file: [github] api-token = XXX
- 19.
- 20.
- 21.
- 22.
- 23. Main components Have funplaying with them!
- 24. /before_start NOTE: To avoidinstalling components one by one, to speed up training: $ python3 -m venv /tmp/grimoirelab $ source /tmp/grimoirelab/bin/activate (grimoirelab) $ pip install --upgrade pip setuptools wheel ... (grimoirelab) $ pip install grimoire-mordred
- 25.
- 26. /perceval Perceval github.com/chaoss/grimoirelab-perceval PyPI package: $pip3 install perceval From sources: $ git clone https://github.com/chaoss/grimoire-perceval.git $ pip3 install -r requirements.txt $ python3 setup.py install Docker: https://hub.docker.com/r/grimoirelab/perceval/
- 27. /perceval From command line: $perceval [-c <file>] [-g] <backend> [<args>] |--help | --version In your Python code: … from perceval.backends.core.<backend> import <Backend> … backend_repo = <Backend>(<params>) for item in backend_repo.fetch(): …
- 28. /perceval Backends: askbot Fetch questionsand answers from Askbot site bugzilla Fetch bugs from a Bugzilla server bugzillarest Fetch bugs from a Bugzilla server (>=5.0) using its REST API confluence Fetch contents from a Confluence server discourse Fetch posts from Discourse site dockerhub Fetch repository data from Docker Hub site gerrit Fetch reviews from a Gerrit server git Fetch commits from Git github Fetch issues from GitHub gitlab Fetch issues from GitLab gmane Fetch messages from Gmane hyperkitty Fetch messages from a HyperKitty archiver jenkins Fetch builds from a Jenkins server jira Fetch issues from JIRA issue tracker mbox Fetch messages from MBox files mediawiki Fetch pages and revisions from a MediaWiki site meetup Fetch events from a Meetup group nntp Fetch articles from a NNTP news group phabricator Fetch tasks from a Phabricator site pipermail Fetch messages from a Pipermail archiver redmine Fetch issues from a Redmine server rss Fetch entries from a RSS feed server slack Fetch messages from a Slack channel stackexchange Fetch questions from StackExchange sites supybot Fetch messages from Supybot log files telegram Fetch messages from the Telegram server
- 29. /perceval Write your ownbackends! github.com/chaoss/grimoirelab-perceval/tree/master/perceval/backends/core … class <Backend>(Backend): … @metadata def fetch(self): … … class <Backend>Client: … class <Backend>Command(BackendCommand): …
- 30.
- 31.
- 32. /grimoirelab-elk Grimoirelab ELK github.com/chaoss/grimoirelab-elk $p2o.py --enrich --index git_raw --index-enrich git -e http://localhost:9200 --no_inc --debug git https://github.com/chaoss/grimoirelab-perceval
- 33. /sortinghat Sorting Hat github.com/chaoss/grimoirelab-sortinghat Maintainsan SQL database with identities that can be merged in the same unique identity. For each unique identity, a profile can be defined: name, email, and other data. Each unique identity can be related to one or more affiliations, for different time periods.
- 34. /sortinghat Manrique López <jsmanrique@bitergia.com> JoséManrique López de la Fuente <jsmanrique@gmail.com> Manrique López <jsmanrique@gmail.com> jsmanrique jsmanrique@gmail.com jsmanrique@bitergia.com correo@jsmanrique.es jsmanrique jsmanrique@bitergia.com 02/2005 - 12/2010 CTIC 01/2010 - 12/2012 Andago 01/2013 - 06/2013 TapQuo 07/2013 - 12/2015 freelance (ASOLIF, CENATIC) 07/2013 - now Bitergia
- 35. /sortinghat github.com/chaoss/grimoirelab-sortinghat/blob/maste r/sortinghat/api.py … def add_identity(db, source,email=None, name=None, username=None, uuid=None): … def add_organization(db, organization): … def add_domain(db, organization, domain, is_top_domain=False, overwrite=False): … def add_enrollment(db, uuid, organization, from_date=None, to_date=None): … def edit_profile(db, uuid, **kwargs): … def merge_unique_identities(db, from_uuid, to_uuid): … def search_unique_identities(db, term, source=None): …
- 36.
- 37. /visualization Kibiter github.com/chaoss/grimoirelab-kibiter Sigils github.com/chaoss/grimoirelab-sigils Kidashgithub.com/chaoss/grimoirelab-kidash
- 38. /visualization (grimoirelab) $kidash --help usage: usage: kidash [options] Import or Export a Kibana Dashboard optional arguments: -h, --help show this help message and exit -e ELASTIC_URL, --elastic_url ELASTIC_URL Host with elastic search(default: http://127.0.0.1:9200) --elastic_url-enrich ELASTIC_URL_ENRICH Host with elastic search and enriched indexes --dashboard DASHBOARD Kibana dashboard id to export --export EXPORT_FILE file with the dashboard exported --import IMPORT_FILE file with the dashboard to be imported --kibana KIBANA_INDEX Kibana index name (.kibana default) --list list available dashboards -g, --debug --data-sources DATA_SOURCES [DATA_SOURCES ...] Data sources to be included --add-vis-studies Include visualizations for studies
- 39. /sigils github.com/chaoss/grimoirelab-sigils/tree/master/sch ema Data model usedfor each of the standard panels and indexes in GrimoireLab.
- 40. Playing with Data(bonus included) Data Science (sort of) for Community Managers
- 41.
- 42.
- 43. /manuscripts Manuscripts github.com/chaoss/grimoirelab-manuscripts $ pipinstall manuscripts $ manuscripts -d /tmp/report-result -u http://twitteross:**********@twitteross.biterg.io/data -n TwitterOSS --data-sources git
- 44.
- 45.
- 46.
- 47.
- 48.
- 49. /join #GrimoireLab in irc.freenode.net bit.ly/grimoirelab-mailmangithub.com/chaoss/grimoirelab-* (repositories beingmigrated from github.com/grimoirelab ) @GrimoireLab #GrimoireLab
- 50. Software Development Analytics foryour peace of mind chaoss.github.io/grimoirelab chaoss.community jsmanrique@bitergia.com CEO at Bitergia