Perceval is a software tool that gathers data from various sources and formats it consistently. It retrieves information like issues, commits, and other items from sources like GitHub. Perceval can be run from the command line or used as a Python library. It allows users to analyze software project data over time to answer questions about new contributors, bugs fixed, and changes in gender diversity. Perceval retrieves data and stores it in a standardized format to facilitate analysis of software projects and communities.

1. Perceval
Software Project Data at your will
Valerio Cosentino
@_valcos_
valcos@bitergia.com
https://speakerdeck.com/bitergia
FOSDEM, Brussels, 04-02-2018

2. Outline
Context
Perceval
Perceval in action
Explotation

3. /context

4. /context
Are there new contributors?

5. /context
How many bugs were fixed
past month?

6. /context
Does our gender diversity has
increased lately?

7. /perceval
Perceval gathers data for you

8. /perceval
Perceval gathers data for you

9. /perceval
Goal -> retrieve information* from data sources
...and many more
* information: collection of items (issues, commits, ...)

10. /perceval
Data source and Perceval data
{ "backend_name": "GitHub",
"backend_version": "0.13.0",
"category": "issue",
"data": {
"comments": 5,
"comments_data": {...}
"created_at": "2017-02-28T05:33:10Z",
....,
"id": 210691361,
"state": "closed",
"title": "..",
"updated_at": "2017-03-02T09:51:49Z"
},
"origin": "https://github.com/grimoirelab/perceval",
"perceval_version": "0.9.10",
"tag": "https://github.com/grimoirelab/perceval",
"timestamp": 1517479766.878609,
"updated_on": 1488448309.0,
"uuid": "77a42463d5d0a34b1d58006263b85909a9788b52" }
Data source
data
}
]Perceval
data

11. /perceval
Organization -> 3 actors
Backend Client
CommandLine

12. /perceval
Operations -> fetch & fetch-from-archive
Backend Client
CommandLine
Data source
Perceval
data
Data source
data
Archive

13. /perceval
Operations -> fetch & fetch-from-archive
Backend Client
CommandLine
Archive
Perceval
data
Data source
data

14. /perceval_in_action
As a program
As a Python3 library
$ perceval github grimoirelab perceval --from-date=2017-03-01 --api-token=5e3d7cc…
[2018-02-01 12:31:10,683] - Sir Perceval is on his quest.
[2018-02-01 12:31:11,712] - Getting info for https://api.github.com/users/jgbarah
[2018-02-01 12:31:12,069] - Getting info for https://api.github.com/users/acs
...
[2018-02-01 12:34:22,728] - Sir Perceval completed his quest.
from perceval.backends.core.github import GitHub
from_date = datetime.datetime(2017, 3, 1)
github = GitHub("grimoirelab", "perceval", api_token="5e3d7cc...")
for issue in github.fetch(from_date=from_date):
print(issue['data'])

15. /explotation

16. /explotation

17. /resources
grimoirelab/perceval
grimoirelab.github.io
youtube.com/watch?v=eH1sYF0Hdc8
@grimoirelab
@_valcos_