Originally presented at Collab Summit 2016, this talk covers the use of GHTorrent to gather and analyze public repo and community data from GitHub. We talk about using Azure Data Lake as well as how you can set up this infrastructure yourself.

1. GitHub InsightsUnderstanding Open Source
@jeffmcaffer
Microsoft Open Source Programs Office

2. Microsoft and open source

3. Microsoft repo trends

4. Business insights

5. Operational insights

6. Project health: Pull request performance

7. User Geolocation

8. Factors affecting Pull Request merge

9. Approach
Data for the masses

10. GitHub by the numbers (Early 2016)

11. Approach
http://ghtorrent.org

12. How does it work?
http://api.github.com/events

13. Example event (condensed)
https://api.github.com/users/Cephei
https://api.github.com/repos/PowerDMS/Owin.Scim
https://api.github.com/repos/PowerDMS/Owin.Scim/commits/c751014f634d73e0b72f78a53c8cf137888b309d
https://api.github.com/orgs/PowerDMS

14. Entities

15. GHTorrent by the numbers

16. Using the data

17. Using the data: Hosted
http://ghtorrent.org

18. Using the data: Self-service
https://github.com/ghtorrent/ghtorrent-webhook

19. Using the data: Azure Data Lake

20. Wrap-up

21. Privacy

22. Futures

23. Questions