Exploring the GitHub Service Universe

5,871 views

Published on

>>> View this presentation online at http://github-service-universe.kimminich.de/ <<<

PDF version of the slide deck for my JavaLand 2015 talk "All-round careful Software Development with GitHub Services"

Tools for unit testing, building applications, analyzing software quality and planning release scopes are an essential aspect of modern software development. With GitHub and "pluggable" external services there are lots of options to move these aspects into "the Cloud". For open source projects this is a viable alternative to on-premise solutions. In this talk I will present and demonstrate the CI lifecycle of some of my recent projects hosted on GitHub where I tried to integrate modern tools (e.g. Gradle, npm, bower) and external services (e.g. Travis-CI, Code Climate, Coveralls, HuBoard, AmazonSNS, NMA). The benefits and limitations of those services will be honestly illuminated. I am not affiliated with any of the providers mentioned, so this talk will not end up as a marketing show! Instead, the audience is supposed go out of this talk with some new things to try out with their own GitHub projects while hopefully being able to avoid some of the ramp-up difficulties.

Published in: Technology
0 Comments
19 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,871
On SlideShare
0
From Embeds
0
Number of Embeds
2,334
Actions
Shares
0
Downloads
64
Comments
0
Likes
19
Embeds 0
No embeds

No notes for slide

Exploring the GitHub Service Universe

  1. 1. Exploring the GitHub Service UniverseAll-round carefreeful Software Development with GitHub Services Created by /Björn Kimminich @bkimminich Follow @bkimminich   Tweet 1   Follow @bkimminich 35   Star 1
  2. 2. Björn Kimminich Division Architect & Security Officer @ Lecturer for Software Development @ Member & Contributor @ Master of the (highly recommended) Kuehne + Nagel (AG & Co.) KG Nordakademie gAG Open Web Application Security Project Code School Git Path
  3. 3. Disclaimer This is not a marketing talk. It is a compilation of personal experience gathered while working on two of my own public repositories. I am neither affiliated with nor paid or otherwise reimbursed by GitHub or any other company behind the products mentioned in this presentation. No product evaluation or comparison study of any kind was conducted prior to choosing the services presented here. Only services that are entirely free for open source projects are presented in this talk.
  4. 4. Agenda A very brief introduction to GitHub & Showcase repositories & 15 valuable GitHub Services in practical use GitHub WebHooks Services Hooks kata-tcg juice-shop
  5. 5. GitHub Collaborative repository hosting service.Git http://github.com/
  6. 6. If you've never heard about GitHub before...
  7. 7. You don't trust... ...cloud service providers with your code? Fact #1: GitHub offers free hosting of public Git repositories!
  8. 8. You are still... ...on ?Subversion Fact #2: Offering a sophisticated web-based graphical interface, GitHub still remains 100% compatible with the git CLI.
  9. 9. Or even... ... ?CVS Fact #3: GitHub supports collaborative development through e.g. forking and pull requests.
  10. 10. Not really... ...still or ?RCS SCCS Fact #4: GitHub (optionally) adds an issue tracker, wiki and project page to each repository.
  11. 11. Or seriously... ...no version control system at all? Fact #5: Repository statistics and social extras like Feeds, Followers & Favorites are part of GitHub.
  12. 12. WebHooks & Service Hooks Individual & Third Party Service Integration https://developer.github.com/webhooks https://github.com/github/github-services
  13. 13. Wait a moment! What are ?WebHooks Simply put: User-defined HTTP callbacks. More specifically: HTTP POSTs that occur when something happens. So basically a simple event-notification via HTTP POST.
  14. 14. WebHooks on GitHub Subscription to events on GitHub.com Used to integrate individual applications or tools Installation on or Types & payloads mirror the organization repository level Event API
  15. 15. Service Hooks on GitHub Service Hooks can only be installed on repositories Only one Service Hook per integrator Supported events depend on service implementation Services come with their own unique configuration
  16. 16. Account Level Integration Close integration with GitHub by demanding repo or account access Do not require any manual setup by the user on the GitHub page Configured by the service provider via its own user interface
  17. 17. Integration Chain 3rd party does not integrate directly with GitHub Instead integration with APIs of other service providers Very useful in Continuous Integration context. Example:
  18. 18. What way of Integration should I use? GitHub recommends WebHooks for all new integrations If required use to manage authorization The existing is not accepting any new services OAuth github-service repo
  19. 19. Adding a WebHook to a repository
  20. 20. Repository WebHook Event Types Organization WebHooks send events for all repositories in that organization. New events for repository creation and team membership are also available on organization-level.
  21. 21. Service Hook example: Twitter
  22. 22. WebHooks & Services in Settings menu
  23. 23. Adding Twitter as a Service
  24. 24. The Twitter Service configuration
  25. 25. Authorizing GitHub to send tweets
  26. 26. Setting tweet format and trigger branch
  27. 27. Sending a test payload The Test Service function triggers the real Service once for the most recent commit!
  28. 28. The published push notification tweet
  29. 29. The Showcase Repositories
  30. 30. Kata TCG Code Kata for a two-player trading card game loosely based on Hearthstone - Heroes of Warcraft https://github.com/bkimminich/kata-tcg
  31. 31. Sample Implementations Java (JUnit, Hamcrest, Mockito) Groovy (Spock) Javascript (Karma, Jasmine) Clojure (work in progress...)
  32. 32. Polyglot Build Multi-module build using language specific plugins to build & test all sample implementations in one execution. Gradle
  33. 33. Juice Shop An intentionally insecure Javascript Web Application http://bkimminich.github.io/juice-shop/
  34. 34. Technology Stack
  35. 35. Build Setup
  36. 36. 15 valuable GitHub Services in practical use in &kata-tcg juice-shop
  37. 37. Notifications
  38. 38. NMA Platform for delivering push notifications from virtually any application to an Android device. http://www.notifymyandroid.com/
  39. 39. Install free* NMA Android App *The number of receivable notifications per day is limited. Unlimited premium account available via in-app purchase.
  40. 40. Generate API Keys for each Notifier
  41. 41. Enter API Key in NMA Service config For convenience you can use the same API Key for all your GitHub repositories.
  42. 42. Notification on every push to GitHub
  43. 43. Amazon SNS Simple Notification Service enables applications, end-users, and devices to instantly send and receive notifications from the cloud. http://aws.amazon.com/sns/
  44. 44. Create global topic in SNS Dashboard
  45. 45. Create Subscriber for SNS Topic
  46. 46. Configure SNS Service in GitHub For convenience you can use the same SNS Topic for all your GitHub repositories.
  47. 47. Receiving sexy* JSON email on a push *If you're not so much into JSON I'm sure you'll find a WebHook subscriber that is...
  48. 48. Continuous Integration
  49. 49. Travis-CI Hosted continuous integration service providing different runtimes for different languages. https://travis-ci.org/
  50. 50. Last result for each repo & build history
  51. 51. Detailed build log for failure analysis
  52. 52. Build results per pull request
  53. 53. Build results per commit of a PR
  54. 54. Warning about ongoing PR build Of course there is also a warning when the PR build failed.
  55. 55. Watching the console of the running build
  56. 56. Merging PR with a warm All is well-feeling
  57. 57. Build configuration via .travis.yml file
  58. 58. Saucelabs Automated cross-browser and mobile testing in the cloud for CI. https://saucelabs.com/
  59. 59. List of last test run results for juice-shop
  60. 60. Overview of the last failed test run
  61. 61. Live screencast of ongoing e2e test run
  62. 62. Triggering Saucelabs from Travis CI The secure tokens are your SAUCE_USERNAME and SAUCE_ACCESS_KEY.
  63. 63. Quality Assurance
  64. 64. Coveralls Works with continuous integration servers to provide test coverage history and statistics. https://coveralls.io/
  65. 65. Coveralls repository dashboard
  66. 66. Coverage of latest builds of a repo
  67. 67. Coverage per file for specific build
  68. 68. Drilldown into file coverage
  69. 69. Integration into PR overview screen
  70. 70. Passing test results to Coveralls
  71. 71. Setup NMA email* on any coverage drop For each new API key apikey@nmamail.net that can be used for custom notifications. NMA automatically creates an email address
  72. 72. Notification on a ( ) coverage dropforged
  73. 73. Coverity Provides software quality and security testing solutions. http://www.coverity.com/
  74. 74. Coverity's Analysis Dashboard
  75. 75. Details on a specific issue
  76. 76. Coverity scan setup on a separate branch Coverity to 1-3 builds/day (and 2-12 builds/week) depending the project's LOC.limits the build submission frequency
  77. 77. Codeclimate Automated code review for Ruby, JS, and PHP providing feedback on code quality and test coverage. https://codeclimate.com/
  78. 78. Quality overview in Codeclimate Feed
  79. 79. Quality metrics and test coverage per file
  80. 80. Code smells identified by Codeclimate
  81. 81. Coverage details show a missed function
  82. 82. Send merged data to CodeclimateLCOV
  83. 83. Automatically open issues for code smells
  84. 84. Refactoring issue created by Codeclimate
  85. 85. Dependency Management
  86. 86. Versioneye Notification System for Software Libraries showing outdated dependencies in different supported project files. https://www.versioneye.com/
  87. 87. Versioneye Project Overview Supported Languages: Java - Ruby - Python - PHP - Node.js - JS - Objective-C - Clojure - CSS - R
  88. 88. Dependency details on project level
  89. 89. Graph with all indirect dependencies shows all the dependencies brought into the JS implementation of kata-tcg by the used testing libraries!This graph
  90. 90. Gemnasium Monitoring of project dependencies and alerts for updates and security vulnerabilities. https://gemnasium.com/
  91. 91. Dependency status overview for all repos
  92. 92. Outdated Jasmine test dependencies
  93. 93. Email with security alert
  94. 94. David-DM Watching your dependencies.Node https://david-dm.org/
  95. 95. Automatically discovered Node.js projects Unfortunately David-DM (v9.0.0) can only discover Node.js projects with a package.json in the repository root folder.
  96. 96. Dependency status with security advisory A module without security warnings might still contain undiscovered vulnerabilities! On the other hand proven vulnerabilities of a module might be irrelevant in the context it is used in.
  97. 97. Security vulnerability details David-DM cooperates with to determine and link to vulnerabilities.Node Security Project
  98. 98. Continuous Deployment
  99. 99. Heroku Build and Run Your Apps, Your Way. https://www.heroku.com/
  100. 100. of Juice ShopHeroku instance Heroku .offers a free small instance per personal application
  101. 101. Application status dashboard Heroku supports Ruby, Node.js, Python, Java, and PHP.
  102. 102. Application deployment history
  103. 103. Setting up deployment in .travis.yml By default only a successful build of the master branch triggers a deployment.
  104. 104. Docker Open platform for distributed applications for developers and sysadmins. https://docker.com/
  105. 105. Autobuild Repository on Docker Hub
  106. 106. Activated Docker Service Hook on GitHub
  107. 107. The Dockerfile of Juice Shop
  108. 108. Collaboration
  109. 109. HuBoard Lightweight Kanban Board offering instant project management for GitHub issues. https://huboard.com/
  110. 110. Kanban Board based on GitHub issues
  111. 111. DnD for priorization and process flow
  112. 112. Simple creation and tagging of story cards
  113. 113. Authorizing access to GitHub repos
  114. 114. Let HuBoard setup the GitHub integration
  115. 115. Service Hook generated by HuBoard
  116. 116. Bountysource Funding platform for open-source software where users can create/collect bounties and pledge to fundraisers. https://www.bountysource.com/
  117. 117. Overview of issues to place bounties on
  118. 118. Picking to place a bounty onan issue
  119. 119. Placing a 10$ bounty for a new logo
  120. 120. The new bounty in the Activity feed
  121. 121. Issue augmented with bounty information
  122. 122. Developer starting to work on issue
  123. 123. Developer claims bounty for closed issue
  124. 124. Approved and paid bounty for new logo
  125. 125. Gitter Chat. For GitHub. https://gitter.im/
  126. 126. The official Gitter chatroom of Juice-Shop Disclaimer: Chatroom might appear more desolated on screenshot than in reality.
  127. 127. Activity sidebar populated via WebHooks
  128. 128. GitHub-side of the Gitter-WebHook With granted repository access Gitter will setup its WebHook on GitHub automatically.
  129. 129. One final takeaway If the services you are using offer  status  badges  for your README.md... ... use  them ... ...on  every  occasion ...
  130. 130. ...because they are  just  cool !
  131. 131. Thanks for your attention! by  Björn  Kimminich  / kimminich.de These slides are publicly available on and .GitHub Slideshare
  132. 132. Q&A
  133. 133. Credits - The HTML Presentation Framework - Turns text into UML sequence diagrams - The official Octocat gallery reveal.js js-sequence-diagrams GitHub Octodex Copyright (c) 2015 Björn Kimminich

×