How and Why you can and should Participate in Open Source Projects (AMIS, Software Engineering SIG, February 2022)

Lucas Jellema
Architect & CTO AMIS | Conclusion
How and Why
you can and
should
contribute to an
open source
project?!

Publiek
Well known Open Source Products
Linux
Apache Http
Server
Apache
Tomcat
Apache
Struts
Hibernate
MySQL GlassFish Ant Maven
Hudson
Jenkins
JUnit
Spring
Framework
Log4J Wordpress Kubernetes
Apache
Kafka
Apache
Spark
VS Code Eclipse Ubuntu
Azure
Functions
Firefox Magento Diagrams.net OpenOffice
How and Why participate in an Open Source Project
I am
because
we are

Publiek
Quiz
• How many open source products did you inspect the source of?
• For how many open source products did you change code?
• and submitted it?
• For how many did you:
• file issues & enhancement requests
• contribute to documentation
• submitted examples or test cases
• About how many did you
• present
• blog
• Tweet?
• How many open source products did you make use of?
• today?
• this month?
• in your professional career

Publiek
Open Source code analyzed and contributed

Publiek
Contributions to or about open source projects

Publiek
Number of open source products used in your career

Publiek
Why Participate?
• To give back/make a contribution (“moral debt”)
• Make the product better
• Influence the product
• Plug your own product
• Help grow and spread, thrive and persist
• Learn about and understand the product (technically and strategically)
• To increase your own personal status
• For fun, thrill
• Meet new people, belong to a team
• Grow as a software engineer
• learn from process, tooling, communication
• learn from studying examples and emulating them
• learn from review feedback and interaction

Publiek
Why not?
• I think lack the skills to be of value
• Scary to be exposed to public scrutiny
• Don’t know how to
• I do not really care
• I have a day job
• I do not particularly like IT
• I have other things to do

Publiek

Publiek
Terms to come to grips with
LGTM
PR
Clone
Fork
Merge
Push
Pull
Origin
Commit (verb & noun)
Branch
Milestone
Issue
upstream
repository
review
alpha
beta
stable
Git
GitHub
GitHub Action
GitHub Code Space
Dev Container
Markdown
Founder
Maintainer
Committer
Author
Contributor
unit test
conformance test
end 2 end test
integration test
Hugo
Go
Go Mod Tidy
Go Lint
Go Test
fumpt

Publiek
Types of Open Source Projects
Code dumped by commercial vendor
One person effort go-ora
Project with wonderful past but
bleak future
Apache Struts, Apache MyFaces, Apache
Xalan, Apache Ant
Closely controlled by and mixed up
with a single commercial entity
Elastic Stack, OpenShift, VirtualBox, VS Code,
MySQL
Strategic support [and interest] from
multiple commercial vendors
OpenJDK, Kubernetes, Apache Spark
Unaffiliated project with large
committer community
Python, Linux, NodeJS

Publiek
OutSystems “donating open source”

Publiek
Assess status of an Open Source Project
• # issues (open and closed) (bug reports, enhancement requests)
• # pull requests (open and closed) – these constitute the actual evolution of the product
(although the number does not prove the value and scope of the changes)
• # authors (how many different contributors have made a substantial contribution)
• who is using the product/who have dependencies and direct stakes?
• for example (large) commercial players? other (big) open source products?
• # forks (how many times have the repositories been forked
• this would be done for local experimentation and customizations and for making contributions
• release history (frequency of new releases and delta between releases)
• # commits (individual code changes
• # stars (how many "likes" have been awarded by the community)
• # threads on StackOverflow (recently); Google Trends - # recent searches
• visibility through presentations on conferences, articles on Medium, tweets
• activity on the community discussion forum (such as Slack or Discord); #members, #threads,
#messages in recent times
• # different people participating in issues and PRs
• immerse oneself in a project for a few weeks

Publiek
Dapr
• Runtime
• Personal Assistant for Applications
• Distributed Application Runtime
• Microservice architecture
• Decoupling
Code Café Speed Dating - December 2021 14

Publiek
Dapr.io
• Launched by Microsoft (2019)
• Application runtime framework –
supporting cloud native and serverless
• Every application gets a uniform Personal Assistant
that takes care of common tasks
• remember (take note and reproduce)
• keep log
• restrict access
• handle incoming
• route outgoing
• interact with
other PAs
Application
Microservice or
Monolith
Personal
Assistant
(Dapr)
Personal
Assistant
(Dapr)
Personal
Assistant
(Dapr)
Personal
Assistant
(Dapr)
Personal
Assistant
(Dapr)
Personal
Assistant
(Dapr)

Publiek
Dapr.io
• Personal Assistant is Dapr Side Car
• companion process or side car container in Kubernetes Pod
• Out of the Box
support for
• local products &
cloud services
• Kafka, AWS SQS
Azure Event Hub,
GCP Pub/Sub,
RabbitMQ
• DynamoDB, RDS,
CosmosDB,
PostgreSQL,
Redis, MySQL,
MongoDB,
Zookeeper
• SMTP, Twillio,
Twitter, Consul
= Custom Application

Publiek
Dapr Statestores
• No Oracle anywhere in Dapr
• Introducing Oracle based components has value
• for Dapr
• for Oracle
• for me (to use Dapr in Oracle context)
• And: I can bring value as bridge between Dapr, microservices & Oracle & OCI
• at last my chance to contribute to OSS!
• One little obstacle: Dapr is Go – I do not know Go.
• My Objective: contribute Dapr State Store component for OCI Object Storage
• and subsequently Oracle Database/ATP, OCI Streaming, Oracle NoSQL, …
• how to go get going?

Publiek
Observability
Prometheus AppInsights Jaeger
Zipkin
Bindings
& Triggers
State
Stores
Secret
Stores
PubSub
Brokers
Dapr Architecture
Dapr main
state store
interface
pub/sub
interface
binding
interface
secret
interface
register all
components
(per type)
observable
interface
do something for me – store
or retrieve state, subscribe
to topic (and callback with
messages), invoke service,
retrieve secret
components.yaml

Publiek
How to contribute to Dapr.io?
• Analyze Dapr.io website & GitHub
• Repositories, subprojects
• Discord community communication platform
• Developer instructions
• process & tooling
• technical docs?
• existing code & tests
• PRs and review conversations
• submission steps

Publiek
Dapr Community

Publiek
project X repo
lucasjellema/
project X
fork
1
my laptop
2
local-dev/
project X
branch
3
Create Docs,
Code, Example, Test
4
Commit Changes
5
push
6
c
l
o
n
e
PR
7
my-contribution-branch
my-contribution
-branch
Fork-Clone-Branch-Code-Commit-Push-PR
Milestone
8
my GitHub account

Publiek
dapr/
components-contrib
Issue: Proposal
1
lucasjellema/
components-contrib
fork
2
my laptop
3
dapr-dev/
components-contrib
branch
4
Create Test & Code
5
Commit Changes
(with sign-off)
6
push
7
dapr/
docs
Issue: Proposal
9
c
l
o
n
e
PR
8
oci-objectstorage-state
oci-objectstorage
-state
Milestone
10

Publiek
Good First Issue

Publiek
My Contribution: a new State Store component based on
Oracle Cloud Object Storage Service
• State is stored as object in a special bucket on OCI Object Storage
Object
Storage
Custom
State Store
Component
OCI Object
Storage
set & get state
components.yaml components.yaml
set & get state

Publiek
Create Issue to Propose Change
https://github.com/dapr/components-contrib/issues/1400

Publiek
Encouragement from one of the founders himself

Publiek
Fork Repository
dapr/
components-contrib
lucasjellema/
components-contrib
fork

Publiek
Repository Fork in my GitHub account
lucasjellema/
components-contrib

Publiek
Set up local environment
• WSL2 + Ubuntu
• Go Runtime
• VS Code with Go extensions
• Clone Repository Fork
lucasjellema/
components-contrib
my laptop
dapr-dev/
components-contrib
c
l
o
n
e

Publiek
35
My
Application
Object
Storage
tenancy, user,
fingerprint,
private key
create bucket
create object
What I need to develop ….
OCI Go
SDK

Publiek
Create branch to develop on
my laptop
dapr-dev/
components-contrib
branch
Create Code Commit Changes
(with sign-off)

Publiek
Create Test, Create Code

Publiek
Create Test, Create Code
• Two levels of testing
• Unit Testing (with Mock OCI interactor)
• Integration Testing (actually talking to OCI environment)

Publiek
Code Formatting and Linting
• Apply formatting rules for uniform code layout
• [also] to facilitate compare and merge
• Check Code Quality against rules
• formatting, commenting, complexity of functions, variable naming,
(un)reachable code

Publiek
Add, Commit, Push
• Add changed files to change set
• Commit change set
• with Sign Off in message
• Push Changes - to Repository Fork
on GitHub
lucasjellema/
components-contrib
dapr-dev/
components-contrib
Create Test & Code Commit Changes
(with sign-off)
push
c
l
o
n
e
oci-objectstorage
-state

Publiek
lucasjellema/
components-contrib
dapr-dev/
components-contrib
Create Test & Code Commit Changes
(with sign-off)
push
c
l
o
n
e
oci-objectstorage
-state

Publiek
Changes have landed in GitHub – branch in forked repo

Publiek
Next: the big PR!
fork
push
c
l
o
n
e
PR
my-contribution
-branch
my GitHub account
lucasjellema/
components-contrib
dapr/
components-contrib

Publiek
First: the big PrepaRe!

Publiek
First: the big PrepaRe – Fetch from Upstream

Publiek
Create the Pull Request for merging my branch to the master

Publiek
Next: the big PR!

Publiek
Initial feedback on PR

Publiek
Automated Checks – did not all succeed

Publiek
Extend PR with additional commits

Publiek
Waiting….
• The automated checks are not triggered automatically

Publiek
Enter PR conversation

Publiek
Wait for OK

Publiek
Tests are broken - not my code

Publiek
At last! PR is Merged
now as Contributor my commits will trigger the automated checks

Publiek
Entering the Community

Publiek
Extend Documentation
• Issue – link to component issue
• Fork dapr/docs repository
• Clone
• Install Hugo Server
• Create Branch
• Make changes, commit, push
• Create PR
dapr/
docs
Issue: Proposal
PR

Publiek
dapr/
components-contrib
Issue: Proposal
1
lucasjellema/
components-contrib
fork
2
my laptop
3
dapr-dev/
components-contrib
branch
4
Create Test & Code
5
Commit Changes
(with sign-off)
6
push
7
dapr/
docs
Issue: Proposal
9
c
l
o
n
e
PR
8
oci-objectstorage
-state
Milestone 1.6
10

Publiek
Done?
• Yes
• Now wait for milestone 1.6 to be released

Publiek
Dapr 1.6 Release Day – January 25, 2022

Publiek
Party? Kudos?

Publiek
The Effort of Setting up and
Running an Open Source Project
• Vision, High level design, architecture, roadmap, license
• Attract, organize, facilitate a contributor community
• Guidelines, Developer How-Tos, DevContainer
• Code of Conduct
• Community Communication – Discord, Slack
• Manage people (egos, ambitions, emotions, opinions)
• Involve vendors
• Design and Implement structure and automated processes for software engineering
• GitHub Set Up – Repositories, Issues, branching strategy
• Automated Code Quality checks and End to End tests – GitHub Actions
• Active management of Issues, Pull Requests, Tags (milestones), Dependencies
• Release process and mechanism
• Attract and Foster the User Community
• Quality & Documentation (activity, bugfixing)
• Website
• Content
• “noise/buzz”
• Provide support to users
• Build relations (vendors, other projects, VIPs, analysts)

Publiek
Dapr.io has dozens of committers,
100s of contributors

Publiek
Dapr.io has dozens of components

Publiek
On the essence of automation, human attention and civil
conversation

Publiek
On the essence of automation, human attention and civil
conversation (part 2)

Publiek
What happened since 25th January?
• Improvements in OCI Object Storage
state store component
• Introduced new Oracle Database state
store component
• inspired by the existing PostgreSQL
state store component
• small contribution to sijms/go-ora
project (Go driver for Oracle Database)

Publiek
72
Custom
State Store
Component
Oracle
Database
set & get state
components.yaml components.yaml
set & get state
host, port
sid, username,
password,
tableName

Publiek
PR was merged on February 8th

Publiek
Personal Findings
• Learning experience
• I can play at this level
• Join, participate, contribute
• Patience (time zones and more)
• Complexity of managing an open source
project
• fragility

Live Open Source Contribution
Flashback History
in Oracle Database State Store

Publiek
Live Open Source Coding:
Let’s enrich the Oracle Database State Store Component
• Idea
• Issue – Feature Proposal
• Issue – Documentation modification
• (Fetch Upstream, Pull latest)
• Create Test
• Create Code (run test, run lint)
• (Fetch Upstream, Pull latest)
• Create Branch
• Commit changes
• Push
• Create Pull Request
• wait for automated checks to give initial verdict
• (refine based on failed checks, commit, push)
• wait for human reviewers to give feedback

Publiek
Idea: Flashback Feature - hints to retrieve historical state
• Allow the use of “flashback hints” in the metadata sent in Get requests; these hints
instruct the state store to return for a specific key
• previous generations or versions of a state element
• the state as it existed X seconds ago
• the state as it existed at a specific timestamp in the past
possibly even return the state from before it expired or was deleted
• The response will provide metadata to indicate
• what generation of the state is returned
(current, previous, one before previous)
• when the state returned was created and when it ceased to exist
• if the element state element for this key no longer exists (expired or deleted)
• If the requested history cannot be returned, no data is returned (and no error)

Publiek
Idea: flashback– hints in get-requests
• Proposed Metadata in Get requests
• flashback-generation – values 0 (for current value), -1 (for previous value), -2
(for the one but previous value), -3...-X – to ask for a specific generation of
the state under the specified key
• flashback-atTime – a UTC timestamp in the past - to ask for the value of the
state under the specified key at a specific moment in the past
• flashback- age– a positive integer - to ask for the value of the state under the
specified key at the specified number seconds ago
• flashback-allow-deleted – true or false – to indicate whether a value should
be returned if the key currently is deleted or expired from the state store

Publiek
Idea: flashback– metadata in get-responses
• Proposed Metadata in Get responses
• flashback-generation – values 0 (for current value), -1 (for previous
value), -2 (for the one but previous value), -3...-X – to indicate the specific
generation of the state returned in the response
• flashback-startTime – a UTC timestamp in the past – to indicate the time
at which the state that is returned was created
• flashback-endTime – a UTC timestamp in the past – to indicate the time at
which the state that is returned was updated, deleted or expired
• flashback-deleted – true or false – to indicate whether the key is deleted
or expired from the state store at the present moment

Publiek
Issue – Feature Proposal
dapr/
components-contrib
Issue: Proposal
10
issue

Publiek
Issue – Documentation modification

Publiek
(Fetch Upstream, Pull latest)
• To start from the latest code on the master branch
• and prevent ugly merge operations later on
• Fetch Upstream – to get current latest from Upstream repo to my Fork
• In my Local Environment – pull to get the latest commit
dapr/
components-contrib
lucasjellema/
components-contrib
fetch
upstream
my laptop
dapr-dev/
components-contrib
p
u
l
l
10

Publiek
Create Test – Unit Test
Label Actions *) Outcome
flashback-generation is 0 or negative
integer
get – flashback-generation:24 error (incorrect value - positive integer
- for flashback-generation)
flashback-age is zero or positive
number
get – flashback-age:-3 error (incorrect value - negative value -
for flashback-age)
flashback-atTime is valid timestamp get – flashback-atTime:3 error (incorrect value – not a valid
timestamp - for flashback-atTime)
flashback-atTime is valid timestamp in
the past
get – flashback-atTime:2050-01-
31T23:59:59
error (incorrect value – future
flashback-atTime is valid timestamp in
the past after 10th November 2009 at
23:00 UTC
get – flashback-atTime:2009-11-
10T22:0:0
error (incorrect value – too old
flashback-generation is 0 or negative
integer
get – flashback-generation:XYZ error (incorrect value – not a number -
for flashback-generation)
flashback-age and flashback-atTime
are both set
get – flashback-age:50, flashback-
atTime:2009-11-10T22:0:0
error (only one flashback metadata
can be set – to prevent conflicts)
*) a random key is generated for each test and used for all get, set and delete operations

Publiek
Create Test –Integration Test
Label Actions *) Outcome
Not found returned for non
existing generation
set, get – flashback-generation:-1 not found
Value returned for existing
generation
set, get – flashback-generation:0 value returned
Value returned for existing
generation
set, set, get – flashback-generation:-1 first (previous) value returned
Original value returned after
delete
set, delete, get – flashback-generation:-1; flashback-allow-
deleted: true
value returned + flashback-
deleted meta data
Older generation value returned set, set, set, set, get – flashback-generation:-3 first value returned
Value from past timestamp set, wait one second, set, set, get – flashback-age:-0.5 first value returned
Value from before first
timestamp cannot be found
set, wait one second, set, set, get – flashback-age:-2.5 not found
Value from past timestamp set, <time>, wait one second, set, set, get – flashback-
asOf:<time>
first value returned
Value from before first
timestamp cannot be found
<time>, wait, set, set, set, get – flashback-asOf:<time> not found error
*) a random key is generated for each test and used for all get, set and delete operations

Publiek
Run Test
• Add flashback test-cases to unit test and integration test
• Without having touched the state store component’s code – run the test
• the run should be successful – but [most] new tests will fail
• our objective is clear: change the code to make all tests succeed
my laptop
dapr-dev/
components-contrib
add test
cases

Publiek
Create Code – Run Test – Run Lint
• Implement the desired functionality
• by making all tests succeed
• Run gofumpt and lint to prepare code for automated checks that will run
when the PR is submitted
• fix relevant rule violations
my laptop
dapr-dev/
components-contrib
add code implementing
features & satisfying test cases

Publiek
Create Branch, Commit Changes, Push
• Create a new branch flashback-in-oracle-database-state-store based on master
• Switch to branch
• Add changes and commit [with sign-off]
• Push (publish branch to repository fork on GitHub, include local commits)
lucasjellema/
components-contrib
my laptop
dapr-dev/
components-contrib
branch
Commit Changes
(with sign-off)
push
flashback-in-oracle-
database-state-store

Publiek
Create Pull Request
• Create PR for feature branch to upstream master – with link to issue
• wait for automated checks to give initial verdict
• refine based on failed checks, commit modifications, push to add to PR
• wait for human reviewers to give feedback
lucasjellema/
components-contrib
my laptop
dapr-dev/
components-contrib
dapr/
components-contrib
PR
issue

Publiek
Create – Commit – Branch - Push – PR Documentation
• Produce the documentation that describe the new behavior of the Oracle
Database state store component
• Commit on a feature branch
• Push branch and changes to docs repository fork
• Create a PR to merge to the 1.8 documentation branch
lucasjellema/
docs
Branch, Commit, Push
dapr/
docs
PR

Publiek
Uitnodiging
• Heb jij zin om mee te werken aan Dapr componenten?
dapr/
components-contrib
AMIS-services/
components-contrib
fork
c
l
o
n
e
c
l
o
n
e
feature
branch
c
l
o
n
e

Publiek
Uitnodiging
• Heb jij zin om mee te werken aan Dapr componenten?
dapr/
components-contrib
AMIS-services/
components-contrib
fork
c
l
o
n
e
c
l
o
n
e
PR
pull, commit,
push
feature
branch
feature dev
branch
pull, commit,
push
c
l
o
n
e
PR

The End

Publiek
X
dapr/
components-contrib
Issue: Proposal
1
lucasjellema/
components-contrib
fork
2
my laptop
3
dapr-dev/
components-contrib
branch
4
Create Code
5
Commit Changes
(with sign-off)
6
push
7
dapr/
docs
Issue: Proposal
9
c
l
o
n
e
PR
8
oci-objectstorage
-state
Milestone
10

Publiek
Brief introduction on Open Source Projects
• types
• organization
• roles
• process
• tools/infra (GitHub)
• aliveness (see Yammer thread)

How and Why you can and should Participate in Open Source Projects (AMIS, Software Engineering SIG, February 2022)

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to How and Why you can and should Participate in Open Source Projects (AMIS, Software Engineering SIG, February 2022)

Similar to How and Why you can and should Participate in Open Source Projects (AMIS, Software Engineering SIG, February 2022) (20)

More from Lucas Jellema

More from Lucas Jellema (20)

Recently uploaded

Recently uploaded (20)

How and Why you can and should Participate in Open Source Projects (AMIS, Software Engineering SIG, February 2022)

Editor's Notes