Apache Yetus: Helping Solve the Last Mile Problem
Allen Wittenauer
Effective Machines
Apache: Big Data North America 2016-05
ApacheCon North America 2016-05
! email: aw @ effectivemachines.com, aw @ apache.org
! github: aw-was-here
! gitlab: _a__w_
! slideshare: allenwittenauer
! twitter: @_a__w_ ( 1 a 2 w 1 )
22016-05-12
3
4
https://www.flickr.com/photos/83633410@N07/7658225516/ 52016-05-12
! “How do these changes actually impact me?”
! “The method my code uses changed!”
! “The quality of this software is awful. Is it actually tested?”
62016-05-12
Apache Yetus is a collection of libraries and tools
that enable contribution and release processes
for software projects.
Release Doc Maker
92016-05-12
10
releasedocmaker
! Input
! One or more JIRA projects
! Version, multiple versions, or a range of versions
! Optional
! Title
! Project Name
! Linting
! Output
! Changes and Release Notes
! Per version
! Grouped by Type and Flags (Important, Incompatible)
! Markdown
112016-05-12
12
13
14
15
16
Audience Annotations for Java
What’s “safe” to use?
Compiler vs. Human
20
Audience
! Private
! Limited-Private
! Public
212016-05-12
Stability
! Stable
! Evolving
! Unstable
! Deprecated
222016-05-12
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class GlobFilter implements PathFilter
{...}
@InterfaceAudience.LimitedPrivate({"MapReduce"})
@InterfaceStability.Unstable
public class Progress {...}
import
org.apache.yetus.audience.InterfaceAudience
import
org.apache.yetus.audience.InterfaceStability
232016-05-12
JavaDoc/... Support
! ...audience.tools.ExcludePrivateAnnotationsStandardDoclet
! ...audience.tools.ExcludePrivateAnnotationsJDiffDoclet
! ...audience.tools.IncludePublicAnnotationsStandardDoclet
242016-05-12
Shelldocs
HADOOP-9902
## @description Print a message to stderr
## @audience public
## @stability stable
## @replaceable no
## @param string
function yetus_error
{
echo "$*" 1>&2
}
272016-05-12
28
smart-apply-patch
30
smart-apply-patch --plugins=github 
--github-user=apache --github-repo=orc 55
smart-apply-patch --plugins=jira HADOOP-1
smart-apply-patch https://my/cool/server/patch
Committer Mode (--committer)
! Only works with git format-patch created patches
! Always --commit
! Always --signoff
! Always --whitespace=fix
! TODO: Add more support for other patch formats (YETUS-158)
! TODO: Add --sign (YETUS-315)
312016-05-12
precommit (aka test-patch)
33
34
35
36
37
38
39
Driver
Personality Bug Systems Build Tools Tests
Language
Support
Static Analysis
Unit Tests
Test Log
Processing
40
! Compiled Language Support
– C/C++
– Java
– Scala
! Interpreted Language Support
– bash/ksh/sh: shellcheck
– perl: Perl::Critic
– Python: pylint
– Ruby: rubocop, ruby-lint
Tool Support (part 1)
! Build Tools
– ant
– autoconf
– cmake
– gradle
– maven
– make
– “nobuild”
! Bug Systems
– Bugzilla (RO)
– GitHub
– JIRA
412016-05-12
! Unit Test Formats
–ctest
–Junit
–TAP
! Runtime Support
–Automation (e.g., Jenkins)
–Docker
–Multiple JVMs
–Patch branches
–Per-instance maven repos
Tool Support (part 2)
! Specialized Tests
– Apache RAT
–@author
–Java checkstyle
–Javadoc
–Java findbugs
–maven site
–Scaladoc
–whitespace
–XML
422016-05-12
43
44
452016-05-12
test-patch --plugins=all 
--github-user=apache
--github-repo=tajo 
--jira-re='^TAJO-[0-9]+$'
--branch-default=master 
--buildtool=cmake
--basedir=tajo 
TAJO-1
Personality
462016-05-12
#!/usr/bin/env bash
personality_plugins "all"
function personality_globals
{
BUILDTOOL="maven"
PATCH_BRANCH_DEFAULT=master
JIRA_ISSUE_RE='^TAJO-[0-9]+$'
GITHUB_REPO="apache/tajo"
}
Running
test-patch --personality=file --basedir=tajo TAJO-1
test-patch --personality=file --basedir=tajo 
https://example.com/file-branch.patch
test-patch --personality=file --basedir=tajo /tmp/patchfile
smart-apply-patch --personality=file --basedir=tajo TAJO-1
472016-05-12
Bundled Sample Personalities
482016-05-12
49
qbt (quality build tool)
51
https://www.flickr.com/photos/83633410@N07/7658230838/ 522016-05-12
Administrivia
! Website: https://yetus.apache.org
! Mailing list: dev@yetus.apache.org
! Twitter: @ApacheYetus
! Current Release: 0.2.1 (2016-04-14)
! Yetus is an archaic synonym of the Cymbium genus of gastropods
! First ASF TLP with a ‘Y’
532016-05-12

Apache Yetus: Helping Solve the Last Mile Problem