Open MPI Project
June 2015
Updated Version Numbering Scheme
and Release Planning
Table of contents
• These slides cover several related topics:
1. Open MPI’s new version numbering scheme
2. Transition plan to the new version numbering
3. Release planning roadmap
4. The bottom line (TL;DR)
• Let’s jump right in…
2
Before July 2015…
• Open MPI used an “odd / even” numbering
scheme
 1.odd: “feature” series
 1.even: “stable” series
• But it’s not working out as well as we’d like
3
One problem
• Very few users actually use the “odd”
versions
 Users equate “odd” with “unstable”
• As a direct result:
 New features don’t get real-world tested
 …until the “even” releases
4
Another problem
• Users want new features faster
 A “stable” series (intentionally) does not
receive new features
• As a direct result:
 New features take a long time to get to users
5
Let’s fix that
6
Goodbye odd / even scheme!
7
New version numbering scheme
• Open MPI will (continue to) use a “A.B.C”
version number triple
• Each number now has a specific meaning:
This number changes when backwards
compatibility breaks
This number changes when new features are
added
This number changes for all other releases
A
B
C 8
Examples
• Pretend we’re in the future
 The current Open MPI release is v3.4.2
• What will be the next release number?
• Let’s look at a few cases…
9
Example 1
• Current release: v3.4.2
• Situation:
 Bugs are fixed
 No new features are added
 Backwards compatibility is preserved
 Next release will be v3.4.3
10
Example 2
• Current release: v3.4.2
• Situation:
 Bugs are fixed
 User-noticeable new features are added
 Backwards compatibility is preserved
 Next release will be v3.5.0
11
Example 3
• Current release: v3.4.2
• Situation:
 Major changes occur (new features, etc.)
 Backwards compatibility is broken
 Next release will be v4.0.0
12
Wait…
How exactly are you defining the term
“backwards
compatibility”
?
13
Definition
• Open MPI vY is backwards compatible with
Open MPI vX (where Y>X) if:
 Users can compile a correct MPI / OSHMEM
program with vX
 Run it with the same CLI options and MCA
parameters using vX or vY
 The job executes correctly
14
What does that encompass?
• “Backwards compatibility” covers several
areas:
 Binary compatibility, specifically the MPI /
OSHMEM API ABI
 MPI / OSHMEM run time system
 mpirun / oshrun CLI options
 MCA parameter names / values / meanings
15
How will I know when backwards
compatibility breaks?
• Two ways:
1. The first digit of the Open MPI version
number changes
2. Read the NEWS file
• When the first digit of the version number
changes, NEWS will contain a list of what issues
broke backwards compatibility
16
Versioning note
• Open MPI only supports running exactly
the same version of the runtime and MPI /
OSHMEM libraries in a single job
 If you mix-n-match different versions in a
single job…
17
ERROR
Versioning:
beware of static builds!
• When an MPI app is statically linked, it is
“locked” to a specific version of Open MPI
 mpicc myapp.c –static –o myapp
 It is erroneous to mpirun with a different
version (e.g., mpirun vY)
18
myapp
Open MPI vX
Transition to the New
Version Numbering Scheme
19
How to move to the new
numbering?
20
v1.8.6
Released
June 19, 2015
How to move to the new
numbering?
21
What’s next?
v1.8.6
Released
June 19, 2015
How to move to the new
numbering?
22
v1.8.6
Released
June 19, 2015
What’s next?
Note: it would be crazy confusing
to change the version number scheme
in the middle of the v1.8.x series.
We won’t be doing that.
How to move to the new
numbering?
23
v1.8.6
Released
June 19, 2015
v1.10.0
Will contain the usual bug fixes
And (a small number of) new features
How to move to the new
numbering?
24
v1.8.6
Released
June 19, 2015
v1.10.0
Starting with v1.10.0,
future releases
will abide by the
new versioning
scheme
Release Planning Roadmap
25
What’s next?
• We are planning for v1.10.0
 Within the next few months
 Contains the usual bug fixes and minor
improvements (over v1.8.6)
 Also contains a small number of new features
• libfabric support
• Mellanox Yalla PML
• Intel PSM2 for OmniPath
26
What’s next?
• We anticipate v2.0.0
 Later this year
 Will contain larger new features
 Will not be backwards compatible with v1.10.x
27
28
Transition definition
for the technically inclined
Git master development
v2.x branch
v1.7.x /
v1.8.x branch
v1.10.x branch
v1.8 v1.8.5… v1.8.6
No further v1.8.x
releases planned unless
serious bugs are found
Why “v1.10.0” (vs. “v1.9.0”)?
1. Before June 2015, we referred to the next
major release as the “v1.9 series”
 “v1.10.0” clearly distinguishes from that idea
 “v2.0.0” conveys a significant difference (i.e.,
a major new release series)
2. It will take a while for the new scheme to
become common knowledge
 We didn’t want users to think “v1.9” = “odd” =
“unstable”
29
What’s the plan over time?
• Plan for a new release series once a year
 v2.x: release in mid / late 2015
 v3.x: release in mid / late 2016
 v4.x: release in mid / late 2017
 …etc.
30
NOTE: Scheduled releases is a new concept
for the Open MPI developer community. We’ll
continue to evaluate this plan over time.
What will be supported?
• (Continue the) Support “current version
and one prior” philosophy
 Mid 2015 – mid 2016
• Support v1.10.x, and v2.x
• Special case for the transition: also support v1.8.x
 Mid 2016 – mid 2017
• Support v2.x and v3.x
 Mid 2017 – mid 2018
• Support v3.x and v4.x
 …etc. 31
Planned development and
support cycle
Version 1.8.x series
2016 20172014 2018
Version 1.10.x series
Active
development
Slowed
development
Maintenance
(bug fix only)Key:
2015
32
Planned development and
support cycle
Version 1.8.x series
2016 20172014 2018
Version 1.10.x series
Version 2.x series
Active
development
Slowed
development
Maintenance
(bug fix only)Key:
2015
33
Planned development and
support cycle
Active
development
Slowed
development
Maintenance
(bug fix only)
Version 1.8.x series
2015 2016 20172014 2018
Version 1.10.x series
Key:
Version 2.x series
Version 3.x series
34
The Bottom Line
35
The bottom line
• Starting with v1.10.0:
 No more odd/even series
 “A.B.C”: each number has a specific meaning
• Read the NEWS file when “A” changes
 Release new features faster
• Aim to limit life release series
 ~1 year of devel + ~1 year of bug fixes
36

Open MPI new version number scheme and roadmap

  • 1.
    Open MPI Project June2015 Updated Version Numbering Scheme and Release Planning
  • 2.
    Table of contents •These slides cover several related topics: 1. Open MPI’s new version numbering scheme 2. Transition plan to the new version numbering 3. Release planning roadmap 4. The bottom line (TL;DR) • Let’s jump right in… 2
  • 3.
    Before July 2015… •Open MPI used an “odd / even” numbering scheme  1.odd: “feature” series  1.even: “stable” series • But it’s not working out as well as we’d like 3
  • 4.
    One problem • Veryfew users actually use the “odd” versions  Users equate “odd” with “unstable” • As a direct result:  New features don’t get real-world tested  …until the “even” releases 4
  • 5.
    Another problem • Userswant new features faster  A “stable” series (intentionally) does not receive new features • As a direct result:  New features take a long time to get to users 5
  • 6.
  • 7.
    Goodbye odd /even scheme! 7
  • 8.
    New version numberingscheme • Open MPI will (continue to) use a “A.B.C” version number triple • Each number now has a specific meaning: This number changes when backwards compatibility breaks This number changes when new features are added This number changes for all other releases A B C 8
  • 9.
    Examples • Pretend we’rein the future  The current Open MPI release is v3.4.2 • What will be the next release number? • Let’s look at a few cases… 9
  • 10.
    Example 1 • Currentrelease: v3.4.2 • Situation:  Bugs are fixed  No new features are added  Backwards compatibility is preserved  Next release will be v3.4.3 10
  • 11.
    Example 2 • Currentrelease: v3.4.2 • Situation:  Bugs are fixed  User-noticeable new features are added  Backwards compatibility is preserved  Next release will be v3.5.0 11
  • 12.
    Example 3 • Currentrelease: v3.4.2 • Situation:  Major changes occur (new features, etc.)  Backwards compatibility is broken  Next release will be v4.0.0 12
  • 13.
    Wait… How exactly areyou defining the term “backwards compatibility” ? 13
  • 14.
    Definition • Open MPIvY is backwards compatible with Open MPI vX (where Y>X) if:  Users can compile a correct MPI / OSHMEM program with vX  Run it with the same CLI options and MCA parameters using vX or vY  The job executes correctly 14
  • 15.
    What does thatencompass? • “Backwards compatibility” covers several areas:  Binary compatibility, specifically the MPI / OSHMEM API ABI  MPI / OSHMEM run time system  mpirun / oshrun CLI options  MCA parameter names / values / meanings 15
  • 16.
    How will Iknow when backwards compatibility breaks? • Two ways: 1. The first digit of the Open MPI version number changes 2. Read the NEWS file • When the first digit of the version number changes, NEWS will contain a list of what issues broke backwards compatibility 16
  • 17.
    Versioning note • OpenMPI only supports running exactly the same version of the runtime and MPI / OSHMEM libraries in a single job  If you mix-n-match different versions in a single job… 17 ERROR
  • 18.
    Versioning: beware of staticbuilds! • When an MPI app is statically linked, it is “locked” to a specific version of Open MPI  mpicc myapp.c –static –o myapp  It is erroneous to mpirun with a different version (e.g., mpirun vY) 18 myapp Open MPI vX
  • 19.
    Transition to theNew Version Numbering Scheme 19
  • 20.
    How to moveto the new numbering? 20 v1.8.6 Released June 19, 2015
  • 21.
    How to moveto the new numbering? 21 What’s next? v1.8.6 Released June 19, 2015
  • 22.
    How to moveto the new numbering? 22 v1.8.6 Released June 19, 2015 What’s next? Note: it would be crazy confusing to change the version number scheme in the middle of the v1.8.x series. We won’t be doing that.
  • 23.
    How to moveto the new numbering? 23 v1.8.6 Released June 19, 2015 v1.10.0 Will contain the usual bug fixes And (a small number of) new features
  • 24.
    How to moveto the new numbering? 24 v1.8.6 Released June 19, 2015 v1.10.0 Starting with v1.10.0, future releases will abide by the new versioning scheme
  • 25.
  • 26.
    What’s next? • Weare planning for v1.10.0  Within the next few months  Contains the usual bug fixes and minor improvements (over v1.8.6)  Also contains a small number of new features • libfabric support • Mellanox Yalla PML • Intel PSM2 for OmniPath 26
  • 27.
    What’s next? • Weanticipate v2.0.0  Later this year  Will contain larger new features  Will not be backwards compatible with v1.10.x 27
  • 28.
    28 Transition definition for thetechnically inclined Git master development v2.x branch v1.7.x / v1.8.x branch v1.10.x branch v1.8 v1.8.5… v1.8.6 No further v1.8.x releases planned unless serious bugs are found
  • 29.
    Why “v1.10.0” (vs.“v1.9.0”)? 1. Before June 2015, we referred to the next major release as the “v1.9 series”  “v1.10.0” clearly distinguishes from that idea  “v2.0.0” conveys a significant difference (i.e., a major new release series) 2. It will take a while for the new scheme to become common knowledge  We didn’t want users to think “v1.9” = “odd” = “unstable” 29
  • 30.
    What’s the planover time? • Plan for a new release series once a year  v2.x: release in mid / late 2015  v3.x: release in mid / late 2016  v4.x: release in mid / late 2017  …etc. 30 NOTE: Scheduled releases is a new concept for the Open MPI developer community. We’ll continue to evaluate this plan over time.
  • 31.
    What will besupported? • (Continue the) Support “current version and one prior” philosophy  Mid 2015 – mid 2016 • Support v1.10.x, and v2.x • Special case for the transition: also support v1.8.x  Mid 2016 – mid 2017 • Support v2.x and v3.x  Mid 2017 – mid 2018 • Support v3.x and v4.x  …etc. 31
  • 32.
    Planned development and supportcycle Version 1.8.x series 2016 20172014 2018 Version 1.10.x series Active development Slowed development Maintenance (bug fix only)Key: 2015 32
  • 33.
    Planned development and supportcycle Version 1.8.x series 2016 20172014 2018 Version 1.10.x series Version 2.x series Active development Slowed development Maintenance (bug fix only)Key: 2015 33
  • 34.
    Planned development and supportcycle Active development Slowed development Maintenance (bug fix only) Version 1.8.x series 2015 2016 20172014 2018 Version 1.10.x series Key: Version 2.x series Version 3.x series 34
  • 35.
  • 36.
    The bottom line •Starting with v1.10.0:  No more odd/even series  “A.B.C”: each number has a specific meaning • Read the NEWS file when “A” changes  Release new features faster • Aim to limit life release series  ~1 year of devel + ~1 year of bug fixes 36