В докладе рассматривается технология сборки C++ проектов на основе maven, применяемая в Odin Service Automation (OSA)

1. Использование maven для сборки
больших модульных C++ проектов
на примере Odin Service Automation
Михаил Кочеров
Sr. Software Developer @ Odin / Parallels

2. Agenda
It’s all about “last mile” – how to effectively build a product from
your C++ code
1. Product description
2. Old build technology
3. Maven overview
4. Maven for C++ in OSA
5. Adoption costs & conclusion
1

3. Odin Service Automation: Purpose and Functions
http://www.odin.com/products/automation/product-overview/
Odin Service Automation – platform to automate cloud services delivery:
– Shared web hosting (Apache, IIS)
– VPS hosting (Virtuozzo)
– Mail hosting (Exchange, Qmail)
– APS-enabled: Office 365, Lync, etc.
• Automates full business cycle of cloud services delivery: online-store,
order placement, payments processing, service provisioning, upsales,
billing
• Users – Cloud Service Providers managing Data Centers
2

4. Odin Service Automation: Traits
• Started in 1999, more than >12 major releases
• > 300 of devs / QA / PM in MSK and NSK
• Multiplatform:
– Windows: MS Windows Server 2008, 2012
– Linux: CentOS / RHEL 5, 6, 7
• Distributed deployment: affects testing
• Multilingual: C++, C, Java, C#, Python
3

5. Odin Service Automation: Structure
4
Apache
IIS
chief
SaaS
VPS
Exchange
C++ / JAVA
PLATFORM
… - C++ service controller
- communication via CORBA

6. CORBA
5
Technology to build distributed programs in a language
agnostic way
APS SC
Apache IDL file
C++ Client
stub
Apache SC
C++ Server
stub
call
generate generate
- C++ CORBA derivatives: .h, .cpp, .inl, …

7. Odin Service Automation: Source Code
6
Lang. What is written Files
count
LOC
C/C++ Platform
Libraries
Service controllers
5300* 1.5M
Java Platform
UI
7100 1M
C# Windows
Provisioning Engine
830 325K
Python Upgrade utilities 300 60K
TOTAL 13530 2.9M
80
10
10
Build Time Consumers (%)
C++ Java Other

8. Factors which Affects Build System
• Multiplatform – different compilers and tools
• Rich dependencies between service controllers –
correct build order of service controllers is req-d
• Large – small overheads starts to matter
• Distributed deployment – need expensive resources to
check if build is valid (BVT test), not available for devs 
• Many people – many commits which could break the
build, ideally need a build for each commit in order to
test it
7

9. Old Build System Overview
8
• BSD make (pmake) on Linux, MSBuild on
Windows
• Harness scripts on bash + python + perl
• C++ distributed compilation: Incredibuild / Distcc
• ccache

10. Flaws of An Old Build System, p. I
9
1) High “cost of ownership”: hand written makefiles, no
frontend (cmake or qmake):
.for _t in all build package rpm install clean cleanup
${_t}: ${BUILD:S,^,${_t}-,}
.endfor
…
_path-${_c}= ${_srcs-${_c}:H:u:S,^,${.CURDIR}/,}
2) “Monolithic” nature of a build – problem in one module
blocks whole build

11. Flaws of An Old Build System, p. II
10
3) Bad incremental builds
– Non-reliable
– Slow
Frequent CLEAN
builds
Windows Linux
other 20 18
c++ 75 58
75
58
20
18
0
10
20
30
40
50
60
70
80
90
100
Minutes
Clean Build Duration (old tech.)

12. Build Process
11
1. Read makefiles
2. Build product
dep. graph
3. Recurse for each component
according to dep. graph
build.mk
build.master.mk
build.submake.mk
components.mk
components.sc.mk
…
Submake
?
2. Build component
dep. graph
3. Build component
+-

13. Make: Curse of the Recurse, p. I
12
a.cpp b.cpp c.cpp d.cpp g.cppf.cppe.cpp
l.cpp
j.cppi.cpp
a.h b.h c.h d.h g.hf.he.h
l.h
j.h
h.hk.h
i.h
SC1 SC2 SC…n
All
Submake 1
Submake 2 Submake n

14. Make: Curse of the Recurse, p. II
13
GIGO principle: Garbage In, Garbage Out
• Poorly defined C++ projects boundaries – no isolation:
implicit dependencies between projects by referencing
files from several global pools
• Aggravated by the use of auto-generated C++ CORBA
derivatives
• Manual tweaking of a build order

15. Make: Slowness Reasons
14
• Expensive submakes: each submakes analyzes the same
things as its peers and its master
• Too much overhead on externals programs: C++ auto deps,
files copying, etc.

16. OSA Build Technology Requirements
15
• SUPPORT MULTIPROJECT CONFIGURATION WITH
COMPONENTS ISOLATION
• FAST COMPILATION OF C++
• FAST AND RELIABLE INCREMENTAL BUILDS
• Break “monolithic” nature of builds - modularization
• Lower “Cost of ownership”: get rid of scripts “zoo”, easy
customization of a build process (C++ CORBA derivatives
generation, EDL-processing, PPM packaging, etc.)

17. What is an Apache Maven?
16
• Build automation tool
• De-facto – industry standard for managing Java projects
• 60% of Java developers use Maven
• More that 100 000 Open Source Projects is managed by
Maven
http://maven.apache.org

18. Why Apache Maven? p. I
17
1) Inherent support of multiproject configuration
2) Dependency management and versioning
• Solves common dep. management problems: transitive
dependencies, cycles, different versioning arbitration, scoping
• Inherent support of artifact repositories: local, remote
• Sophisticated artifacts versioning: coordinates { groupId,
artifactId, version, packaging, classifier }

19. Why Apache Maven? p. II
18
3) Customization through
“Build Lifecycle” concept
• Predefined set of lifecycles:
default, clean, site
• Lifecycle – set of phases.
• Concrete actions performed on
each phase is customized via
goals
• Goals are executed via plugins
Phase Goal
process-
resources
resources:resources
compile compiler:compile
process-test-
resources
resources:testResou
rces
test-compile
compiler:testCompil
e
test surefire:test
package jar:jar
install install:install
deploy deploy:deploy

20. Why Apache Maven? p. III
19
4) Spare costs through “Convention over
Configuration” principle
• Convention of source files layout
• Unified list of plugins used during build
• Build lifecycle
The idea: need to specify only exceptions,
everything else is by default

21. pom.xml
20

22. Make vs Maven
21
A B
c d e
All
f
B
e f
Make Maven
PROJECT
Dep
A
Dep
B
Dep
C

23. MAVEN
REPOSITORY
OSA “CELL” Concept
22
CELL – unified, (deps, lin/win, assembly) isolated C++
maven project for OSA which consumes and contributes
C++ artifacts (proxies) via maven bus
1.
CONSUME
PROXIES
2.
BUILD
3.
CONTRIBUTE
PROXIES
Other cells:
C++ CORBA derivatives
C++ headers
C++ obj files
C++ libraries
This cell:
C++ CORBA derivatives
C++ headers
C++ obj files
C++ libraries

24. Cells Dependency Graph
23
cell
Apache
cell
BrandingManager
cell
pem_clientcell
chief
cell
libservices
cell
ConfigurationManager
EDL
EDL
CORBA-FULL
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL
CORBA-CLIENT,
EDL
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL

25. Cells Isolation via Proxies
24
cell
Apache
cell
ConfigurationManager
No C++ cross
component
dependencies
chief
proxies

26. C++ Compilation on Maven
25
Plugin Pros Cons
NAR maven plugin
http://maven-nar.github.io/
Power-full Over-engineered
Limits use of classifiers
native-maven-plugin
http://www.mojohaus.org/mave
n-native/native-maven-plugin/
“Good enough” for OSA Development abandoned

27. Dep. Graph
Basics of native-maven-plugin
26
1. Initialize
2. Analyze
C++ sources
Include dirs.
Lib dirs
.obj files
libraries
Any
changes?
3. Rebuild
+
-
Compiler opts.
Linker opts.
Other opts.
A B
c d e
target
f
B
e f

28. PADependencyImplementor, native-maven-plugin and
Proxies
27
MAVEN LOCAL REPOSITORY (~/.m2)
CELLs’ proxies
(zipped)
native-maven-plugin
UNZIPPED PROXIES
- Headers
- Obj. files
- Inl files
- C++ CORBA derivatives
2. Unzip if
missing to a
shared
location
3. Feed files
locations
(for compile & link)
1. Cell
dependencies:
list of proxies
PADependencyImplementor

29. native-maven-plugin and Proxies
28
g++ -g -pipe -fPIC -DPIC -Wall -W -fno-strict-aliasing -Werror -Wno-parentheses -D_REENTRANT
...
-I/root/.m2/_pa_proxy/com-parallels-poa-sh/apache-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-poa-sh/iis-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-poa-peripheralBus/libservices-assembly-PADEP-CORBA-FULL-
RHEL6/6.1.0.2752
-I/root/.m2/_pa_proxy/com-parallels-poa-vps/vpsmanager-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.3930 -
I/root/.m2/_pa_proxy/com-parallels-poa-sh/configurationmanager-assembly-PADEP-CORBA-FULL-
RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-pa/pacryptoapi-PADEP-HEADER-RHEL6/0.3.2.006
...
-o lin/target/objs/Tasks.o -c cpp/Tasks.cpp -DSINGLETON_MODULE=SAAS

30. C++ Compilation Tuning
29
Project Linux Maven Profile Window Maven
Profile
cppProject g++, -g, -pipe, … cl.exe, /FD, /EHsc, …
cppExeProject <Lin linker opts> <Win linker opts>
cppDynamicProject -shared, … /DLL, …
cppStaticProject ar, … /LIB, …
-

31. C++ Build Lifecycle
30
Phase Plugin Task
initialize native-maven-plugin Read project configuration
PADependencyImple
mentor
Prepare proxies, pass it to native-maven-
plugins
Bin2CProcessor Convert binaries (e.g. png) to a C-code
generate-
sources
EDLProcessor Process .edl files which contain error
description
IDLProcessor Process CORBA interface definition files .idl
precompile native-maven-plugin Generate precompiled headers (.pch)
compile native-maven-plugin Compile & link
package DebugStripper Extract debug info from compiled files to a
separate ones

32. OSA Modularity
31
platform
{ cells }
peripheralBus
{ cells }sh
{ cells }
vps
{ cells } dedicated
{ cells }
…
{ cells }
mc
{ cells }
OSA
{ cells }
BENEFITS
• Extra fast OSA
build
• No failure
propagation
• Custom builds

33. OSA Build Duration: make-based vs maven-based
32
95
75
20,00
11
0
10
20
30
40
50
60
70
80
90
100
Win32 Linux
Minutes
Platforms
Make
Clean Incremental
39
25
5,50 4
0
10
20
30
40
50
60
70
80
90
100
Win32 Linux
Minutes
Platforms
Maven
Clean Incremental
~63% reduction
for clean build
~68% reduction
for incr. build

34. Factors of a Build Speed-up
33
1) Modularity: extensive reuse of prebuild things
2) Much quicker dependency analysis: scope of
search is narrower
3) No expensive subprograms: C++ autodeps,
files ops., etc.

35. Factors of Incremental Build Reliability
34
1) Code Isolation
2) Code Isolation
3) … and Code Isolation one more time
There is no difference between clean build and
incremental – there will be exactly the same
environment for each C++ cell.

36. Maven for C++ in OSA: adoption costs
35
• Converted on the course of 2 OSA releases without stopping
dev. cycle
• Converted >300 projects
• Took 2 years to complete by 2 devs
• Labor cost: 4 m/y
REALLY WORTH ITS COST

37. Thank you!
E-mail: mkocherov@odin.com
Blog: http://develorium.com

38. Generalized Build Scheme
37
Windows
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector
Сборка вместе всех артефакты от
всех платформ – оформление билда
Загрузка оформленного билда на
deliver сервер
Linux 1
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector +distcc+IncrediBuild
Linux …
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector
Linux n
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector+distcc +distcc