SlideShare a Scribd company logo
The Jack and Jill
build system
Eric Lafortune
Developer of ProGuard and DexGuard
CTO at GuardSquare
www.guardsquare.com
The Jack and Jill build system
Inside the build system
and the runtime
Practical implications and
results
Jack and Jill
Java Android Compiler Kit
Jack Intermediate Library Linker
Jack and Jill
● Now default in build process of AOSP
● Optional in Gradle build process:
android {
defaultConfig {
useJack true
...
}
...
}
android {
defaultConfig {
useJack true
...
}
...
}
com.android.tools.build:gradle:2.0.0
android {
defaultConfig {
jackOptions {
enabled true
jackInProcess true
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
...
}
...
}
android {
defaultConfig {
jackOptions {
enabled true
jackInProcess true
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
...
}
...
}
Jack and Jill
com.android.tools.build:gradle:2.1.0 alpha1
android {
defaultConfig {
jackOptions {
enabled true
jackInProcess true
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
...
}
...
}
android {
defaultConfig {
jackOptions {
enabled true
jackInProcess true
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
...
}
...
}
Jack and Jill
...
:generateReleaseSources
:jillReleasePackagedLibraries
:jillReleaseRuntimeLibraries
:processReleaseJavaRes UP-TO-DATE
:transformResourcesWithMergeJavaResForRelease
:compileReleaseJavaWithJack
:compileReleaseNdk UP-TO-DATE
:compileReleaseSources
...
...
:generateReleaseSources
:jillReleasePackagedLibraries
:jillReleaseRuntimeLibraries
:processReleaseJavaRes UP-TO-DATE
:transformResourcesWithMergeJavaResForRelease
:compileReleaseJavaWithJack
:compileReleaseNdk UP-TO-DATE
:compileReleaseSources
...
com.android.tools.build:gradle:2.1.0 alpha1
Why?
Technical
Legal ?
Strategical ?
Android build process
Java bytecode
(.class)
Libraries
(.class)
Dx
Javac
Source code
(.java)
Libraries
(.class)
XML resources
(.xml)
Assets Assets
Compiled
XML resources
(.xml)
Aapt
Dalvik bytecode
(classes.dex)
Android
runtime stubs
(.class)
Transform API
Java bytecode
(.class)
Processor
Libraries
(.class)
Transformed
Java bytecode
(.class)
Dx
Javac
Assets
Compiled
XML resources
(.xml)
Aapt
Dalvik bytecode
(classes.dex)
Source code
(.java)
Libraries
(.class)
XML resources
(.xml)
Assets
Runtime stubs
(.class)
ProGuard
Java bytecode
(.class)
ProGuard
Libraries
(.class)
Optimized
Java bytecode
(.class)
Dx
Javac
Assets
Compiled
XML resources
(.xml)
Aapt
Dalvik bytecode
(classes.dex)
ProGuard
config
(.txt)
ProGuard
mapping
(.txt)
Source code
(.java)
Libraries
(.class)
XML resources
(.xml)
Assets
Runtime stubs
(.class)
ProGuard
Java bytecode
(.class)
ProGuard
Libraries
(.class)
Optimized
Java bytecode
(.class)
Dx
Javac
Assets
Compiled
XML resources
(.xml)
Aapt
Dalvik bytecode
(classes.dex)
ProGuard
config
(.txt)
ProGuard
mapping
(.txt)
Source code
(.java)
Libraries
(.class)
XML resources
(.xml)
Assets
Runtime stubs
(.class)
Jack and Jill
Libraries
(.jayce)
Jack
Assets
Compiled
XML resources
(.xml)
Aapt
Dalvik bytecode
(classes.dex)
ProGuard
config
(.txt)
ProGuard
mapping
(.txt)
Source code
(.java)
Libraries
(.class)
XML resources
(.xml)
Assets
Runtime stubs
(.class)
Jill
Jill Runtime
(.jayce)
Code representations
for (int i = 0; i < 8; i++) ...for (int i = 0; i < 8; i++) ...
iconst 0
istore v0
iload v0
bipush 8
ificmpge +12
...
iinc v0, 1
goto -12
iconst 0
istore v0
iload v0
bipush 8
ificmpge +12
...
iinc v0, 1
goto -12
.class .dex
.java
Javac
Dx
const/4 v0, #0
const/16 v1, #8
if-ge v0, v1, 8
...
add-int/lit8 v0, v0, #1
goto -9
const/4 v0, #0
const/16 v1, #8
if-ge v0, v1, 8
...
add-int/lit8 v0, v0, #1
goto -9
Code representations
for (int i = 0; i < 8; i++) ...for (int i = 0; i < 8; i++) ...
iconst 0
istore v0
iload v0
bipush 8
ificmpge +12
...
iinc v0, 1
goto -12
iconst 0
istore v0
iload v0
bipush 8
ificmpge +12
...
iinc v0, 1
goto -12
.class .dex
.java
Javac
Efficient to store and execute
Inconvenient to process
Dx
const/4 v0, #0
const/16 v1, #8
if-ge v0, v1, 8
...
add-int/lit8 v0, v0, #1
goto -9
const/4 v0, #0
const/16 v1, #8
if-ge v0, v1, 8
...
add-int/lit8 v0, v0, #1
goto -9
LOCAL int n
FOR_STATEMENT:
- EXPRESSION_STATEMENT:
- ASG_OPERATION:
- LOCAL_REF i
- INT_LITERAL 0
- LT_OPERATION:
- LOCAL_REF i
- INT_LITERAL 8
- POSTFIX_INC_OPERATION:
- LOCAL_REF i
- ...
LOCAL int n
FOR_STATEMENT:
- EXPRESSION_STATEMENT:
- ASG_OPERATION:
- LOCAL_REF i
- INT_LITERAL 0
- LT_OPERATION:
- LOCAL_REF i
- INT_LITERAL 8
- POSTFIX_INC_OPERATION:
- LOCAL_REF i
- ...
Code representations
for (int i = 0; i < 8; i++) ...for (int i = 0; i < 8; i++) ...
iconst 0
istore v0
iload v0
bipush 8
ificmpge +12
...
iinc v0, 1
goto -12
iconst 0
istore v0
iload v0
bipush 8
ificmpge +12
...
iinc v0, 1
goto -12
.class .dex
.java
Javac
.jayce
Jack
const/4 v0, #0
const/16 v1, #8
if-ge v0, v1, 8
...
add-int/lit8 v0, v0, #1
goto -9
const/4 v0, #0
const/16 v1, #8
if-ge v0, v1, 8
...
add-int/lit8 v0, v0, #1
goto -9
Jill
Jack
Implications of Jack and Jill
● Java annotation processors – supported in Jack
E.g. Dagger, Butterknife
● Bytecode processors – only supported via Jill
E.g. JaCoCo, Retrolambda
● Other JVM languages – only supported via Jill
E.g. Scala, Groovy, Dart
● Instant Run – not supported yet
Implications of Jack and Jill
Libraries
(.jayce)
Jack
Assets
Compiled
XML resources
(.xml)
Aapt
Dalvik bytecode
(classes.dex)
Javac annotation processors
E.g. Dagger, Butterknife
Source code
(.scala,...)
Libraries
(.class)
XML resources
(.xml)
Assets
Runtime
(.class)
Jill
Jill Runtime
(.jayce)
Bytecode processors
Java bytecode
(.class)Scalac
Application
(.jayce)Jill
Internals
Jack: Java frontend
Eclipse ecj library
Jill: Java frontend
ASM4 library
Java 8: closures
.....
button.setOnClickListener(view -> .....);
.....
.....
button.setOnClickListener(view -> .....);
.....
.....
button.setOnClickListener(new MyActivity$LambdaImpl());
.....
.....
button.setOnClickListener(new MyActivity$LambdaImpl());
.....
public class MyActivity$LambdaImpl {
public void onClick(View view) {
.....
}
}
public class MyActivity$LambdaImpl {
public void onClick(View view) {
.....
}
}
Jack
For all versions
of Android
Device
More changes!
RunDalvik bytecode
(classes.dex)
Source code
(.java)
Libraries
(.class)
Runtime stubs
(.class)
Build Dalvik bytecode
(classes.oat)
Android
runtimeJack & Jill
OpenJDK
RunDalvik bytecode
(classes.dex)
Source code
(.java)
Libraries
(.class)
Build Dalvik bytecode
(classes.oat)
Android +
OpenJDK
runtime
Android +
OpenJDK
runtime stubs
(.class)
OpenJDK
Device
OpenJDK
Java 8: default methods
● In your own code:
● In the Android runtime:
public interface java.util.List<E> extends java.util.Collection<E> {
public abstract int size();
...
public default void sort(java.util.Comparator<? super E>) {
.....
}
}
public interface java.util.List<E> extends java.util.Collection<E> {
public abstract int size();
...
public default void sort(java.util.Comparator<? super E>) {
.....
}
}
public interface Foo {
public abstract void someMethod();
public void someDefaultMethod() {
....
}
}
public interface Foo {
public abstract void someMethod();
public void someDefaultMethod() {
....
}
}
Only on
Android N
New instructions
Dalvik bytecode
RunDalvik bytecode
(classes.dex)
Source code
(.java)
Libraries
(.class)
Build Dalvik bytecode
(classes.oat)
Android +
OpenJDK
runtime
Android +
OpenJDK
runtime stubs
(.class)
New instructions
Device
Java 8: closures
.....
● put-field-lambda
● get-field-lambda
● put-static-lambda
● get-static-lambda
● aget-lambda
● aput-lambda
● move-lambda
● cmp-lambda
● return-lambda
.....
● put-field-lambda
● get-field-lambda
● put-static-lambda
● get-static-lambda
● aget-lambda
● aput-lambda
● move-lambda
● cmp-lambda
● return-lambda Only on
Android N+
New Dalvik instructions:
Build performance
44s – Javac + dx (clean build)
1m9s – Javac + ProGuard optimize + dx (clean build)
24s – Javac + DexGuard (clean build)
2m36s – Javac + DexGuard optimize (clean build)
1m17s – Jack (clean build)
47s – Jack (clear all dex files)
15s – Javac + dx (clear all class files)15s – Javac + dx (clear all class files)
5s – Javac + dx (clear one class file)5s – Javac + dx (clear one class file)
2s – Javac + dx (code unchanged)2s – Javac + dx (code unchanged)
40s – Jack (clear one dex file)40s – Jack (clear one dex file)
3s – Jack (code unchanged)3s – Jack (code unchanged)
Google I/O app
(AMD A10-7800, Ubuntu 14.04, Java 8, Android Gradle 2.1.0 alpha1, Gradle daemon)
49s – Jack minify (clear all dex files)49s – Jack minify (clear all dex files)
21s – Javac + ProGuard + dx (clean build)21s – Javac + ProGuard + dx (clean build)
Application runtime performance
50,000 – Javac + dx
63,800 – Javac + ProGuard optimize + dx
82,300 – Javac + DexGuard optimize
54,500 – Jack
Caffeinemark benchmark
(Nexus 5, Android 6.0, ART)
+9%
+28%
+65%
(higher is better)
Application size
3.1M – Javac + dx
1.9M – Javac + ProGuard optimize + dx
1.8M – Javac + DexGuard optimize
1.9M – Jack + minify
Google I/O app
-39%
-38%
-44%
(smaller is better)
DexGuard
Java bytecode
(.class)
DexGuard
Libraries
(.class)
Javac
Optimized
assets
Optimized
XML resources
(.xml)
Aapt
Optimized
Dalvik bytecode
(classes.dex)
ProGuard
config
(.txt)
ProGuard
mapping
(.txt)
Source code
(.java)
Libraries
(.class)
XML resources
(.xml)
Assets
Android
runtime stubs
(.class)
Compiled
XML resources
(.xml)
Android
runtime stubs
(.class)
Assets
Native code
(.so)Clang
Source code
(.c / .cpp)
Optimized
native code
(.so)
Conclusions
● Huge changes
● Clean implementation
Internally
Conclusions
● Huge changes
● Clean implementation
● Different build process
● New intermediary representation
Internally
For languages and tools
Conclusions
● Huge changes
● Clean implementation
● More efficient builds
● Java 8
● Different build process
● New intermediary representation
Internally
For languages and tools
For app developers
Further reading
● “Experimental New Android Tool Chain - Jack and Jill”
http://tools.android.com/tech-docs/jackandjill
● “Java 8 Language Features”
https://developer.android.com/preview/j8-jack.html
● “Jack (Java Android Compiler Kit)”
https://source.android.com/source/jack.html
● AOSP repository
https://android.googlesource.com/toolchain/jack/
● “State of the ART”
http://androidbackstage.blogspot.com/2016/03/episode-45-state-of-art.html
Questions?
Size
Java bytecode
Performance
Jayce bytecode
Dalvik bytecode
Jack & Jill
ProGuard
DexGuard

More Related Content

What's hot

Eric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDK
Eric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDKEric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDK
Eric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDK
GuardSquare
 
Overview of Android Infrastructure
Overview of Android InfrastructureOverview of Android Infrastructure
Overview of Android Infrastructure
Alexey Buzdin
 
OWF12/PAUG Conf Days Pro guard optimizer and obfuscator for android, eric l...
OWF12/PAUG Conf Days Pro guard   optimizer and obfuscator for android, eric l...OWF12/PAUG Conf Days Pro guard   optimizer and obfuscator for android, eric l...
OWF12/PAUG Conf Days Pro guard optimizer and obfuscator for android, eric l...
Paris Open Source Summit
 
Jdk 7 4-forkjoin
Jdk 7 4-forkjoinJdk 7 4-forkjoin
Jdk 7 4-forkjoin
knight1128
 
Native code in Android applications
Native code in Android applicationsNative code in Android applications
Native code in Android applications
Dmitry Matyukhin
 
Club of anonimous developers "Refactoring: Legacy code"
Club of anonimous developers "Refactoring: Legacy code"Club of anonimous developers "Refactoring: Legacy code"
Club of anonimous developers "Refactoring: Legacy code"
Victor_Cr
 
Java Bytecode For Discriminating Developers - GeeCON 2011
Java Bytecode For Discriminating Developers - GeeCON 2011Java Bytecode For Discriminating Developers - GeeCON 2011
Java Bytecode For Discriminating Developers - GeeCON 2011
Anton Arhipov
 

What's hot (20)

Eric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDK
Eric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDKEric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDK
Eric Lafortune - ProGuard: Optimizer and obfuscator in the Android SDK
 
Overview of Android Infrastructure
Overview of Android InfrastructureOverview of Android Infrastructure
Overview of Android Infrastructure
 
Python Evolution
Python EvolutionPython Evolution
Python Evolution
 
OWF12/PAUG Conf Days Pro guard optimizer and obfuscator for android, eric l...
OWF12/PAUG Conf Days Pro guard   optimizer and obfuscator for android, eric l...OWF12/PAUG Conf Days Pro guard   optimizer and obfuscator for android, eric l...
OWF12/PAUG Conf Days Pro guard optimizer and obfuscator for android, eric l...
 
2013.02.02 지앤선 테크니컬 세미나 - Xcode를 활용한 디버깅 팁(OSXDEV)
2013.02.02 지앤선 테크니컬 세미나 - Xcode를 활용한 디버깅 팁(OSXDEV)2013.02.02 지앤선 테크니컬 세미나 - Xcode를 활용한 디버깅 팁(OSXDEV)
2013.02.02 지앤선 테크니컬 세미나 - Xcode를 활용한 디버깅 팁(OSXDEV)
 
Improving Android Performance at Mobiconf 2014
Improving Android Performance at Mobiconf 2014Improving Android Performance at Mobiconf 2014
Improving Android Performance at Mobiconf 2014
 
Jdk 7 4-forkjoin
Jdk 7 4-forkjoinJdk 7 4-forkjoin
Jdk 7 4-forkjoin
 
soscon2018 - Tracing for fun and profit
soscon2018 - Tracing for fun and profitsoscon2018 - Tracing for fun and profit
soscon2018 - Tracing for fun and profit
 
Java after 8
Java after 8Java after 8
Java after 8
 
Kotlin 101 for Java Developers
Kotlin 101 for Java DevelopersKotlin 101 for Java Developers
Kotlin 101 for Java Developers
 
Kernel Recipes 2013 - Automating source code evolutions using Coccinelle
Kernel Recipes 2013 - Automating source code evolutions using CoccinelleKernel Recipes 2013 - Automating source code evolutions using Coccinelle
Kernel Recipes 2013 - Automating source code evolutions using Coccinelle
 
Native code in Android applications
Native code in Android applicationsNative code in Android applications
Native code in Android applications
 
Club of anonimous developers "Refactoring: Legacy code"
Club of anonimous developers "Refactoring: Legacy code"Club of anonimous developers "Refactoring: Legacy code"
Club of anonimous developers "Refactoring: Legacy code"
 
Kotlin meets Gadsu
Kotlin meets GadsuKotlin meets Gadsu
Kotlin meets Gadsu
 
Java Bytecode For Discriminating Developers - GeeCON 2011
Java Bytecode For Discriminating Developers - GeeCON 2011Java Bytecode For Discriminating Developers - GeeCON 2011
Java Bytecode For Discriminating Developers - GeeCON 2011
 
Java and OpenJDK: disecting the ecosystem
Java and OpenJDK: disecting the ecosystemJava and OpenJDK: disecting the ecosystem
Java and OpenJDK: disecting the ecosystem
 
The Ring programming language version 1.2 book - Part 80 of 84
The Ring programming language version 1.2 book - Part 80 of 84The Ring programming language version 1.2 book - Part 80 of 84
The Ring programming language version 1.2 book - Part 80 of 84
 
Concurrency Concepts in Java
Concurrency Concepts in JavaConcurrency Concepts in Java
Concurrency Concepts in Java
 
Bytecode manipulation with Javassist and ASM
Bytecode manipulation with Javassist and ASMBytecode manipulation with Javassist and ASM
Bytecode manipulation with Javassist and ASM
 
groovy & grails - lecture 7
groovy & grails - lecture 7groovy & grails - lecture 7
groovy & grails - lecture 7
 

Similar to Eric Lafortune - The Jack and Jill build system

Unit Testing RPG with JUnit
Unit Testing RPG with JUnitUnit Testing RPG with JUnit
Unit Testing RPG with JUnit
Greg.Helton
 
Java 5 6 Generics, Concurrency, Garbage Collection, Tuning
Java 5 6 Generics, Concurrency, Garbage Collection, TuningJava 5 6 Generics, Concurrency, Garbage Collection, Tuning
Java 5 6 Generics, Concurrency, Garbage Collection, Tuning
Carol McDonald
 
Clojure - A new Lisp
Clojure - A new LispClojure - A new Lisp
Clojure - A new Lisp
elliando dias
 
Java Bytecode for Discriminating Developers - JavaZone 2011
Java Bytecode for Discriminating Developers - JavaZone 2011Java Bytecode for Discriminating Developers - JavaZone 2011
Java Bytecode for Discriminating Developers - JavaZone 2011
Anton Arhipov
 
JavaScript Growing Up
JavaScript Growing UpJavaScript Growing Up
JavaScript Growing Up
David Padbury
 

Similar to Eric Lafortune - The Jack and Jill build system (20)

Unit Testing RPG with JUnit
Unit Testing RPG with JUnitUnit Testing RPG with JUnit
Unit Testing RPG with JUnit
 
Java 5 6 Generics, Concurrency, Garbage Collection, Tuning
Java 5 6 Generics, Concurrency, Garbage Collection, TuningJava 5 6 Generics, Concurrency, Garbage Collection, Tuning
Java 5 6 Generics, Concurrency, Garbage Collection, Tuning
 
Clojure And Swing
Clojure And SwingClojure And Swing
Clojure And Swing
 
Js tacktalk team dev js testing performance
Js tacktalk team dev js testing performanceJs tacktalk team dev js testing performance
Js tacktalk team dev js testing performance
 
Construire une application JavaFX 8 avec gradle
Construire une application JavaFX 8 avec gradleConstruire une application JavaFX 8 avec gradle
Construire une application JavaFX 8 avec gradle
 
Jdk Tools For Performance Diagnostics
Jdk Tools For Performance DiagnosticsJdk Tools For Performance Diagnostics
Jdk Tools For Performance Diagnostics
 
The Ring programming language version 1.7 book - Part 75 of 196
The Ring programming language version 1.7 book - Part 75 of 196The Ring programming language version 1.7 book - Part 75 of 196
The Ring programming language version 1.7 book - Part 75 of 196
 
The Ring programming language version 1.8 book - Part 77 of 202
The Ring programming language version 1.8 book - Part 77 of 202The Ring programming language version 1.8 book - Part 77 of 202
The Ring programming language version 1.8 book - Part 77 of 202
 
Adopting GraalVM - Scale by the Bay 2018
Adopting GraalVM - Scale by the Bay 2018Adopting GraalVM - Scale by the Bay 2018
Adopting GraalVM - Scale by the Bay 2018
 
Java 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from OredevJava 7 Whats New(), Whats Next() from Oredev
Java 7 Whats New(), Whats Next() from Oredev
 
Object Oriented Programming-JAVA
Object Oriented Programming-JAVAObject Oriented Programming-JAVA
Object Oriented Programming-JAVA
 
Android Jetpack + Coroutines: To infinity and beyond
Android Jetpack + Coroutines: To infinity and beyondAndroid Jetpack + Coroutines: To infinity and beyond
Android Jetpack + Coroutines: To infinity and beyond
 
Scala @ TechMeetup Edinburgh
Scala @ TechMeetup EdinburghScala @ TechMeetup Edinburgh
Scala @ TechMeetup Edinburgh
 
Android and cpp
Android and cppAndroid and cpp
Android and cpp
 
Clojure - A new Lisp
Clojure - A new LispClojure - A new Lisp
Clojure - A new Lisp
 
Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008
 
Native Java with GraalVM
Native Java with GraalVMNative Java with GraalVM
Native Java with GraalVM
 
Java Bytecode for Discriminating Developers - JavaZone 2011
Java Bytecode for Discriminating Developers - JavaZone 2011Java Bytecode for Discriminating Developers - JavaZone 2011
Java Bytecode for Discriminating Developers - JavaZone 2011
 
The curious Life of JavaScript - Talk at SI-SE 2015
The curious Life of JavaScript - Talk at SI-SE 2015The curious Life of JavaScript - Talk at SI-SE 2015
The curious Life of JavaScript - Talk at SI-SE 2015
 
JavaScript Growing Up
JavaScript Growing UpJavaScript Growing Up
JavaScript Growing Up
 

Recently uploaded

AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
Alluxio, Inc.
 

Recently uploaded (20)

Crafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationCrafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM Integration
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
 
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfA Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
 
Benefits of Employee Monitoring Software
Benefits of  Employee Monitoring SoftwareBenefits of  Employee Monitoring Software
Benefits of Employee Monitoring Software
 
A Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data MigrationA Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data Migration
 
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product UpdatesGraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 
10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf10 Essential Software Testing Tools You Need to Know About.pdf
10 Essential Software Testing Tools You Need to Know About.pdf
 
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
 
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with StrimziStrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi
 
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
 
SOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBroker
 
Studiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareStudiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting software
 
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
 
Secure Software Ecosystem Teqnation 2024
Secure Software Ecosystem Teqnation 2024Secure Software Ecosystem Teqnation 2024
Secure Software Ecosystem Teqnation 2024
 
iGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by SkilrockiGaming Platform & Lottery Solutions by Skilrock
iGaming Platform & Lottery Solutions by Skilrock
 
How To Build a Successful SaaS Design.pdf
How To Build a Successful SaaS Design.pdfHow To Build a Successful SaaS Design.pdf
How To Build a Successful SaaS Design.pdf
 

Eric Lafortune - The Jack and Jill build system

  • 1. The Jack and Jill build system Eric Lafortune Developer of ProGuard and DexGuard CTO at GuardSquare www.guardsquare.com
  • 2. The Jack and Jill build system Inside the build system and the runtime Practical implications and results
  • 3. Jack and Jill Java Android Compiler Kit Jack Intermediate Library Linker
  • 4. Jack and Jill ● Now default in build process of AOSP ● Optional in Gradle build process: android { defaultConfig { useJack true ... } ... } android { defaultConfig { useJack true ... } ... } com.android.tools.build:gradle:2.0.0
  • 5. android { defaultConfig { jackOptions { enabled true jackInProcess true } compileOptions { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 } ... } ... } android { defaultConfig { jackOptions { enabled true jackInProcess true } compileOptions { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 } ... } ... } Jack and Jill com.android.tools.build:gradle:2.1.0 alpha1
  • 6. android { defaultConfig { jackOptions { enabled true jackInProcess true } compileOptions { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 } ... } ... } android { defaultConfig { jackOptions { enabled true jackInProcess true } compileOptions { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 } ... } ... } Jack and Jill ... :generateReleaseSources :jillReleasePackagedLibraries :jillReleaseRuntimeLibraries :processReleaseJavaRes UP-TO-DATE :transformResourcesWithMergeJavaResForRelease :compileReleaseJavaWithJack :compileReleaseNdk UP-TO-DATE :compileReleaseSources ... ... :generateReleaseSources :jillReleasePackagedLibraries :jillReleaseRuntimeLibraries :processReleaseJavaRes UP-TO-DATE :transformResourcesWithMergeJavaResForRelease :compileReleaseJavaWithJack :compileReleaseNdk UP-TO-DATE :compileReleaseSources ... com.android.tools.build:gradle:2.1.0 alpha1
  • 8. Android build process Java bytecode (.class) Libraries (.class) Dx Javac Source code (.java) Libraries (.class) XML resources (.xml) Assets Assets Compiled XML resources (.xml) Aapt Dalvik bytecode (classes.dex) Android runtime stubs (.class)
  • 9. Transform API Java bytecode (.class) Processor Libraries (.class) Transformed Java bytecode (.class) Dx Javac Assets Compiled XML resources (.xml) Aapt Dalvik bytecode (classes.dex) Source code (.java) Libraries (.class) XML resources (.xml) Assets Runtime stubs (.class)
  • 10. ProGuard Java bytecode (.class) ProGuard Libraries (.class) Optimized Java bytecode (.class) Dx Javac Assets Compiled XML resources (.xml) Aapt Dalvik bytecode (classes.dex) ProGuard config (.txt) ProGuard mapping (.txt) Source code (.java) Libraries (.class) XML resources (.xml) Assets Runtime stubs (.class)
  • 11. ProGuard Java bytecode (.class) ProGuard Libraries (.class) Optimized Java bytecode (.class) Dx Javac Assets Compiled XML resources (.xml) Aapt Dalvik bytecode (classes.dex) ProGuard config (.txt) ProGuard mapping (.txt) Source code (.java) Libraries (.class) XML resources (.xml) Assets Runtime stubs (.class)
  • 12. Jack and Jill Libraries (.jayce) Jack Assets Compiled XML resources (.xml) Aapt Dalvik bytecode (classes.dex) ProGuard config (.txt) ProGuard mapping (.txt) Source code (.java) Libraries (.class) XML resources (.xml) Assets Runtime stubs (.class) Jill Jill Runtime (.jayce)
  • 13. Code representations for (int i = 0; i < 8; i++) ...for (int i = 0; i < 8; i++) ... iconst 0 istore v0 iload v0 bipush 8 ificmpge +12 ... iinc v0, 1 goto -12 iconst 0 istore v0 iload v0 bipush 8 ificmpge +12 ... iinc v0, 1 goto -12 .class .dex .java Javac Dx const/4 v0, #0 const/16 v1, #8 if-ge v0, v1, 8 ... add-int/lit8 v0, v0, #1 goto -9 const/4 v0, #0 const/16 v1, #8 if-ge v0, v1, 8 ... add-int/lit8 v0, v0, #1 goto -9
  • 14. Code representations for (int i = 0; i < 8; i++) ...for (int i = 0; i < 8; i++) ... iconst 0 istore v0 iload v0 bipush 8 ificmpge +12 ... iinc v0, 1 goto -12 iconst 0 istore v0 iload v0 bipush 8 ificmpge +12 ... iinc v0, 1 goto -12 .class .dex .java Javac Efficient to store and execute Inconvenient to process Dx const/4 v0, #0 const/16 v1, #8 if-ge v0, v1, 8 ... add-int/lit8 v0, v0, #1 goto -9 const/4 v0, #0 const/16 v1, #8 if-ge v0, v1, 8 ... add-int/lit8 v0, v0, #1 goto -9
  • 15. LOCAL int n FOR_STATEMENT: - EXPRESSION_STATEMENT: - ASG_OPERATION: - LOCAL_REF i - INT_LITERAL 0 - LT_OPERATION: - LOCAL_REF i - INT_LITERAL 8 - POSTFIX_INC_OPERATION: - LOCAL_REF i - ... LOCAL int n FOR_STATEMENT: - EXPRESSION_STATEMENT: - ASG_OPERATION: - LOCAL_REF i - INT_LITERAL 0 - LT_OPERATION: - LOCAL_REF i - INT_LITERAL 8 - POSTFIX_INC_OPERATION: - LOCAL_REF i - ... Code representations for (int i = 0; i < 8; i++) ...for (int i = 0; i < 8; i++) ... iconst 0 istore v0 iload v0 bipush 8 ificmpge +12 ... iinc v0, 1 goto -12 iconst 0 istore v0 iload v0 bipush 8 ificmpge +12 ... iinc v0, 1 goto -12 .class .dex .java Javac .jayce Jack const/4 v0, #0 const/16 v1, #8 if-ge v0, v1, 8 ... add-int/lit8 v0, v0, #1 goto -9 const/4 v0, #0 const/16 v1, #8 if-ge v0, v1, 8 ... add-int/lit8 v0, v0, #1 goto -9 Jill Jack
  • 16. Implications of Jack and Jill ● Java annotation processors – supported in Jack E.g. Dagger, Butterknife ● Bytecode processors – only supported via Jill E.g. JaCoCo, Retrolambda ● Other JVM languages – only supported via Jill E.g. Scala, Groovy, Dart ● Instant Run – not supported yet
  • 17. Implications of Jack and Jill Libraries (.jayce) Jack Assets Compiled XML resources (.xml) Aapt Dalvik bytecode (classes.dex) Javac annotation processors E.g. Dagger, Butterknife Source code (.scala,...) Libraries (.class) XML resources (.xml) Assets Runtime (.class) Jill Jill Runtime (.jayce) Bytecode processors Java bytecode (.class)Scalac Application (.jayce)Jill
  • 18. Internals Jack: Java frontend Eclipse ecj library Jill: Java frontend ASM4 library
  • 19. Java 8: closures ..... button.setOnClickListener(view -> .....); ..... ..... button.setOnClickListener(view -> .....); ..... ..... button.setOnClickListener(new MyActivity$LambdaImpl()); ..... ..... button.setOnClickListener(new MyActivity$LambdaImpl()); ..... public class MyActivity$LambdaImpl { public void onClick(View view) { ..... } } public class MyActivity$LambdaImpl { public void onClick(View view) { ..... } } Jack For all versions of Android
  • 20. Device More changes! RunDalvik bytecode (classes.dex) Source code (.java) Libraries (.class) Runtime stubs (.class) Build Dalvik bytecode (classes.oat) Android runtimeJack & Jill
  • 21. OpenJDK RunDalvik bytecode (classes.dex) Source code (.java) Libraries (.class) Build Dalvik bytecode (classes.oat) Android + OpenJDK runtime Android + OpenJDK runtime stubs (.class) OpenJDK Device OpenJDK
  • 22. Java 8: default methods ● In your own code: ● In the Android runtime: public interface java.util.List<E> extends java.util.Collection<E> { public abstract int size(); ... public default void sort(java.util.Comparator<? super E>) { ..... } } public interface java.util.List<E> extends java.util.Collection<E> { public abstract int size(); ... public default void sort(java.util.Comparator<? super E>) { ..... } } public interface Foo { public abstract void someMethod(); public void someDefaultMethod() { .... } } public interface Foo { public abstract void someMethod(); public void someDefaultMethod() { .... } } Only on Android N
  • 23. New instructions Dalvik bytecode RunDalvik bytecode (classes.dex) Source code (.java) Libraries (.class) Build Dalvik bytecode (classes.oat) Android + OpenJDK runtime Android + OpenJDK runtime stubs (.class) New instructions Device
  • 24. Java 8: closures ..... ● put-field-lambda ● get-field-lambda ● put-static-lambda ● get-static-lambda ● aget-lambda ● aput-lambda ● move-lambda ● cmp-lambda ● return-lambda ..... ● put-field-lambda ● get-field-lambda ● put-static-lambda ● get-static-lambda ● aget-lambda ● aput-lambda ● move-lambda ● cmp-lambda ● return-lambda Only on Android N+ New Dalvik instructions:
  • 25. Build performance 44s – Javac + dx (clean build) 1m9s – Javac + ProGuard optimize + dx (clean build) 24s – Javac + DexGuard (clean build) 2m36s – Javac + DexGuard optimize (clean build) 1m17s – Jack (clean build) 47s – Jack (clear all dex files) 15s – Javac + dx (clear all class files)15s – Javac + dx (clear all class files) 5s – Javac + dx (clear one class file)5s – Javac + dx (clear one class file) 2s – Javac + dx (code unchanged)2s – Javac + dx (code unchanged) 40s – Jack (clear one dex file)40s – Jack (clear one dex file) 3s – Jack (code unchanged)3s – Jack (code unchanged) Google I/O app (AMD A10-7800, Ubuntu 14.04, Java 8, Android Gradle 2.1.0 alpha1, Gradle daemon) 49s – Jack minify (clear all dex files)49s – Jack minify (clear all dex files) 21s – Javac + ProGuard + dx (clean build)21s – Javac + ProGuard + dx (clean build)
  • 26. Application runtime performance 50,000 – Javac + dx 63,800 – Javac + ProGuard optimize + dx 82,300 – Javac + DexGuard optimize 54,500 – Jack Caffeinemark benchmark (Nexus 5, Android 6.0, ART) +9% +28% +65% (higher is better)
  • 27. Application size 3.1M – Javac + dx 1.9M – Javac + ProGuard optimize + dx 1.8M – Javac + DexGuard optimize 1.9M – Jack + minify Google I/O app -39% -38% -44% (smaller is better)
  • 28. DexGuard Java bytecode (.class) DexGuard Libraries (.class) Javac Optimized assets Optimized XML resources (.xml) Aapt Optimized Dalvik bytecode (classes.dex) ProGuard config (.txt) ProGuard mapping (.txt) Source code (.java) Libraries (.class) XML resources (.xml) Assets Android runtime stubs (.class) Compiled XML resources (.xml) Android runtime stubs (.class) Assets Native code (.so)Clang Source code (.c / .cpp) Optimized native code (.so)
  • 29. Conclusions ● Huge changes ● Clean implementation Internally
  • 30. Conclusions ● Huge changes ● Clean implementation ● Different build process ● New intermediary representation Internally For languages and tools
  • 31. Conclusions ● Huge changes ● Clean implementation ● More efficient builds ● Java 8 ● Different build process ● New intermediary representation Internally For languages and tools For app developers
  • 32. Further reading ● “Experimental New Android Tool Chain - Jack and Jill” http://tools.android.com/tech-docs/jackandjill ● “Java 8 Language Features” https://developer.android.com/preview/j8-jack.html ● “Jack (Java Android Compiler Kit)” https://source.android.com/source/jack.html ● AOSP repository https://android.googlesource.com/toolchain/jack/ ● “State of the ART” http://androidbackstage.blogspot.com/2016/03/episode-45-state-of-art.html
  • 33. Questions? Size Java bytecode Performance Jayce bytecode Dalvik bytecode Jack & Jill ProGuard DexGuard