Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tips for better CI on Android

8,495 views

Published on

This slides explains how Mercari work on Continuous Integration and presents some useful tips to start better CI on Android.

Published in: Software

Tips for better CI on Android

  1. 1. Tips for better CI on Android Tomoaki Imai @Potatotips #22 2015/10/13
  2. 2. Hi I’m Tomo(今井智章)
  3. 3. Hi I’m Tomo(今井智章)
  4. 4. Hi I’m Tomo(今井智章)
  5. 5. Hi I’m Tomo(今井智章)
  6. 6. Hi I’m Tomo(今井智章)
  7. 7. Continuous Integration on Android
  8. 8. 1. How do we CI on Android 2. 4 tips for better CI
  9. 9. 1. How do we CI on Android 2. 4 tips for better CI
  10. 10. How do we CI Mercari Android App?
  11. 11. CI for Mercari Android App • Started April 2015 • Purpose - Build Check each Flavour and BuildType - Automate app tests - Continuous deployment
  12. 12. Phase 1: Travis CI
  13. 13. Good!! • Easy setup with yml • Build production app without including secret keys into the repo(http://docs.travis-ci.com/user/encrypting- files/) • Most of Mercari project run CI on Travis CI
  14. 14. So-so… • Need some tweaks to avoid Out of Memory • Slow. Takes 12-15min/per build check
  15. 15. Phase 2: circleCI
  16. 16. Good!! • Has better features for testing and debugging • Bit faster than Travis CI (1- 2min for each build) • Mercari iOS project runs CI on CircleCI
  17. 17. So-so… • Need some tweaks to avoid Out of Memory • Building production app looked a bit tricky (Haven’t tried yet)
  18. 18. Both CI service have good parts…
  19. 19. Why not use both?
  20. 20. Phase 3: Travis CI & circleCI
  21. 21. Check Production App Check Development App Run JUnit Tests Deploy Let each CI service do different tasks
  22. 22. Total CI Running time reduced to 8-7 min(about 2x shorter)
  23. 23. 1. How do we CI on Android 2. 4 tips for better CI
  24. 24. Tip 1: Avoid OOM(CircleCI) • 4GB memory limit
  25. 25. Tip 1: Avoid OOM(CircleCI) • 4GB memory limit
  26. 26. Tip 1: Avoid OOM(CircleCI) https://circleci.com/docs/oom but was in vain… Guide for 4GB limit
  27. 27. Check memory_usage.txt
  28. 28. PID RSS %CPU COMMAND 21787 2250024 165 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xmx4g - Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /usr/ local/android-sdk-linux/build-tools/21.1.1/lib/dx.jar com.android.dx.command.Main -- dex --force-jumbo --output /home/ubuntu/mercari-android-as/app/build/intermediates/ dex/us/dev /home/ubuntu/mercari-android-as/app/build/intermediates/classes-proguard/ us/dev/classes.jar 9477 1628300 104 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -XX: +HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US - Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-2.2.1- all/6dibv5rcnnqlfbq9klf8imrndn/gradle-2.2.1/lib/gradle-launcher-2.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 /home/ubuntu/.gradle/daemon 10800000 b3531fa1-fad3-4599-a284-96a81cc9336b -XX:+HeapDumpOnOutOfMemoryError - Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant 9145 279164 4.7 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m - Dorg.gradle.jvmargs=-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError - Dorg.gradle.appname=gradlew -classpath /home/ubuntu/mercari-android-as/gradle/wrapper/ gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain assembleDev -PdisablePreDex Tip 1: Avoid OOM(CircleCI)
  29. 29. PID RSS %CPU COMMAND 21787 2250024 165 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xmx4g - Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /usr/ local/android-sdk-linux/build-tools/21.1.1/lib/dx.jar com.android.dx.command.Main -- dex --force-jumbo --output /home/ubuntu/mercari-android-as/app/build/intermediates/ dex/us/dev /home/ubuntu/mercari-android-as/app/build/intermediates/classes-proguard/ us/dev/classes.jar 9477 1628300 104 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -XX: +HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US - Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-2.2.1- all/6dibv5rcnnqlfbq9klf8imrndn/gradle-2.2.1/lib/gradle-launcher-2.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 /home/ubuntu/.gradle/daemon 10800000 b3531fa1-fad3-4599-a284-96a81cc9336b -XX:+HeapDumpOnOutOfMemoryError - Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant 9145 279164 4.7 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m - Dorg.gradle.jvmargs=-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError - Dorg.gradle.appname=gradlew -classpath /home/ubuntu/mercari-android-as/gradle/wrapper/ gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain assembleDev -PdisablePreDex Tip 1: Avoid OOM(CircleCI) JVM is using too much memory! 2250024 + 1628300 + 279164 = 4157488KB ≒4GB
  30. 30. PID RSS %CPU COMMAND 21787 2250024 165 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xmx4g - Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /usr/ local/android-sdk-linux/build-tools/21.1.1/lib/dx.jar com.android.dx.command.Main -- dex --force-jumbo --output /home/ubuntu/mercari-android-as/app/build/intermediates/ dex/us/dev /home/ubuntu/mercari-android-as/app/build/intermediates/classes-proguard/ us/dev/classes.jar 9477 1628300 104 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -XX: +HeapDumpOnOutOfMemoryError -Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US - Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-2.2.1- all/6dibv5rcnnqlfbq9klf8imrndn/gradle-2.2.1/lib/gradle-launcher-2.2.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 2.2.1 /home/ubuntu/.gradle/daemon 10800000 b3531fa1-fad3-4599-a284-96a81cc9336b -XX:+HeapDumpOnOutOfMemoryError - Xmx2048m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant 9145 279164 4.7 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m - Dorg.gradle.jvmargs=-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError - Dorg.gradle.appname=gradlew -classpath /home/ubuntu/mercari-android-as/gradle/wrapper/ gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain assembleDev -PdisablePreDex Tip 1: Avoid OOM(CircleCI) Progard + Gradle consumes memory a lot
  31. 31. Tip 1: Avoid OOM(CircleCI) Tuning tips: • Lower the total memory(JVM,Gradle) size to 4.0G • Adjust Xmx and MaxPermSize • Set timeout (>1000) for GC Xmx -> Heap space (where object resources are allocated ) MaxPermSize -> Permgen(where class, methods are loaded)
  32. 32. Tip 1: Avoid OOM(CircleCI) machine:
 environment:
 ANDROID_HOME: /usr/local/android-sdk-linux
 _JAVA_OPTIONS: "-Xms256m -Xmx1280m -XX:MaxPermSize=350m"
 GRADLE_OPTS: '-Dorg.gradle.jvmargs="-Xmx1536m - XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError"' … override:
 - ./gradlew assembleDev -PdisablePreDex:
 timeout: 1500 Heap Perm Heap Perm other JVM(dex) Gradle 4GB • There is no best practice but I suggest that large memory be allocated to Xmx
  33. 33. Tip 2: Save the JUnit test result(CircleCI) test:
 post:
 - mkdir -p $CIRCLE_TEST_REPORTS/junit/
 - find . -type f -regex ".*/test-results/*/.*xml" -exec cp {} $CIRCLE_TEST_REPORTS/junit/ ; • You can check Test result from ‘Test Failure’ http://qiita.com/tomoima525/items/b4a91d1624aaf0112b3f Ref. CircleCIでJUnitの結果を見る
  34. 34. Tip 3: Continuous deployment for specific branches(CircleCI) • Run shell script for specific branches deployment:
 jp:
 branch: [/deploy_jp_.*/, /release_.*/]
 commands:
 - ./deploy_jp.sh $CIRCLE_BRANCH
 us:
 branch: [/deploy_us_.*/, /release_.*/]
 commands:
 - ./deploy_us.sh $CIRCLE_BRANCH RELEASE_MESSAGE=$1 #deploy
 curl -F "file=@app/apk/mercari_dev_jp.apk" -F "token=$ {DEPLOY_GATE_API_KEY}" -F "message=${RELEASE_MESSAGE}" -F "distribution_key=${JP_DEV_DIST_KEY}" -F "release_note=$ {RELEASE_MESSAGE}" https://deploygate.com/api/users/xxxx/apps deploy_jp.sh
  35. 35. Tip 3: Continuous deployment for specific branches(CircleCI) • Set variables at `Environment variables`
  36. 36. Tip 4: Add dependencies manually dependencies: override: - echo y | android update sdk --no-ui --all --filter extra- google-m2repository,extra-android-m2repository,build-tools-23.0.1, android-23 android: components: - build-tools-23.0.1 - android-23 - extra-android-m2repository - extra-google-m2repository .travis.yml circle.yml • The latest packages(ex. Android M) are not installed on the VM • Add manually with command line
  37. 37. • Tip 1: Check memory usage of JVM to avoid OOM • Tip 2: Save the JUnit test result on the VM • Tip 3: Continuous deployment for specific branches • Tip 4: Add dependencies manually
  38. 38. Try it out for your project!

×