Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bringing TensorFlow to Android: a war story - Yoni Tsafir, JoyTunes

69 views

Published on

As presented in Droidcon Tel Aviv 2017:
http://il.droidcon.com

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Bringing TensorFlow to Android: a war story - Yoni Tsafir, JoyTunes

  1. 1. ⚔Bringing TensorFlow To Android - a War Story⚔ YoniTsafir, DroidconTel-Aviv 2017
  2. 2. About Me 🤓 • Tech Lead @ • Mobile Developer, new to ML • https://medium.com/@theyonibomber/ • Twitter: @theyonibomber
  3. 3. Simply Piano 🎹 • Teaches the world to play piano • Works with any piano or keyboard • iOS: 2015,Android: 2017 • Google 💗,“Best Apps of 2017” • Technically interesting: UX, rendering, audio and ML challenges
  4. 4. Machine Learning in Simply Piano 🤖 Deep Learning acoustic engine for recognizing notes: • Polyphonic, with background music and in noisy environments • Cutting-edge ML concepts • Low latency, real-time performance
  5. 5. Why on-device ML? 📱 • Real-time • Network issues • Cheap processing power • Example apps: GoogleTranslate, Not Hotdog https://play.google.com/store/apps/details?id=com.seefoodtechnologies.nothotdog&hl=en https://play.google.com/store/apps/details?id=com.google.android.apps.translate&hl=en
  6. 6. Our Old Solution 🐢 • Implement model in code • Some 3rd parties for acceleration, but most of heavy-lifting ourselves • Unlike iOS: performance & audio issues => smaller model & worse results • New model implementation takes weeks • Differences between research and production
  7. 7. What Is TensorFlow? 🤔 • Google’s Open Source ML framework • Used in out research environment • Usually:“TensorFlow Serving” • Can also be compiled to mobile…
  8. 8. TensorFlow in Production - Motivation ) • From research to production: trivial • Reduce implementation diff ‣ Except for frame rate & pre/post-processing • Better optimization => Better performance
  9. 9. Our New Solution 🐇 • Obtain frozenTensorFlow model (.pb) • Add to .apk or download • AddTensorFlow libs • Run model: // Initialize tensorflow = new TensorFlowInferenceInterface(assetManager, modelFileName); // Run single inference tensorflow.feed("input", input, shape); tensorflow.run(new String[] {"output"}); tensorflow.fetch("output", output);
  10. 10. TensorFlow Libs - Then (Mar 2017) + • Clone & compile • Little documentation, hard to troubleshoot • Didn’t support all required Ops • Specific NDK version must be used • A guide written in blood 💉
 http://joytun.es/tf-guide
  11. 11. TensorFlow Libs - Today - • Since 1.2.0, available via Gradle:
 compile('org.tensorflow:tensorflow-android:1.2.0') • November 2017,TF Lite developer beta: ‣ Optimized for mobile ‣ Needs conversion to .tflite ‣ Subset of architectures and Ops • We still compile, but you probably don’t have to! https://techcrunch.com/2017/05/17/googles-tensorflow-lite-brings-machine-learning-to-android-devices/
  12. 12. APK Size: Selective Registration 🔍 Build -> Analyze APK… ~16MB ~6MB 61% Saved
  13. 13. Step I - Compare Same Model ⚖ • Branch withTF-based infra • Manual testing: up to x4 faster • Rollout to 50% in Play Console • Compare major KPIs across a few weeks (conversion, % course completion)
  14. 14. Step I - Results 📊 • At first, old solution = 10% better conversion 😣 • Reasons: ‣ JNI leak ‣ Model wasTOO fast! • After fix - slight advantage toTF impl. https://www.klipfolio.com/resources/articles/what-is-digital-dashboard
  15. 15. Next Up - New Model Pipeline 2 • Test in research environment => Candidate • Android based debug tool (in development) • Infra improvements? 
 Staged rollout on same model • Same Infra? 
 Our internal A/B testing framework on various models
  16. 16. The War Is Not Over 3 • A performance bug I opened a few days ago: • To be continued… TF 1.0.1 TF 1.4.0
  17. 17. 4Questions? 5

×