SlideShare a Scribd company logo
Caffe2 on Android
Koan-Sin Tan

freedom@computer.org

March 8th, 2018

Hsinchu Coding Serfs Meeting
Quick Intro
• Caffe 2

• 2nd generation of Caffe, which was the most popular deep learning
framework (before TensorFlow) from Berkeley

• What's the difference? Caffe2 improves Caffe 1.0 in a series of directions:

• first-class support for large-scale distributed training

• mobile deployment
• new hardware support (in addition to CPU and CUDA)

• flexibility for future directions such as quantized computation

• stress tested by the vast scale of Facebook applications
https://caffe2.ai/docs/caffe-migration.html
Caffe2 on Android
• Official Android demo

• https://caffe2.ai/docs/AI-Camera-demo-android.html, https://github.com/caffe2/
AICamera

• SqueezeNet 1.1:

• 5.8/5.7 fps on Samsung S7 and Google Pixel

• not very impressive

• OpenGL backend

• https://www.facebook.com/Caffe2AI/videos/126340488008269/

• up to 6X speedup (24 FPS) compared to CPU on high-end Android devices (e.g.
Galaxy S8) for style transfer models
https://trends.google.com/trends/explore?q=tensorflow,caffe2
• Tensorflow Lite is also looking for the possibility of
OpenGL ES backend

• https://github.com/tensorflow/tensorflow/issues/16189
What can we use on
Android now
https://github.com/caffe2/caffe2/tree/master/caffe2/mobile/contrib
Caffe2 backends for
Android I know
• ARM CPU:

• NNPACK, Eigen: quite mature

• OpenGL ES:

• OpenGL: not actively maintained (?)

• ARM Compute Library (GL ES part): newly added, still growing

• NEON, and OpenCL

• NNAPI: not fully integrated yet.
How to build
• > scripts/build_android.sh
• With that, no test command line binary test

• Caffe 2 has some tests and a simple command line benchmark tool
called speed_benchmark
> scripts/build_android.sh -DBUILD_TEST -DBUILD_BINARY

• then we can get build_android/bin/speed_benchmark and
other test binaries

• Pytorch has a good tutorial on using it, http://pytorch.org/tutorials/
advanced/super_resolution_with_caffe2.html
Some results
• > ./speed_benchmark --input_file input.blobproto --input
data --init_net init_net.pb --net predict_net.pb --
caffe2_log_level=0

01-06 23:15:42.073 32623 32623 I native : [I net_simple.cc:101] Starting benchmark.
01-06 23:15:42.074 32623 32623 I native : [I net_simple.cc:102] Running warmup runs.
01-06 23:15:42.074 32623 32623 I native : [I net_simple.cc:112] Main runs.
01-06 23:15:43.805 32623 32623 I native : [I net_simple.cc:123] Main run finished. Milliseconds per iter:
173.15. Iters per second: 5.77535
Some results
• ARM Compute Library backend: Caffe2 addend a Compute Libarry backend on in the end of Februrary 2018. With some tweaks, it's
possible to run SqueezeNet 1.1 faster than CPU (NNPAC) with OpenGL

01-04 03:41:38.297 25523 25523 I native : [I gl_model_test.h:52] [C2DEBUG] Benchmarking OpenGL Net

01-04 03:41:38.297 25523 25523 I native : [I net_gl.cc:104] Starting benchmark.

01-04 03:41:38.297 25523 25523 I native : [I net_gl.cc:105] Running warmup runs.

01-04 03:41:38.796 25523 25523 I native : [I net_gl.cc:121] Main runs.

01-04 03:41:43.107 25523 25523 I native : [I net_gl.cc:134] [C2DEBUG] Main run finished. Milliseconds per iter: 43.1077. Iters per
second: 23.1977

01-04 03:41:43.110 25523 25523 I native : [I gl_model_test.h:66] [C2DEBUG] Benchmarking CPU Net

01-04 03:41:43.110 25523 25523 I native : [I net_simple.cc:101] Starting benchmark.

01-04 03:41:43.110 25523 25523 I native : [I net_simple.cc:102] Running warmup runs.

01-04 03:41:43.768 25523 25523 I native : [I net_simple.cc:112] Main runs.

01-04 03:41:50.229 25523 25523 I native : [I net_simple.cc:123] Main run finished. Milliseconds per iter: 64.6136. Iters per
second: 15.4766
Comparing with TF Lite
• cmake is easier than bazel :-)

• Relatively large, or say comprehensive. If you want to enable something like on-device learning. It's
easier to start with TFLite.

• binary could be large

• Code looks cleaning

• Review process, or say, software engineering not as rigid as TensorFlow

• TF has a larger team (?)

• See, https://www.oreilly.com/ideas/how-the-tensorflow-team-handles-open-source-support

• Some interesting code,

• The Observer design pattern could be used to measure performance, https://en.wikipedia.org/wiki/
Observer_pattern

• https://github.com/caffe2/caffe2/tree/master/caffe2/observers
fin

More Related Content

What's hot

TFLite NNAPI and GPU Delegates
TFLite NNAPI and GPU DelegatesTFLite NNAPI and GPU Delegates
TFLite NNAPI and GPU Delegates
Koan-Sin Tan
 
Running TFLite on Your Mobile Devices, 2020
Running TFLite on Your Mobile Devices, 2020Running TFLite on Your Mobile Devices, 2020
Running TFLite on Your Mobile Devices, 2020
Koan-Sin Tan
 
Understanding Android Benchmarks
Understanding Android BenchmarksUnderstanding Android Benchmarks
Understanding Android BenchmarksKoan-Sin Tan
 
Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU
Andrés Leonardo Martinez Ortiz
 
Exploring Your Apple M1 devices with Open Source Tools
Exploring Your Apple M1 devices with Open Source ToolsExploring Your Apple M1 devices with Open Source Tools
Exploring Your Apple M1 devices with Open Source Tools
Koan-Sin Tan
 
Dark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source SolutionsDark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source SolutionsKoan-Sin Tan
 
Exploring Thermal Related Stuff in iDevices using Open-Source Tool
Exploring Thermal Related Stuff in iDevices using Open-Source ToolExploring Thermal Related Stuff in iDevices using Open-Source Tool
Exploring Thermal Related Stuff in iDevices using Open-Source Tool
Koan-Sin Tan
 
OpenCL - The Open Standard for Heterogeneous Parallel Programming
OpenCL - The Open Standard for Heterogeneous Parallel ProgrammingOpenCL - The Open Standard for Heterogeneous Parallel Programming
OpenCL - The Open Standard for Heterogeneous Parallel Programming
Andreas Schreiber
 
Hacking - high school intro
Hacking - high school introHacking - high school intro
Hacking - high school intro
Peter Hlavaty
 
One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...
One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...
One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...
Priyanka Aash
 
Minko - Scripting 3D apps with Lua and C++
Minko - Scripting 3D apps with Lua and C++Minko - Scripting 3D apps with Lua and C++
Minko - Scripting 3D apps with Lua and C++Minko3D
 
180 nm Tape out experience using Open POWER ISA
180 nm Tape out experience using Open POWER ISA180 nm Tape out experience using Open POWER ISA
180 nm Tape out experience using Open POWER ISA
Ganesan Narayanasamy
 
Rainbow Over the Windows: More Colors Than You Could Expect
Rainbow Over the Windows: More Colors Than You Could ExpectRainbow Over the Windows: More Colors Than You Could Expect
Rainbow Over the Windows: More Colors Than You Could Expect
Peter Hlavaty
 
Is Python still production ready ? Ludovic Gasc
Is Python still production ready ? Ludovic GascIs Python still production ready ? Ludovic Gasc
Is Python still production ready ? Ludovic Gasc
Pôle Systematic Paris-Region
 
Cloud Deep Learning Chips Training & Inference
Cloud Deep Learning Chips Training & InferenceCloud Deep Learning Chips Training & Inference
Cloud Deep Learning Chips Training & Inference
Mr. Vengineer
 
Introduction to Python GUI development with Delphi for Python - Part 1: Del...
Introduction to Python GUI development with Delphi for Python - Part 1:   Del...Introduction to Python GUI development with Delphi for Python - Part 1:   Del...
Introduction to Python GUI development with Delphi for Python - Part 1: Del...
Embarcadero Technologies
 
Python for Delphi Developers - Part 2
Python for Delphi Developers - Part 2Python for Delphi Developers - Part 2
Python for Delphi Developers - Part 2
Embarcadero Technologies
 
Python on Android with Delphi FMX - The Cross Platform GUI Framework
Python on Android with Delphi FMX - The Cross Platform GUI Framework Python on Android with Delphi FMX - The Cross Platform GUI Framework
Python on Android with Delphi FMX - The Cross Platform GUI Framework
Embarcadero Technologies
 
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
Introduction to Jupyter notebook and MS Azure Machine Learning StudioIntroduction to Jupyter notebook and MS Azure Machine Learning Studio
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
Muralidharan Deenathayalan
 
Memory Optimization
Memory OptimizationMemory Optimization
Memory Optimizationguest3eed30
 

What's hot (20)

TFLite NNAPI and GPU Delegates
TFLite NNAPI and GPU DelegatesTFLite NNAPI and GPU Delegates
TFLite NNAPI and GPU Delegates
 
Running TFLite on Your Mobile Devices, 2020
Running TFLite on Your Mobile Devices, 2020Running TFLite on Your Mobile Devices, 2020
Running TFLite on Your Mobile Devices, 2020
 
Understanding Android Benchmarks
Understanding Android BenchmarksUnderstanding Android Benchmarks
Understanding Android Benchmarks
 
Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU
 
Exploring Your Apple M1 devices with Open Source Tools
Exploring Your Apple M1 devices with Open Source ToolsExploring Your Apple M1 devices with Open Source Tools
Exploring Your Apple M1 devices with Open Source Tools
 
Dark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source SolutionsDark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source Solutions
 
Exploring Thermal Related Stuff in iDevices using Open-Source Tool
Exploring Thermal Related Stuff in iDevices using Open-Source ToolExploring Thermal Related Stuff in iDevices using Open-Source Tool
Exploring Thermal Related Stuff in iDevices using Open-Source Tool
 
OpenCL - The Open Standard for Heterogeneous Parallel Programming
OpenCL - The Open Standard for Heterogeneous Parallel ProgrammingOpenCL - The Open Standard for Heterogeneous Parallel Programming
OpenCL - The Open Standard for Heterogeneous Parallel Programming
 
Hacking - high school intro
Hacking - high school introHacking - high school intro
Hacking - high school intro
 
One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...
One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...
One bite and all your dreams will come true: Analyzing and Attacking Apple Ke...
 
Minko - Scripting 3D apps with Lua and C++
Minko - Scripting 3D apps with Lua and C++Minko - Scripting 3D apps with Lua and C++
Minko - Scripting 3D apps with Lua and C++
 
180 nm Tape out experience using Open POWER ISA
180 nm Tape out experience using Open POWER ISA180 nm Tape out experience using Open POWER ISA
180 nm Tape out experience using Open POWER ISA
 
Rainbow Over the Windows: More Colors Than You Could Expect
Rainbow Over the Windows: More Colors Than You Could ExpectRainbow Over the Windows: More Colors Than You Could Expect
Rainbow Over the Windows: More Colors Than You Could Expect
 
Is Python still production ready ? Ludovic Gasc
Is Python still production ready ? Ludovic GascIs Python still production ready ? Ludovic Gasc
Is Python still production ready ? Ludovic Gasc
 
Cloud Deep Learning Chips Training & Inference
Cloud Deep Learning Chips Training & InferenceCloud Deep Learning Chips Training & Inference
Cloud Deep Learning Chips Training & Inference
 
Introduction to Python GUI development with Delphi for Python - Part 1: Del...
Introduction to Python GUI development with Delphi for Python - Part 1:   Del...Introduction to Python GUI development with Delphi for Python - Part 1:   Del...
Introduction to Python GUI development with Delphi for Python - Part 1: Del...
 
Python for Delphi Developers - Part 2
Python for Delphi Developers - Part 2Python for Delphi Developers - Part 2
Python for Delphi Developers - Part 2
 
Python on Android with Delphi FMX - The Cross Platform GUI Framework
Python on Android with Delphi FMX - The Cross Platform GUI Framework Python on Android with Delphi FMX - The Cross Platform GUI Framework
Python on Android with Delphi FMX - The Cross Platform GUI Framework
 
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
Introduction to Jupyter notebook and MS Azure Machine Learning StudioIntroduction to Jupyter notebook and MS Azure Machine Learning Studio
Introduction to Jupyter notebook and MS Azure Machine Learning Studio
 
Memory Optimization
Memory OptimizationMemory Optimization
Memory Optimization
 

Similar to Caffe2 on Android

Community works for muli core embedded image processing
Community works for muli core embedded image processingCommunity works for muli core embedded image processing
Community works for muli core embedded image processingJeongpyo Kong
 
Cuda
CudaCuda
Introduction to OpenCL
Introduction to OpenCLIntroduction to OpenCL
Introduction to OpenCL
Unai Lopez-Novoa
 
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
Nick Galbreath
 
"The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ...
"The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ..."The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ...
"The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ...
Edge AI and Vision Alliance
 
Mesa and Its Debugging, Вадим Шовкопляс
Mesa and Its Debugging, Вадим ШовкоплясMesa and Its Debugging, Вадим Шовкопляс
Mesa and Its Debugging, Вадим Шовкопляс
Sigma Software
 
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
Sencha
 
How to Puppetize Google Cloud Platform - PuppetConf 2014
How to Puppetize Google Cloud Platform - PuppetConf 2014How to Puppetize Google Cloud Platform - PuppetConf 2014
How to Puppetize Google Cloud Platform - PuppetConf 2014
Puppet
 
From Zero to Hero - All you need to do serious deep learning stuff in R
From Zero to Hero - All you need to do serious deep learning stuff in R From Zero to Hero - All you need to do serious deep learning stuff in R
From Zero to Hero - All you need to do serious deep learning stuff in R
Kai Lichtenberg
 
Trends in Systems and How to Get Efficient Performance
Trends in Systems and How to Get Efficient PerformanceTrends in Systems and How to Get Efficient Performance
Trends in Systems and How to Get Efficient Performance
inside-BigData.com
 
september13.ppt
september13.pptseptember13.ppt
september13.ppt
CharlesMatu2
 
Modern Web-site Development Pipeline
Modern Web-site Development PipelineModern Web-site Development Pipeline
Modern Web-site Development Pipeline
GlobalLogic Ukraine
 
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with KubernetesKubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
SeungYong Oh
 
ASP.NET Core: The best of the new bits
ASP.NET Core: The best of the new bitsASP.NET Core: The best of the new bits
ASP.NET Core: The best of the new bits
Ken Cenerelli
 
Introduction to Civil Infrastructure Platform
Introduction to Civil Infrastructure PlatformIntroduction to Civil Infrastructure Platform
Introduction to Civil Infrastructure Platform
SZ Lin
 
Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...
Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...
Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...
gree_tech
 
CG-Orientation ppt.pptx
CG-Orientation ppt.pptxCG-Orientation ppt.pptx
CG-Orientation ppt.pptx
1AT19CS125YeshwanthS
 
Bring-your-ML-Project-into-Production-v2.pdf
Bring-your-ML-Project-into-Production-v2.pdfBring-your-ML-Project-into-Production-v2.pdf
Bring-your-ML-Project-into-Production-v2.pdf
Liang Yan
 
Getting Started with Apache Spark on Kubernetes
Getting Started with Apache Spark on KubernetesGetting Started with Apache Spark on Kubernetes
Getting Started with Apache Spark on Kubernetes
Databricks
 
Using Deep Learning Toolkits with Kubernetes clusters
Using Deep Learning Toolkits with Kubernetes clustersUsing Deep Learning Toolkits with Kubernetes clusters
Using Deep Learning Toolkits with Kubernetes clusters
Joy Qiao
 

Similar to Caffe2 on Android (20)

Community works for muli core embedded image processing
Community works for muli core embedded image processingCommunity works for muli core embedded image processing
Community works for muli core embedded image processing
 
Cuda
CudaCuda
Cuda
 
Introduction to OpenCL
Introduction to OpenCLIntroduction to OpenCL
Introduction to OpenCL
 
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
Care and Feeding of Large Scale Graphite Installations - DevOpsDays Austin 2013
 
"The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ...
"The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ..."The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ...
"The Caffe2 Framework for Mobile and Embedded Deep Learning," a Presentation ...
 
Mesa and Its Debugging, Вадим Шовкопляс
Mesa and Its Debugging, Вадим ШовкоплясMesa and Its Debugging, Вадим Шовкопляс
Mesa and Its Debugging, Вадим Шовкопляс
 
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe
 
How to Puppetize Google Cloud Platform - PuppetConf 2014
How to Puppetize Google Cloud Platform - PuppetConf 2014How to Puppetize Google Cloud Platform - PuppetConf 2014
How to Puppetize Google Cloud Platform - PuppetConf 2014
 
From Zero to Hero - All you need to do serious deep learning stuff in R
From Zero to Hero - All you need to do serious deep learning stuff in R From Zero to Hero - All you need to do serious deep learning stuff in R
From Zero to Hero - All you need to do serious deep learning stuff in R
 
Trends in Systems and How to Get Efficient Performance
Trends in Systems and How to Get Efficient PerformanceTrends in Systems and How to Get Efficient Performance
Trends in Systems and How to Get Efficient Performance
 
september13.ppt
september13.pptseptember13.ppt
september13.ppt
 
Modern Web-site Development Pipeline
Modern Web-site Development PipelineModern Web-site Development Pipeline
Modern Web-site Development Pipeline
 
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with KubernetesKubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
 
ASP.NET Core: The best of the new bits
ASP.NET Core: The best of the new bitsASP.NET Core: The best of the new bits
ASP.NET Core: The best of the new bits
 
Introduction to Civil Infrastructure Platform
Introduction to Civil Infrastructure PlatformIntroduction to Civil Infrastructure Platform
Introduction to Civil Infrastructure Platform
 
Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...
Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...
Common Pitfalls of Functional Programming and How to Avoid Them: A Mobile Gam...
 
CG-Orientation ppt.pptx
CG-Orientation ppt.pptxCG-Orientation ppt.pptx
CG-Orientation ppt.pptx
 
Bring-your-ML-Project-into-Production-v2.pdf
Bring-your-ML-Project-into-Production-v2.pdfBring-your-ML-Project-into-Production-v2.pdf
Bring-your-ML-Project-into-Production-v2.pdf
 
Getting Started with Apache Spark on Kubernetes
Getting Started with Apache Spark on KubernetesGetting Started with Apache Spark on Kubernetes
Getting Started with Apache Spark on Kubernetes
 
Using Deep Learning Toolkits with Kubernetes clusters
Using Deep Learning Toolkits with Kubernetes clustersUsing Deep Learning Toolkits with Kubernetes clusters
Using Deep Learning Toolkits with Kubernetes clusters
 

Recently uploaded

Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 

Recently uploaded (20)

Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 

Caffe2 on Android

  • 1. Caffe2 on Android Koan-Sin Tan freedom@computer.org March 8th, 2018 Hsinchu Coding Serfs Meeting
  • 2. Quick Intro • Caffe 2 • 2nd generation of Caffe, which was the most popular deep learning framework (before TensorFlow) from Berkeley • What's the difference? Caffe2 improves Caffe 1.0 in a series of directions: • first-class support for large-scale distributed training • mobile deployment • new hardware support (in addition to CPU and CUDA) • flexibility for future directions such as quantized computation • stress tested by the vast scale of Facebook applications https://caffe2.ai/docs/caffe-migration.html
  • 3. Caffe2 on Android • Official Android demo • https://caffe2.ai/docs/AI-Camera-demo-android.html, https://github.com/caffe2/ AICamera • SqueezeNet 1.1: • 5.8/5.7 fps on Samsung S7 and Google Pixel • not very impressive • OpenGL backend • https://www.facebook.com/Caffe2AI/videos/126340488008269/ • up to 6X speedup (24 FPS) compared to CPU on high-end Android devices (e.g. Galaxy S8) for style transfer models
  • 5. • Tensorflow Lite is also looking for the possibility of OpenGL ES backend • https://github.com/tensorflow/tensorflow/issues/16189
  • 6. What can we use on Android now https://github.com/caffe2/caffe2/tree/master/caffe2/mobile/contrib
  • 7. Caffe2 backends for Android I know • ARM CPU: • NNPACK, Eigen: quite mature • OpenGL ES: • OpenGL: not actively maintained (?) • ARM Compute Library (GL ES part): newly added, still growing • NEON, and OpenCL • NNAPI: not fully integrated yet.
  • 8. How to build • > scripts/build_android.sh • With that, no test command line binary test • Caffe 2 has some tests and a simple command line benchmark tool called speed_benchmark > scripts/build_android.sh -DBUILD_TEST -DBUILD_BINARY • then we can get build_android/bin/speed_benchmark and other test binaries • Pytorch has a good tutorial on using it, http://pytorch.org/tutorials/ advanced/super_resolution_with_caffe2.html
  • 9. Some results • > ./speed_benchmark --input_file input.blobproto --input data --init_net init_net.pb --net predict_net.pb -- caffe2_log_level=0 01-06 23:15:42.073 32623 32623 I native : [I net_simple.cc:101] Starting benchmark. 01-06 23:15:42.074 32623 32623 I native : [I net_simple.cc:102] Running warmup runs. 01-06 23:15:42.074 32623 32623 I native : [I net_simple.cc:112] Main runs. 01-06 23:15:43.805 32623 32623 I native : [I net_simple.cc:123] Main run finished. Milliseconds per iter: 173.15. Iters per second: 5.77535
  • 10. Some results • ARM Compute Library backend: Caffe2 addend a Compute Libarry backend on in the end of Februrary 2018. With some tweaks, it's possible to run SqueezeNet 1.1 faster than CPU (NNPAC) with OpenGL 01-04 03:41:38.297 25523 25523 I native : [I gl_model_test.h:52] [C2DEBUG] Benchmarking OpenGL Net 01-04 03:41:38.297 25523 25523 I native : [I net_gl.cc:104] Starting benchmark. 01-04 03:41:38.297 25523 25523 I native : [I net_gl.cc:105] Running warmup runs. 01-04 03:41:38.796 25523 25523 I native : [I net_gl.cc:121] Main runs. 01-04 03:41:43.107 25523 25523 I native : [I net_gl.cc:134] [C2DEBUG] Main run finished. Milliseconds per iter: 43.1077. Iters per second: 23.1977 01-04 03:41:43.110 25523 25523 I native : [I gl_model_test.h:66] [C2DEBUG] Benchmarking CPU Net 01-04 03:41:43.110 25523 25523 I native : [I net_simple.cc:101] Starting benchmark. 01-04 03:41:43.110 25523 25523 I native : [I net_simple.cc:102] Running warmup runs. 01-04 03:41:43.768 25523 25523 I native : [I net_simple.cc:112] Main runs. 01-04 03:41:50.229 25523 25523 I native : [I net_simple.cc:123] Main run finished. Milliseconds per iter: 64.6136. Iters per second: 15.4766
  • 11. Comparing with TF Lite • cmake is easier than bazel :-) • Relatively large, or say comprehensive. If you want to enable something like on-device learning. It's easier to start with TFLite. • binary could be large • Code looks cleaning • Review process, or say, software engineering not as rigid as TensorFlow • TF has a larger team (?) • See, https://www.oreilly.com/ideas/how-the-tensorflow-team-handles-open-source-support • Some interesting code, • The Observer design pattern could be used to measure performance, https://en.wikipedia.org/wiki/ Observer_pattern • https://github.com/caffe2/caffe2/tree/master/caffe2/observers
  • 12. fin