Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OSG(a)i: because AI needs a runtime - Tim Verbelen (imec)

33 views

Published on

OSGi Community Event 2018 Presentation by Tim Verbelen (imec)

Abstract: Nowadays AI is reaching new heights on the hype cycle, especially due to recent advances in deep learning techniques. A lot of deep learning frameworks exist for creating and training deep neural networks, the most popular ones being PyTorch and TensorFlow. However, how to integrate, deploy and manage these neural networks in complex software systems is often overlooked. In this talk we show how OSGi can be used as a modular runtime for deep learning models. We embed those models inside OSGi bundles, and use the extender pattern to make these available as OSGi services. You can then use your favorite OSGi specs such as DS and PushStreams to integrate these into your application.

Published in: Technology
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

OSG(a)i: because AI needs a runtime - Tim Verbelen (imec)

  1. 1. PUBLIC1 OSG(A)I: BECAUSE A.I. NEEDS A RUNTIME TIM VERBELEN Senior Researcher imec – Ghent University
  2. 2. PUBLIC WHAT IS A.I.? 2 THE NEXT BIG THING (AFTER BLOCKCHAIN OFC!)
  3. 3. PUBLIC WHAT IS MACHINE LEARNING? 3 LEARNING A FUNCTION APPROXIMATION MAPPING INPUTS TO OUTPUTS ModelModel outputsinputs
  4. 4. PUBLIC WHAT IS DEEP LEARNING? 4 USING DEEP NEURAL NETWORKS AS MACHINE LEARNING MODEL Neural NetworkNeural Network “cat”
  5. 5. PUBLIC DEEP LEARNING FRAMEWORKS AND TOOLS 5
  6. 6. PUBLIC THE AVERAGE DATA SCIENTIST WORKFLOW 1. Inspect and clean up the data 2. Select and encode features / outputs 3. Script together a model training procedure 4. Find some good hyperparameters 5. Dump the trained model 6
  7. 7. PUBLIC BUT WHAT ABOUT PRODUCTION? 7
  8. 8. PUBLIC TENSORFLOW SERVING 8 Client Servable Handler Dynamic Manager VersionPolicy models/ 1/ … 2/ … Server gRPC/REST request
  9. 9. PUBLIC CHECK THE BOXES I want to query my models with an RPC/REST call I train my models using TensorFlow I deploy my models on a containerized infrastructure I don’t need additional metadata/versioning besides a single incrementing integer 9
  10. 10. PUBLIC OSGI CAN HELP!  A unit of deployment  Package ML model as an OSGi bundle  A resolveable artifact with requirements and capabilities  Requirements: what do you need to run this ML model  Capabilities: what kind of inputs can it process and what kind of outputs does it give you?  A lightweight service model  Access your model via an OSGi service  Service selection at runtime  Use service properties and target filters to select the best model at runtime 10
  11. 11. PUBLIC A WAY TO SHARE ML MODELS A common format for describing computation graphs  defined as protocol buffers  specifies data types and operators  framework agnostic 11 OPEN NEURAL NETWORK EXCHANGE (ONNX)
  12. 12. PUBLIC A WAY TO EXECUTE MODELS IN OSGI  A modular deep learning framework developed in OSGi  Neural networks, operators, datasets, learners, … as an OSGi service  Low-level operations via JNI and blas/cublas/cudnn backends  Distributed deployments using OSGi remote services  Web UI to build, deploy, train models 12 DIANNE FRAMEWORK http://dianne.intec.ugent.be
  13. 13. PUBLIC DEMO TIME 13
  14. 14. PUBLIC NOW LET’S ADD A ROBOT 14
  15. 15. PUBLIC NOW LET’S ADD A ROBOT 15
  16. 16. PUBLIC PUTTING THINGS TOGETHER 16 @Reference Camera camera; @Reference NeuralNetwork nn; @Reference ArmController controller; public Promise<Boolean> grasp(){ return camera.stream() // returns PushStream<Frame> // timeout after 10 seconds .timeout(Duration.ofSeconds(10)) // process camera frame with neural network .map(frame -> nn.forward(toTensor(frame)) // use result to update controller .map(r -> controller.update(r)) // short circuit in case of success .anyMatch(s -> s); }
  17. 17. PUBLIC NOW LET’S ADD A ROBOT 17
  18. 18. PUBLIC18
  19. 19. PUBLIC19 THANK YOU! http://dianne.intec.ugent.be

×