Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2019 10-21 Java in the Age of Serverless


Published on

The use of Serverless computing has reached a point where any serious company would be foolish to ignore its cost savings and other benefits against a wide array of proven use cases. From API Management, Artificial Intelligence (AI) and Cognitive to DevOps, Mobile and the Internet-of-Things (IoT), Serverless is proving itself as an accessible means for efficiently processing and transforming data in large volumes, on-demand, with little or no operational considerations for developers. As Serverless adoption increases, what are the considerations and implications for the Java developer and as a language going forward? We will explore this question from the perspective of the speaker, who was an early Java fan and developer, but whose path diverged to projects that used other popular languages and today serves as a Serverless technologist who understands the mind-set and expectations of Serverless developers

Published in: Technology
  • Be the first to comment

  • Be the first to like this

2019 10-21 Java in the Age of Serverless

  1. 1. Java in the Age of Serverless The Path Forward … Wed, 23 October, 2:15 PM Matt Rutkowski STSM Cloud Open Tech., CTO Open Serverless Technologies Twitter: @mrutkowski, Slack: @mrutkowski 1
  2. 2. My Java Serverless Journey Against the Java release timeline… 2
  3. 3. Awakening PeriodAntiquity Living through the “Early Ages” Age of Exploration • HotSpot, RMI, JNDI, Synthetics, JavaSound, Debugger (JPDA), 2000 Java 3 ”Trade Routes” • JavaBeans, Math, Reflection, RMI, JAR, JNI, JDBC Java 1.1(.4) ”Building Roads” JDBC 1997 Java (1.)2 ”Swing Time” 1998 • Swing, JIT, Java Plug-in, Collections Swing 3 B.J.E. “Before Java Era” BJE 1996 Java 1(.02) ”The Enlightenment” • Applet, AWT, IO, Lang, Net, Util Applets
  4. 4. Age of Expansion “Go forth and Conquer!” 2009 Java 1.4 “Hardening for business” • RegEx, XML, XSLT, JAXP, IPv6, Logging, PNG JPG I/O, Security, Crypto 2002 Portlets (JSR 168) Java (1.)5 ”Developer Efficiencies” • Generics, Enums, Iterables, Static imports, Concurrency 2004 Eclipse 3.x 2006 Java (1.)6 ”Need for speed” • JavaScript, Pluggable Annotations, Updated JVM/JAXB/JDBC/Swing 2008 Ganymede P2P Broadcast Encryption Protocol (Java Ref. Impl.) 4 2001 WebSphere Transcoding Publisher Transcoders for: • Image, Video, • CGM, SVG, MS Word Renderers
  5. 5. Dark Ages ”A Cloud is Upon Us” Rediscovery 2019 Java Runtime (2018) • w/ AdoptOpenJDK8 Java 7 ??? 2011 Diablo 2014 Java 8 ??? Mitaka 2009 ”A Light Shineth” 20182016 Scala (Platform) • Incubator 2017-18 • Graduated July ‘19 GoLang (Client/CLI) 5 Renaissance Java Runtime (2016) • w/ Oracle JDK8
  6. 6. What is Serverless? 6 Thanks to some “Marketing Person”! The ideal of Serverless “in-a-nutshell” is … Event-driven programming using stand-alone functions with no deployment or operational considerations
  7. 7. Platform provider “implements” Ops − Maximizing Utilization • Compute / Memory / Networking − Guaranteeing • Performance, Scaling, Multi-region, High Availability (HA), Security, Monitoring, Logging, etc. • Service License Agreements (SLAs) may vary… − Minimizing Compute Costs • Pay-As-You-GO (PAYGO) model 7 Taking the “Ops” out of “DevOps” Serverless … “is a” misnomer − Servers exist, but are not the Developer’s concern “→ NoOps” for the Developer is #1 goal Photo by Tim Gouw The “DevOps” Blues
  8. 8. 8 Thereby … Optimizing Coding Velocity and Developer Joy Photo by Radu Florin • Implementing App and Business Logic − Using simple, Single-tasked Functions − Interacting with Discrete data − Simple, “app-friendly”, structured data like JSON − Or optionally “Raw” data from HTTP message Body • Composing Functions into compelling Workflows and Applications − Connect to Events Providers and React to Events Serverless Developers Focus on…“Joy”
  9. 9. Trigger T Function Fx Event Triggers- are named channels for a class of Events that “Feed” the Fx • Typically Named resources often backed by Message Queues • Potential Normalization of data Events- representation of real- world, “Source” events that carry actionable Input data • Manual (CLI) or Automated Events • Carrying structured or unstructured data 9 Serverless’ Least Common Denominator (LCD) Programming Model Feed Data MAY include: • Context Metadata (Protocol, Event & Operational Environment) • Optional Parameters • Raw Data Raw Event Data • Origination Context & Data Functions - relatively short- running, standalone functions invoked Reactively as an event handler Based upon Reactive, Event-Driven Functions “React”“Feed”“Source”
  10. 10. Serverless Workloads 10 Serverless typically implemented as Container or Functional Workloads Increasing ability to Horizontally Scale • Decreased Management of Infra., Frameworks, Platform & Runtime Stacks Increasing Developer Agility • Increased focus on implementing Application & Business logic Native (Bare metal) Virtual Machines (IaaS) Platform Native (PaaS) • Tied to proprietary PaaS services & workloads Allowing greater portability & maximizing hybrid & cross-Cloud scaling Embracing 12-factor Application Methodology Pseudo Serverless Functions (on FaaS) • include only function and dependent libs. Containers (on CaaS) • include function’s language and framework stack
  11. 11. Why Serverless? Eliminating Ops & Cost guesswork when creating new applications Developer Agility • build lightweight, flexible applications that can be expanded or updated quickly • Author small “glue code” that react to events “as needed” against discrete data 11 Cost Reduction • PAYGO – “Pay-As-You-GO”, Only pay for the milliseconds of CPU used for running the function • Fixed Price Options – Takes the guesswork over variable (compute) costs when creating new applications • Inconsistent Usage - Do not pay for hosting a server if your application sees inconsistent usage Scaling is automated • Peak Demand - Handling peak demand periods or irregular “spikes” of activity • Guaranteed Resources - Serverless platform guarantees Concurrency, CPU and Memory capacity
  12. 12. What can Serverless be used for? . Any Event, Anytime, Anywhere Image credit: “Person Search Universe Silhouette” by Joy Hug Star 12
  13. 13. ServerlessUseCases 13 Scheduled, Periodic Tasks | “Cron Jobs” Configurable Alarm Feed to fire a Trigger for a Function at a specified frequency Alarm Triggered Functions can process waiting “Data-at-Rest” • Periodic Intervals - every X secs/mins • Batch process orders every 24 hours at 12 PM • Time Windows - Start / Stop by Date-Time • Accept lunch orders Mon-Fri at 11AM Stop accepting at 2PM • Specific date/time - “fire once”, or recurring • Launch a new website on Jan. 1st 2020 Serverless Platform FxT Alarm “Feed” Examples
  14. 14. ServerlessUseCases Serverless Platform Mapping of REST API Endpoints to your Serverless Functions 14 API Management for Serverless MicroServices Effectively limitless, “free” public or private APIs without any hosting costs T Fx createOrder Fx getOrder Fx deleteOrder Define API Endpoints (URLs) and map to Actions • POST:…/order • GET:…/order • DELETE:…/order 2 Create Serverless Functions 1 POST GET DELETE
  15. 15. ServerlessUseCases Serverless Platform 15 Extract, Transform and Load (ETL) Pipelines Arrival of Raw Data in datastore or message queue Triggers ETL pipeline Fx Analyze, Annotate / Enhance Data-at-Rest Enhanced Data DB Transform Raw Data Data-in-Motion Queue T Improved Data Integration for downstream consumers Streaming Data • Live Log data • IoT sensor data • Financial (market) data • In-Stream Analytics • e.g., TensorFlow TensorFlow
  16. 16. ServerlessUseCases 16 Operations Research / Combinatorial Optimization Any kind of Embarrassingly Parallel task is well-suited for Serverless functions • Map-Reduce Operations • Monte-Carlo Simulations • Genetic Algorithms • Hyperparameter tuning • Web scraping • Genome processing “Divide and Conquer” large data sets using large #s of concurrent Functions Sometimes used as a “Cheaper” option to Serverless Providers competing to give each Function large amounts of Concurrent Executions, Memory and Execution Time T Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx Fx
  17. 17. Spotting a Good Serverless platform As a top 10 list… 17
  18. 18. SpottingaGoodServerlessPlatform 18 1. Based on a Single, Open Source Codebase Managed by a robust Community with good governance practices Java Community is well versed in Licensing “nuances”; even more diligence needed for Serverless open source • Clear and Transparent licensing • Track record of addressing Bug fixes and Security patches • Straightforward process to Participate and Contribute as a Developer Trusted to permit choice of private, public, hybrid Cloud deployments Governance Maintained Secure Transparency Open Source Serverless Platform 1011 0010 0010 1110 1001 Community
  19. 19. SpottingaGoodServerlessPlatform 19 2. Maintains a consistent set of versioned REST APIs Developed against an established Programming Model • Includes versioning of Functions and their Configuration metadata • Assured compatibility & connectivity to Event Sources and Triggers • Management APIs friendly to Java packaging artifacts (e.g., JARs) Avoid Serverless Platforms that suffer API “churn” Developers Users Apps Open Source Serverless Platform Management APIs • Functions, Triggers, Event Source Config Invocation APIs • Public, Private endpoints • Access Control • Function Parameters CLI GUI For both User Invocation and Developer Management APIs
  20. 20. SpottingaGoodServerlessPlatform 20 3. Supports deploying Functions as Containers • Containers are the apparent “path to Cloud” if you want to scale • MicroServices with REST have been the “push” for the past ~8 years • Java community has fully embraced • supporting Inversion-of-Control (IoC) and Dependency Injection (DI) patterns • Providers need migration path to support Java MicroService Frameworks Serverless using Containers helps preserve Java’s investment in MicroService Frameworks Serverless Platform Management Plane • Functions, Triggers, … • API data Developers ManagementAPIs Artifact Store Built-in Language Runtimes ... Migrating to Serverless Containers via the “Lift and Shift” paradigm
  21. 21. SpottingaGoodServerlessPlatform 21 5. Supports “Source-to-Container” solutions for deployment • Serverless “platform-aware” Tooling “bakes” Functions into a Runtime Container 1. Starts with compatible “Base” Java Runtime image 2. ”Pulls” & Builds Application Source — Places JAR in a “known” location 3. Stores final Image to chosen target Image Repository Tooling and “final” image often dedicated to single target platform Using a general Containerization approach for Serverless “Lift and Shift” GitHubDocker Hub Serverless Container Builder Image Repo. 1 2 3
  22. 22. SpottingaGoodServerlessPlatform 22 4. Supports Java Functions* as 1st Class Citizens Developers package and manage their functions as Executable JARs • Tooling is Maven aware • Enable “Thin” JARs with choice of library-rich, server-side Java Runtimes Using well-maintained, Versioned Java Runtimes used for Dev/Test/ Prod • built from versioned (Open) JVMs, JDKs • With robust sets of Common libraries (e.g., JSON, Apache Commons, etc.) * Functions > Containers for Cold Start and Scaling performance Serverless Platform Management Plane • Functions, Triggers, … • API data Developers ManagementAPIs Built-in Language Runtimes Artifact Store ... Functions managed as executable JARs files which scale more efficiently than Containers
  23. 23. SpottingaGoodServerlessPlatform Serverless Platform 23 6. Scales Horizontally, On-Demand from 0..N instances Meaning Intelligent Scheduling to use “Pre-Warmed” and “Warm” Containers • Stem Cells: Term for Containers ready and waiting for Language Runtimes (DNA) to be loaded into them • Pre-Warm: Runtimes ready for Function to be “loaded” into • Warm: Requests for Same Function routed to already ”loaded” Runtime Bonus points for platforms that can scale cross Cloud / Container Technologies Then Scales to Zero when no more requests are detected (after some time) Stem Cells Pre-Warmed Containers Warm Containers Fx Fx Fx Fx Scheduler Requests Clusters of Container Pools ? ? ? ? Implementations often rely on technologies like Kubernetes and Knative that can manage and scale Container Pods within Clusters
  24. 24. SpottingaGoodServerlessPlatform Serverless Platform 24 7. Supports API Gateway-“like” Functionality • Map Functions (internal names) – to API endpoints • Easy socialization (sharing) to consumers • Security – API keys, OAuth validation • Create/Update using OpenAPI or “Swagger” standard • CORS headers – “dynamic” content for web apps. • Rate-Limiting – prevents backend services overload • Analytics – API usage, response time • Test APIs (changes) – privately before making public API Management is the #1 use case for Serverless for managing Serverless Functions as public or private endpoints
  25. 25. SpottingaGoodServerlessPlatform 25 8. Supports “out-of-box” connectivity to Event Sources No-Ops Event Sources with simple configurations • Data Stores - SQL, NoSQL, S3, Real-time, Memory … • Message Queues - Kafka, RabbitMQ, MQTT … • Social / Team - Slack, Twitter, Mobile Push … • DevOps - GitHub, Jenkins … Extensible for your own Event Sources Secure, “On Premise” data connectivity too your functions • e.g., via RPC or Message Queues DevOps burden should NOT be shifted to managing Event Sources “Hooking up” events to your function should be very easy Serverless Platform Integrated Event Sources Slack S3 S3 Object Storage GitHub Mobile Push Alarms Twitter RabbitMQ
  26. 26. SpottingaGoodServerlessPlatform 26 9. Integrated Logging and Metrics Independent of Java Frameworks or Plug-ins (zero Function overhead or config) Developers do NOT need to write any Logging or Metric Code, It’s all done for you! Integrated Logging • Real-time log event data from Functions − Supports “live” debug • Retention / Backup • Logical Groupings – Region, Version, etc. Per-Function, Per-API Metrics • Usage - Function & Corresponding API Usage Metrics • Granularity - Duration, Cold Start, Histograms, etc. via Dashboards or CLIs
  27. 27. SpottingaGoodServerlessPlatform 27 10. Supports Compositions to Create Serverless Apps Better ones support Compositions with Conditional logic & implicit shared state At minimum supports simple declarative Sequences Fx Fx Fx Fx Fx Fxif then else Fx Fx with Conditional Logic fanout sequence Serverless Application RESTAPIs • SQL, NoSQL, • S3 (Objects), • Key-Value (KV) Data Stores Queues Application State Externalized
  28. 28. SpottingaGoodServerlessPlatform 28 endeavors to provide these features… Graduated as a Top-Level Apache Project July 2019 Language Runtimes “Pluggable” Provider Platform Services Storage Log Analysis Logging Compliance Monitoring RESTAPI Identity,AccessMgmt. IAM IAM Built-In Event Integrations Slack WebUI Log Search S3 Cloud Object Storage InvokersInvokersInvokersController API GW / Router Message Bus Artifact Stores OpenWhisk Platform Components Service Provider Interfaces (SPIs) REST Recently added a developer Implementation that runs as an Executable JAR file
  29. 29. Advancing Java Serverless “in the Open” Serverless is the ultimate Microservice framework For Inversion of Control and a natural next step for Java 29 The Way Forward…
  30. 30. My hope is to “awaken” the Java Community to Open Serverless “The Java must flow!” in Serverless 30 Dune (1984 film) based on the 1965 Frank Herbert novel
  31. 31. TheWayForward 31 Today’s Serverless challenges for Java Script languages dominate Serverless today • Perception – Only Script languages are fast (command line interpreters) • JavaScript (NodeJS) in the general space and by Python for Analytics and AI Client-side Issues - JARs too “Fat” for FaaS • FaaS – Must package every class library your Function may need • JAR size - JARs to large to upload / scale efficiently (> allowed limits) • Tooling - not optimized for Serverless functions (Fat JARs, Test decoupling) • Microservice Frameworks – taking your existing “Framework” often not easy • Custom annotations and contexts / duplicate Serverless configs. Serverside Issues - Cold Start times • CaaS – Scaling (largish) Containers not efficient • FaaS – Generalized Java runtimes don’t have your libraries • Both – Contend with JVM + Class “loading” (all that entails) for general case
  32. 32. TheWayForward 32 Addressing the Challenges – Creating Java Serverless “Highways” • Purpose-Built Java Runtime Profiles “on top” of base Java Runtimes • Use Case-Specific - e.g., Analytic, AI, Mobile, ETL image/video transform, etc. • for Migration - with MicroService Frameworks “built in” - e.g., Eclipse MicroProfile, Spring, Quarkus • Allows us to host diverse, “Pre-Warmed” Pools of Profiled Java Runtimes So much more Open Serverless communities could do! BUT we need the Java Community’s help to get it right! • “QuickStart” Maven Archetypes for Serverless Java Profiles • Smart Compiles – Know what Dependencies are already in target Java Runtime Profiles • Java-Centric “Source-to-Image” Pipelines to Simplify CI/CD • Bringing Runtimes, Apps, Profiles and Tooling together • Annotations Mapping - (e.g., JAX-RS) to Serverless Configurations / APIs • Tooling that can parse, map and possibly strip unsupported Annotations • Object Relational Mapping (ORM) – Bring to Serverless as Functions! • Strong Typing (Function I/O) – Data Schema validation for Compositions and Event Feeds
  33. 33. TheWayForward 33 Java Serverless CI/CD Pipeline Vision Being Developed Now! Automatically building Java “Profiles” with optional Frameworks 1 PipelineResource • type: git Create-Jar- With-Maven Task JavaServerlessPipeline 3 4 5 6 7 Clones Java App Source as Resource Select “base image” VM, JDK version 1 Select Optional Framework Select optional Profile libraries finalize Serverless Function Image & Publish/Deploy Compile & Create Shared Class Cache for JVM Build-Uber-Runtime- with-Gradle Task Build-Shared- Class-Cache Task Finalize-Runtime- with-Function Task 762 2 6a 7a Build JAR using Maven 3 4 5 2a Lead by Priti Desai, a Tekton Contributor, from the IBM Open Source team
  34. 34. Tekton Java Pipeline Demo 34
  35. 35. 35
  36. 36. What’s Next? 36
  37. 37. Java is the trusted engine for all major industries 37 Cloud Providers are slowly realizing they want you on their Serverless platform energyManufacturing Insurance Communication Financial HealthCare Transportation
  38. 38. 38 energyManufacturing Insurance Communication Financial HealthCare Transportation Contribute, Lead and Claim the Serverless Throne ! The Java Community has the experience to make Serverless better
  39. 39. Thank you! 39 Slack: @mrutkowski Github: mrutkows
  40. 40. 40 Containers Approach - ”Build your own Docker image with everything” Runtime (Container image) Server Framework or HTTP Server Output Framework Context Fx “App” Libraries Environment Vars. Input CaaS Platform Language Runtime “Profile” Libraries (Provider dependent) “System” Libraries 1 2 3 4 5 6 7 Developers Create a “Full Stack” Runtime image with Function and dependencies “baked in” Handle Granular logging or monitoring6 Select base language runtime image • Compatible with target cloud’s hardware architecture • Typically language communities provide versioned Docker image to select from 1 Install ”Profile” libraries • approved by your project 2 Installing Service Framework or Http Server • to run your “function” as a service 3 Install “function” + Dependencies into Service Framework • Export route to your “function” on provider designated port 4 Configure your own Context • Fixed into Environment Variables 5 Manage Concurrency7 Serverless Platform Invokes Endpoint and Scales Container
  41. 41. 41 Functions Approach – “JAR your function with its libraries” When Function is “Triggered” the Platform Proxy • Injects “Function” as raw code or archive • Loads Function’s Deterministic Dependencies — Packaged with your Function q e.g., JAR or ZIP and/or — Declared in a language manifest q e.g., NodeJS “package.json”, Java Maven “POM” XML • Sets Context Information In Environment or in Input Arguments • Manages the Logging and Monitoring Runtime (Container image) FaaS Platform Environment Vars. Platform Proxy (Enforce Platform Conventions) Function Context “App” Libraries OutputInput Fx 1 FaaS Platform scales just Functions by injecting them into ready-made pools of compatible runtimes “Create” Function (Archive) in FaaS Platform • Select a supported Language Runtime image • Opinionated with “built-in” System & Profile Libraries • Library-based “Profiles” may be selectable • Configure FaaS-dependent Settings • e.g., Function’s Namespaced Path Name, • Limits (e.g., CPU, memory, timeouts, etc.), 1