Downloaded 269 times
![Legal Disclaimer
• © IBM Corporation 2014. All Rights Reserved.
• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained
in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are
subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing
contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and
conditions of the applicable license agreement governing the use of IBM software.
• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or
capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to
future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by
you will result in any specific sales, revenue growth or other results.
• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will
experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs
and performance characteristics may vary by customer.
• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM
Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server).
Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your
presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in
your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International
Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.
• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.
• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:
Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and
other countries.
• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:
UNIX is a registered trademark of The Open Group in the United States and other countries.
• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of
others.
• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta
Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration
purposes only.](https://image.slidesharecdn.com/1713deepdivelibertybuildpack-140503190519-phpapp01/85/A-Deep-Dive-into-the-Liberty-Buildpack-on-IBM-BlueMix-37-320.jpg)
Uploaded byRohit Kelapure
A Deep Dive into the Liberty Buildpack on IBM BlueMix
The document provides an in-depth overview of the Liberty Buildpack used within IBM Bluemix, detailing its architecture, components, and integration with Cloud Foundry. It outlines the various roles of the buildpack in managing applications, including transformation of app artifacts, service binding, and auto-configuration. Additionally, it emphasizes the performance benefits of the buildpack for deploying Java EE applications to the cloud environment.
More Related Content
Similar to A Deep Dive into the Liberty Buildpack on IBM BlueMix
A Deep Dive into the Liberty Buildpack on IBM BlueMix
- 1. © 2014 IBMCorporation 1713 Deep Dive into the Liberty Buildpack on IBM BlueMix Rohit Kelapure IBM Senior Software Engineer
- 2. Please Note IBM’s statementsregarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
- 3. Cloud Foundry (20,000ft view)
- 4. BlueMix & CloudFoundry Concepts Applications: Artifact that the end developer is building Services: Code that BlueMix hosts that offers functionality for apps ranging from utility functions to very complex business logic Buildpack: A collection of code that is responsible for transforming pushed application artifacts into a ready to run droplet, in a process referred to as ‘staging’. Admin Buildpack: A mechanism to allow a buildpack to be packaged, along with its associated binaries, to allow a cloud foundry administrator to install it via the cf command line tool. Droplet: A package containing everything that is needed in order to successfully run your application, e.g. JVM, Liberty core libraries, the application itself, short of the operating system. Warden: The mechanism used to achieve application isolation in the cloud foundry environment. 3
- 5. WebSphere Liberty ProfileBuildpack provides Java EE Web Profile runtime environment for cloud Buildpack for running applications on IBM WebSphere Application Server Liberty Profile. Open Source and Internal repositories differ only in the service plugins Lightweight composable Liberty runtime to rapidly build and deploy next generation dynamic java web applications and services. Cloud Foundry Buildpack architecture, allows WebSphere Liberty applications to easily run in the Cloud. Auto-service wiring and auto configuration generation for easy service integration with no changes to application code for IBM created services in Bluemix Industry leading small footprint, high performing IBM JDK Enhanced serviceability, debugging, monitoring, auto-scaling and troubleshooting of web applications in the cloud Ability to push server packages as well as web applications enabling a smooth transition from bare metal to the cloud Leverage the Liberty composability to dynamically provision the liberty runtime based on the pushed application Supports JavaEE6 out of the box for pushed war and ear applications Allows for customization of the Liberty and JRE runtime in the cloud 4
- 6. Components of aBuildpack • How an application is runContainers : • Additional Application transformationsFrameworks: • Select a java runtimeJREs: • Server transformations for services Service Plugins: 5
- 7. Where is theBuildpack ? Liberty buildpack is open source on github Open for Pull Requests and Enhancements https://github.com/cloudfoundry/ibm-websphere-liberty- buildpack Developer workflow documented 6
- 8. Architecture – CloudFoundry Build Pack Each container, framework, and JRE provider is declared in Build back iterates over each component in three phases • Construct and initialize each component with the build pack context • Drive detect against JREs, frameworks, and then containers – Construct list of components that provided responses from detect – Verify that only one JRE responded – Verify that only one container responded • Drive compile against JRE, frameworks, and container – Provision the necessary binaries – Put assets in appropriate locations – Generate appropriate configuration • Drive release against the JRE, frameworks, and container – The container generates the start command for the droplet
- 9. Buildpack API • Onlyone JRE can respond • Only one container can respond • Multiple frameworks can respond • Provision necessary binaries - Obtain and install Liberty runtime, JRE, additional features and client libraries • Put assets in appropriate location and generate appropriate configuration • Multiple tiers of caches in place that host the application & runtime binaries (Application, Admin, Buildpack, Repository) • Container generates the app start command for the droplet 8
- 10.
- 11. Droplet – Fundamentalunit of execution & scaling • Staging results in creation of the Droplet • The droplet contains: • Pushed application artefacts • Liberty binaries + (optional) additional installed features • Fully configured Liberty server including runtime environment variables from CF • IBM ‘headless’ JRE binary • (optional) Libraries needed for service interaction • (optional) Service client features needed for service interaction • Scripts 10
- 12. Liberty Buildpack caching Stagingemploys a network of hierarchical caches Application bits resource cache - located in the Cloud Controller blob store Application Staging Cache - located on the Cloud Controller blob store & DEA Admin Cache - located on the DEA -Contains Liberty and IBM JRE binaries, populated by the admin buildpack Buildpack Cache - located on the DEA - Contains openJDK, Spring auto configuration binaries populated by BOSH Repository - Configured HTTP File repository for binaries – location specified in ibm-websphere-liberty-buildpack / config 11
- 13. How applications arestaged 12 Buildpack
- 14. Packaging Buildpacks IBM andPivotal have introduced the concept of an admin Buildpack CF provider can install a 3rd party build pack and select its precedence or position in the list of buildpacks that can respond. The CF provider can select the default processing build pack for any of the runtime environments of their choosing. The administrator can manipulate the position of various versions of the build pack for maintenance purposes. The build pack processing order is predictable, understandable and visible to the CF consumer 13
- 15. Contents of abuldpack 14
- 16. Deploying Liberty applicationsto BlueMix Server centric paradigm • Focus on runtime infrastructure available in the cloud • server.xml provided along with the application • User understands how to configure Liberty for their application • User wants that configuration reflected in the cloud deployment Cloud centric paradigm • Focus on the application, not on the runtime infrastructure • server.xml is generated by the buildpack • User does not have to configure the Liberty server instance • User expects the environment to just do the right thing in order for their application to run
- 17. Supported Application Types cfpush - Push a single app (with or without a manifest): Standalone jar files (*.jar) Web Applications (*.war) Enterprise Applications (*.ear) Liberty profile server package (*.zip) Liberty profile server directory (dir.) https://www.ng.bluemix.net/docs/Liberty/LibertyApp.html 16
- 18. Server configuration whenpushing war/ear apps 17
- 19. Responsibility of theLiberty Buildpack The buildpack will convert VCAP_SERVICES and VCAP_APPLICATION environment variables provided by CF into configuration variables for the Liberty server. The variables end up in runtime-vars.xml, and are therefore referenceable from a pushed server.xml • For example ${vcap_app_port}: The port where the app server is listening (usually the same as ${port}) The buildpack will auto generate configuration for the bound services The buildpack will auto wire multiple resources references to the appropriate cloud service resource The buildpack will be responsible for starting and stopping the Liberty server and controlling the process environment Generate the Liberty server configuration when war/ear files are pushed Pull in additional Liberty runtime packages when the pushed applications need them 18
- 20. Service Integration withthe Liberty buildpack Bluemix IBM Created services • Configuration service • SQLDB (DB2) • Monitoring service • Auto Scaling service • Security Service • Data Cache • Session Cache • Log Analytics • Elastic MQ • Cloudant Bluemix IBM Community Services • MySQL • PostgreSQL • MongDB • Twillio 19 Information about bound services is available in the VCAP_SERVICES env var. Some services are container managed only (SessionCache) Require xml. Some services can be either container managed or application managed. (RDB) Some services contain multiple features which can be separately enabled. (LogAnalysis) Some services have local analogs (RDB, mongo) and some do not (LogAnalysis) Services may require client driver jars, extension features (wxs esa), Liberty features, bootstrapping.properties.
- 21. Service Plugins Service pluginscan be written to handle automatic configuration of a bound service during application staging. This is also known as auto-configuration. • One Service Plug-in for each service type (mongo, DataCache, SQLDB, etc.,) Each service Plug-in implements the plugin API. A service plug-in requires : • Service.yml -Configuration for the service • Service.rb - Impl. class for the service ServiceManager.rb in the liberty buildpack orchestrates the creation of service plugin instances • Parse VCAP_SERVICES to determine bound services • Map each bound services to a plug-in instance type. • Create instances and drive life-cycle The same set of service plug-ins can be used to support service binding either at application deployment time, or at post deployment time (late binding). • This is due to re-stage being required for any service binding update A default service plugin generates the runtime variables that are included in the server.xml for every bound service 20
- 22. Auto configuration ofLiberty server When an application is staged, the buildpack service plugin examines the bound service information (VCAP_SERVICES) & generates the corresponding server.xml snippets for the bound services 21 <server> <featureManager> <feature>webProfile-6.0</feature> </featureManager> … </server> <server> <featureManager> <feature>webProfile-6.0</feature> <feature>eXtremeScale.webapp- 1.1</feature> </featureManager> … <xsWebApp objectGridName=“${cloud…}“ catalogHostPort=“${cloud…}" securityEnabled=“${cloud…}" /> </server> Before After
- 23. Auto Configuration &Auto-Wiring of services Option to opt-out of auto- configuration when the generated configuration is not correct or if the service plugin cannot discern intent correctly Additional Liberty features are provisioned from the Liberty runtime-extended package if needed by the service In depth treatment of services and how to write a service provider covered in session In addition to auto- configuration Liberty exposes auto-wiring of data sources at runtime • • • 22
- 24. Constraints on applicationsrunning on Liberty in BlueMix Special BlueMix packaging of WebSphere Liberty profile server for the cloud • A subset of liberty features will “not” work in the cloud Special packaging of the IBM JRE for BlueMix • Removal of APIs not applicable in a cloud server environment Applications should not write files to the local file system • Local file system storage is ephemeral • Instances of the same application do not share a local file system HTTP and HTTPS Port Limitations • Only one inbound HTTP port into the Warden container All Persistent state needs to be stored in services • HTTP Sessions Not Persisted or Replicated SSL is terminated at the load balancer • will NOT work in web.xml Two phase commit transactions will not work 23
- 25. WebSphere Application MigrationToolkit Rules for assist migration of Liberty apps to the cloud Avoid writing to the local file system Transport security is terminated at the router Client certificate authentication is not available Headless JRE vs JDK on BlueMix Advise on porting persistence data to services • HTTPSessions, WXS, JMS, legacy data, MongoDB, DB2 etc., Advise on Transaction Recovery and flagging for 2 phase commits 24
- 26. Zero application downtimefor java apps in BlueMix 25 URL: https://myapp.bluemix.net/ App Version N URL: https://myapp.bluemix.net/ App Version N+1 URL: https://myapp.bluemix.net/ App Version N App Version N URL: https://myapp.bluemix.net/ App Version N App Version N 1. App Version N is running. 2. Deploy Version N+1 using another name, but bind to the same URL. App Version N 3. Decrease instance number of Version N, while increasing the instance number of Version N+1. Can be done “gracefully” without stopping existing request processing in old instances. 4. Stop Version N; May delete it later and optionally rename Version N+1 to the old app name.
- 27.
- 28. Support Statement forJava apps deployed to BlueMIX We will keep two Liberty buildpacks in bluemix • Latest and one back Buildpacks can be explicitly targeted with the –b option Default Liberty Buildpack will always be the latest valid service level Cumulative service will only be provided on the latest Buildpack level We will retire any level of the buildpack as we see fit Provide scripts & documentation to perform the “blue-green” application update at eGA IBM reserves the right to restart and in very rare situations restage applications due to operational processes or maintenance requirements 27
- 29. Buildpack Performance Memory 512m CFv 166 Ubuntu 64 bit 10.04 OpenStack Grizzly Individual VM details • 7 GB RAM • 4 Virtual CPUs • 60 GB disk space PushTime Comparison -Tomcat and Liberty 64.2 80.7 64.4 80.3 0 10 20 30 40 50 60 70 80 90 Tradelite Application Total Push Time (seconds) Acme Air Application Total Push Time (seconds Timeinseconds M10 Liberty Tomcat Average Memory Comparison - Tomcat and Liberty 128 223 166 323 0 50 100 150 200 250 300 350 Tradelite Average Memory WithoutCache (MB) Acme AirAverage Memory WithoutCache (MB) MemoryinMB M10 Liberty Tomcat
- 30. Debugging and Troubleshootingapps pushed with Liberty buildpack cf push with the latest production client explicitly specifying location with the –p If you see timeouts during push increase environment variable values • 5 Max wait time for buildpack staging, in minutes • Max wait time for app instance startup, in inutes Check to see if you are inadvertently pushing with a manifest file Liberty logs can be viewed and downloaded • Logs and Files can also be looked at from ACE • Liberty logs: messages.log, stderr.log, stdout.log, FFDCs • env.log: environment variables of the application process • staging_task.log: staging logs from the app staging process Set CF_TRACE=true and then use application guid to drive cf curl commands Bind to the Configuration service to dynamically enable trace Set trace specification in server.xml and push the server package again Buildpack logging enabled if you push after setting the JBP_LOG_LEVEL environment variable in manifest.yml or using the cf set-env command – logs available in the diagnostics directory Explicitly set the liberty buildpack start command with –command Inspect the VCAP_SERVICES environment variable and connect directly to the service from the service console 29
- 31. WebSphere Liberty RuntimeSurpasses Tomcat in the Cloud Liberty reduces deployment time by deploying server packages • Server packages eliminate need for manual configuration after application has started Liberty buildpack automatically binds applications to database, elastic cache, session persistence, and configuration services • Developers don’t write binding code • Easier to move applications to the cloud, reducing time to value Users can choose what Liberty runtime features to include with their applications • Reducing storage footprint and deployment time Liberty runtime (buildpack + binaries) supports JEE Web Profile out of the box • Tomcat requires user to provide additional libraries Liberty runtime provides better serviceability • Users can dynamically enable tracing of Liberty apps • Logs contain more information and are collected into a single directory for easier access Liberty buildpack supports pushing of ear and jar files allowing existing applications to be ported to the cloud 30
- 32. Liberty Buildpack ValueStatement Designed for extensibility instead of forking Supports push of server packages and ear files Integrated with BlueMix service eco-system Service Provider and Framework model allows for new services to be plugged in easily Ability to right size the server and the droplet based on the features required by the application in the cloud Increased serviceability through the BlueMix Runtime via ACE and services like the configuration, monitoring & loganalysis Performs better than the open source java buildpack Application can leverage JavaEE6 features which are tested and certified rather than include libraries in the app Service configuration generation and Java EE resource auto- wiring 31
- 33.
- 34.
- 35. We Value YourFeedback Don’t forget to submit your Impact session and speaker feedback! Your feedback is very important to us – we use it to continually improve the conference. Use the Conference Mobile App or the online Agenda Builder to quickly submit your survey • Navigate to “Surveys” to see a view of surveys for sessions you’ve attended 34
- 36.
- 37. Legal Disclaimer • ©IBM Corporation 2014. All Rights Reserved. • The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. • References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. • If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete: Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. • If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete: All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. • Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. • If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete: Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. • If you reference Java™ in the text, please mark the first use and include the following; otherwise delete: Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. • If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete: Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. • If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete: Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. • If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete: UNIX is a registered trademark of The Open Group in the United States and other countries. • If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete: Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. • If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.