Production Plone on OpenStack Cloud

1. Copyright © 2016 Mirantis, Inc. All rights reserved software.mirantis.com Production Plone on OpenStack Cloud Learn how to automate scalable multi-node deployments of Plone for OpenStack Cloud and see how easy it is to have this deployment tweaked to your taste

2. Copyright © 2016 Mirantis, Inc. All rights reserved About me ● 4 Years in OpenStack Community ● Now Development Manager for Murano ● Started as regular developer Sergey Melikyan smelikyan@mirantis.com @sergmelikyan fb.me/sergmelikyan #running #sightseeing #hiking #motorcycles #rock

3. Copyright © 2016 Mirantis, Inc. All rights reserved Production OpenStack Plone Deployments How to deploy Plone on OpenStack ● Introduction to the Cloud and OpenStack ● Basics of automation using built-in OpenStack tools ● How to automate scalable multi-node deployments ● Extend a cloud-ready Plone for your use-cases ● Everything else you want to learn today and I can help

5. Copyright © 2016 Mirantis, Inc. All rights reserved Definition: Cloud As described by the National Institute of Standards and Technology (NIST): “Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”

6. Copyright © 2016 Mirantis, Inc. All rights reserved Essential Characteristics of the Cloud* ● Provisioning of computing capabilities (compute, network, storage) without human interaction On-demand Self-service ● Services are available over the network and accessed through standard mechanisms Broad Network Access ● Capabilities can be elastically provisioned and released to scale rapidly Rapid Elasticity ● Computing resources are pooled to serve multiple consumers using a multi-tenant model ● Customer generally has no knowledge over the exact location of resources Resource Pooling ● Resource usage can be monitored, controlled, and reported Measured Service *According to NIST (National Institute of Standards and Technology)

7. Copyright © 2016 Mirantis, Inc. All rights reserved SPI Model ● Software as a Service (SaaS) ● Provides consumers with an application running on a cloud infrastructure ● Platform as a Service (PaaS) ● Provides consumers with the ability to deploy an application to the cloud utilizing tools and platform supported by the provider ● Infrastructure as a Service (IaaS) ● Provides consumers with the ability to provision fundamental computing resources and deploy arbitrary software ConsumerFlexibility LevelofAbstraction Data Center Hardware, servers, networking Provision CPU, RAM, VM INFRASTRUCTURE AS A SERVICE (IAAS) Remote login to install applications PLATFORM AS A SERVICE (PAAS) Browser or thin client access SOFTWARE AS A SERVICE (SAAS)

8. Copyright © 2016 Mirantis, Inc. All rights reserved Cloud Deployment Models* *According to NIST (National Institute of Standards and Technology) ● Cloud services are only available to members of a single organization. ● Cloud infrastructure can be hosted by the same organization or a third-party. Private Cloud ● Cloud services are offered to members of a community or organization. ● Cloud infrastructure is hosted by members of the same organization. Community Cloud ● Cloud services are offered to the general public. ● Cloud infrastructure is hosted by anyone. Public Cloud ● Composition of two or more distinct cloud infrastructure, such as a Private+Public cloud combination. Hybrid Cloud

12. Copyright © 2016 Mirantis, Inc. All rights reserved Definition: OpenStack As described by the OpenStack Foundation: “Aims to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.”

13. Copyright © 2016 Mirantis, Inc. All rights reserved History Founded by Rackspace and NASA in 2010 Mission: to enable any organization regardless of size to create and offer cloud computing services running on standardized hardware

15. Copyright © 2016 Mirantis, Inc. All rights reserved Openstack Foundation Created in 2012 to provide shared resources to help achieve the OpenStack Mission by Protecting, Empowering, and Promoting OpenStack software and the community around it, including users, developers and the entire ecosystem.

23. Copyright © 2016 Mirantis, Inc. All rights reserved Not going to happen... What I am not going to do today: ● Teach you how to develop software on python ● Explain what is version control systems, specifically git ● Teach you how to use linux or virtualbox … but what I should do if I don’t know how to … ?

26. Copyright © 2016 Mirantis, Inc. All rights reserved ● OS: Ubuntu or CentOS ● OpenStack: DevStack ● Dependencies: ● git, python, pip, tox, virtualenv ● OpenStack System Dependencies ● IDE or text editor Tools

27. Copyright © 2016 Mirantis, Inc. All rights reserved Installing DevStack ● Requirements ● 4Gb RAM ● 2 CPU ● 32 Gb Storage ● Recommendation ● Use VirtualBox https://etherpad.openstack.org/p/murano-development-environment

28. Copyright © 2016 Mirantis, Inc. All rights reserved OpenStack System Dependencies Ubuntu ● python-dev ● libxml2-dev ● libxslt-dev ● libffi-dev ● libpq-dev ● python-openssl ● mysql-client CentOS ● python-devel ● libxml2-devel ● libxslt-devel ● libffi-devel ● postgresql-devel ● pyOpenSSL ● mysql

29. Copyright © 2016 Mirantis, Inc. All rights reserved Tools: pip pip is a package management system used to install and manage software packages written in Python. Many packages can be found in the Python Package Index (PyPI). $ curl -O https://bootstrap.pypa.io/get-pip.py $ sudo python -W ignore get-pip.py --force-reinstall $ rm get-pip.py http://pypi.python.org

30. Copyright © 2016 Mirantis, Inc. All rights reserved Tools: virtualenv virtualenv solves a very specific problem: it allows multiple Python projects that have different (and often conflicting) requirements, to coexist on the same computer. ● Installed as dependency for tox ● Managed through tox https://virtualenv.pypa.io/

31. Copyright © 2016 Mirantis, Inc. All rights reserved Tools: tox tox is a generic virtualenv management and test command line tool you can use for: checking your package installs correctly with different Python versions and interpreters or running your tests in each of the environments, configuring your test tool of choice. $ sudo pip install tox https://tox.readthedocs.org/

34. Copyright © 2016 Mirantis, Inc. All rights reserved Requirements ● OS: Ubuntu (or anything else...) ● Recommended Flavor ● 2Gb RAM ● 40 Gb Storage ● Software Prerequisites: ● Python 2.7 (dev), built with support for expat (xml.parsers.expat), zlib and ssl. ● libz (dev), libjpeg (dev), readline (dev), libexpat (dev), libssl or openssl (dev), libxml2 >= 2.7.8 (dev), libxslt >= 1.1.26 (dev).

35. Copyright © 2016 Mirantis, Inc. All rights reserved Unified Plone UNIX Installer ● The Universal UNIX Installer lets you choose an installation mode: ● a standalone mode - single Zope web application server will be installed. ● or a ZEO cluster mode - ZEO Server and Zope instances will be installed. http://docs.plone.org/manage/installing/installation.html

37. Copyright © 2016 Mirantis, Inc. All rights reserved Heat - OpenStack Orchestration To orchestrate composite cloud applications using a declarative template format through an OpenStack-native REST API. Orchestration for Openstack

38. Copyright © 2016 Mirantis, Inc. All rights reserved Orchestration vs Configuration Management ● Orchestration is a sub-category of automation, concerned with coordination of multiple component: ● For example, servers, networks, volumes, etc.. ● Orchestration is a “higher form” of automation ● Not just simple or lower-level tasks, but multilayer applications ● Configuration Management is automation of server configuration: ● Typically a declarative model, based on “fact” discovery of the server ● Abstracts out the underlying implementation detail of service deployment ● Both are needed to fully automate cloud application deployment

39. Copyright © 2016 Mirantis, Inc. All rights reserved Heat Capabilities ● Describes the infrastructure for a cloud application – stack (deployment): ● OpenStack resources: for example, servers, networks, volumes, etc.. ● Relationships between resources: for example, this volume is connected to this server ● In a text file in the special format – template (blueprint) ● Manages that infrastructure: ● Automatically changes the infrastructure when the template is modified and re-applied ● Deletes infrastructure when the stack is deleted ● Integrates with software configuration management tools such as Puppet and Chef: ● For example: can create VM with puppet server and install puppet clients on VMs ● Can pass parameters to cloud-init, etc.

40. Copyright © 2016 Mirantis, Inc. All rights reserved Heat Orchestration Template ● Declarative definition of resources, based on YAML ● Stack = deployed template, instance of the template ● Structure ● Parameters ● Resources ● Outputs Full Specification: http://docs.openstack.org/developer/heat/template_guide/

41. Copyright © 2016 Mirantis, Inc. All rights reserved Example heat_template_version: 2013-05-23 description: Simple template to deploy a single compute instance resources: my_instance: type: OS::Nova::Server properties: key_name: my_key image: cirros-0.3.0-i386-uec flavor: m1.small

42. Copyright © 2016 Mirantis, Inc. All rights reserved CloudInit ● An open-source package from Ubuntu that is the industry standard for bootstrapping cloud servers (initialization on first boot) ● Available on various Linux distributions such as Ubuntu Cloud Images and the official Ubuntu images available on EC2. ● Some of the things it configures are: ● setting a default locale ● setting hostname ● resizing boot disk to that specified in boot flavor ● adding ssh keys to user's .ssh/authorized_keys so they can log in ● setting up ephemeral mount points

43. Copyright © 2016 Mirantis, Inc. All rights reserved CloudInit: Example resources: database: type: OS::Nova::Server properties: ... user_data: | #!/bin/bash -v yum -y install mariadb mariadb-server systemctl start mariadb.service

44. Copyright © 2016 Mirantis, Inc. All rights reserved Developing Template ● Uses “Unified Plone UNIX Installer” ● Single node deployment ● Allows to configure ● Size of the VM (RAM, Storage) ● Setup password for Plone ● Choose port on which Plone is going to be running

46. Copyright © 2016 Mirantis, Inc. All rights reserved Links ● Project Links: ● https://launchpad.net/heat ● https://wiki.openstack.org/wiki/Heat ● http://docs.openstack.org/developer/heat/ ● MailingList: ● openstack-dev@openstack.org with [Heat] in the subject

48. Copyright © 2016 Mirantis, Inc. All rights reserved Murano Introduce an application catalog to OpenStack, enabling application developers and cloud administrators to publish various cloud-ready applications in a browsable categorized catalog. Application Catalog for OpenStack

50. Copyright © 2016 Mirantis, Inc. All rights reserved Benefits Developer Tooling Application Catalog Cloud Orchestration Onboarding Developers Onboarding Workloads Self-Service Deployment IT Admins End Users Build better cloud apps faster Deploy without assistance from ITManage what end users can deploy

51. Copyright © 2016 Mirantis, Inc. All rights reserved Capabilities ● Supports both Windows and Linux ● Complete application lifecycle management ● Integration with configuration management tools ● Ansible, Chef & Puppet ● Pluggable app definition languages ● Heat Templates ● MuranoPL

53. Copyright © 2016 Mirantis, Inc. All rights reserved Past ● Windows DataCenter as a Service ● Fixed building blocks: AD, IIS, MS SQL, Exchange, Sharepoint ● Complicated software composition ● Only Windows software provisioning Bringing Windows to OpenStack

54. Copyright © 2016 Mirantis, Inc. All rights reserved LayersLayers catalog app interoperability layer orchestration configuration management glare murano-api + dashboard murano pl murano-engine murano-engine heat heat-translator third-partymurano pl murano-agent bash puppet powershellchef

55. Copyright © 2016 Mirantis, Inc. All rights reserved Apps Interoperability MySQL host: instance deploy() createDatabase() createUser() getConnectionS() .destroy() WordPress webserver: httpd db: mysql deploy() .destroy() WordPress: deploy() → … webserver.deploy() … db.createDatabase(<name>) db.createUser(<name>, <pass>) db.assignUser(<dname>, <uname>) ... ● Murano PL: imperative OOP DSL ● Sandboxed, built on top of Python ● Everything is an object ● Dependencies on interface ● Decoupling Apps form an ecosystem where complex environments can be constructed from independent building blocks

56. Copyright © 2016 Mirantis, Inc. All rights reserved YAQL YAQL: Yet Another Query Language ● Embeddable and extensible query language ● Allows performing complex queries against arbitrary objects ● Used by Murano, Mistral, Heat and Fuel pip install yaql $ yaql yaql> @load my_file.json yaql> $.customers.groupBy($.sex) yaql> $.customers.where($.orders.len() >= 1 or name = "John") Find more about YAQL at https://github.com/openstack/yaql or play at http://yaqluator.com/

59. Copyright © 2016 Mirantis, Inc. All rights reserved Hello World: Step 1 Adding manifest.yaml $ mkdir HelloWorld $ vi HelloWorld/manifest.yaml FullName: com.yourdomain.HelloWorld Type: Application Checkout sources for step #1 - git checkout step1

60. Copyright © 2016 Mirantis, Inc. All rights reserved Hello World: Step 2 Adding HelloWorld.yaml $ mkdir Classes $ vi Classes/HelloWorld.yaml Name: com.yourdomain.HelloWorld Extends: io.murano.Application Methods: deploy: Body: - $reporter: $this.find('io.murano.Environment').reporter - $reporter.report($this, "Hello, World!") Checkout sources for step #2 - git checkout step2

61. Copyright © 2016 Mirantis, Inc. All rights reserved Hello World: Step 2 Updating manifest.yaml $ vi HelloWorld/manifest.yaml ... Classes: com.yourdomain.HelloWorld: HelloWorld.yaml Checkout sources for step #2 - git checkout step2

63. Copyright © 2016 Mirantis, Inc. All rights reserved HelloWorld: Deploy Deploy $ vi input.js [{ "op": "add", "path": "/-", "value": { "?": { "name": "Demo", "type": "com.yourdomain.HelloWorld", "id": "42" } } }] http://q.melikyan.me/input-json

64. Copyright © 2016 Mirantis, Inc. All rights reserved HelloWorld: Deploy Deploy $ murano environment-create TestHello $ murano environment-session-create <environment-id> $ murano environment-apps-edit --session-id <session-id> <env-id> ./input.json $ murano environment-deploy --session-id <session-id> <env-id> $ murano environment-show <env-id>

65. Copyright © 2016 Mirantis, Inc. All rights reserved Hello World: Step 3 Adding User Input $ vi Classes/HelloWorld.yaml Properties: username: Usage: In Contract: $.string().notNull() Methods: deploy: Body: - $reporter: $this.find('io.murano.Environment').reporter - $reporter.report($this, "Hello, " + $this.username + "!") Checkout sources for step #3 - git checkout step3

67. Copyright © 2016 Mirantis, Inc. All rights reserved HelloWorld: Deploy again Deploy with an updated input $ vi input.js [{ "op": "add", "path": "/-", "value": { "?": { "name": "Demo", "type": "com.yourdomain.HelloWorld", "id": "42" }, "username": "Alice" } }] http://q.melikyan.me/input-json-updated

68. Copyright © 2016 Mirantis, Inc. All rights reserved HelloWorld: Deploy again Deploy $ murano environment-create TestHello2 $ murano environment-session-create <environment-id> $ murano environment-apps-edit --session-id <session-id> <env-id> ./input.json $ murano environment-deploy --session-id <session-id> <env-id> $ murano environment-show <env-id>

69. Copyright © 2016 Mirantis, Inc. All rights reserved Hello World: Step 4 Adding UI wizard $ mkdir UI $ vi UI/UI.yaml Application: ?: type: com.yourdomain.HelloWorld username: Alice Checkout sources for step #4 - git checkout step4

70. Copyright © 2016 Mirantis, Inc. All rights reserved Hello World: Step 4 (continued) Application: ?: type: com.yourdomain.HelloWorld username: $.step1.username Forms: - step1: fields: - name: username type: string description: Username of the user to say 'hello' to required: true Checkout sources for step #4 - git checkout step4

74. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Goals ● Provision a virtual machine in OpenStack (VM); ● Configure VMs network connectivity and security; ● Download a distribution of Plone from Internet to the virtual machine; ● Install the distribution and configure some of its parameters with user input.

75. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Preparation Preparation $ mkdir PloneApp $ vi PloneApp/manifest.yaml FullName: com.yourdomain.Plone Name: Plone CMS Description: Simple Plone Deployment Type: Application Author: John Doe Checkout sources for step #5 - git checkout step5

76. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Preparation (continued) Add Plone.yaml $ mkdir PloneApp/Classes $ vi PloneApp/Classes/Plone.yaml Namespaces: =: com.yourdomain std: io.murano apps: io.murano.applications Name: Plone Extends: apps:SingleServerApplication Checkout sources for step #6 - git checkout step6

77. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Preparation (ends) Checkout sources for step #6 - git checkout step6 Updating manifest.yaml $ vi PloneApp/manifest.yaml ... Classes: com.yourdomain.Plone: Plone.yaml Require: io.murano.applications:

78. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Properties Adding properties $ vi PloneApp/Classes/Plone.yaml Properties: deploymentPath: Contract: $.string() Default: '/opt/plone' adminPassword: Contract: $.string().notNull() listeningPort: Contract: $.int() Default: 8080

79. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Properties Adding UI.yaml server: ?: type: io.murano.resources.LinuxMuranoInstance name: $.instanceConfiguration.hostname image: $.instanceConfiguration.image flavor: $.instanceConfiguration.flavor assignFloatingIp: $.instanceConfiguration.assignFloatingIp Checkout sources for step #7 - git checkout step7

80. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Configuration Management ● Use “Unified Plone UNIX Installer” ● Use almost same script as we did for Heat template http://q.melikyan.me/install-plone-sh

81. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Configuration Checkout sources for step #8 - git checkout step8 Add install-plone.sh $ mkdir PloneApp/Resources <download and put install-plone.sh in the directory above>

82. Copyright © 2016 Mirantis, Inc. All rights reserved Plone: Deploy Checkout sources for step #9 - git checkout step9 Adding logic $ vi PloneApp/Classes/Plone.yaml ... Methods: onInstallServer: ... Body: - $file: sys:Resources.string('install-plone.sh').replace({"$1" => $this.deploymentPath, "$2" => $this.adminPassword, "$3" => $this.listeningPort}) - conf:Linux.runCommand($server.agent, $file)

83. Copyright © 2016 Mirantis, Inc. All rights reserved Bonus: Logo & Security Rules ● Please checkout step10 and take a look ● Try to analyze what we are doing and why ● Explain Checkout sources for step #10 - git checkout step10

86. Copyright © 2016 Mirantis, Inc. All rights reserved Screencasts ● Openstack Application Catalog. Quick introduction ● HA & Autoscaling for Applications ● Docker Integration ● Support for Heat based application ● Composing Murano Application Package Please, find more screencasts published on wiki: https://wiki.openstack.org/wiki/Murano/Screencasts

87. Copyright © 2016 Mirantis, Inc. All rights reserved Links ● Project Links: ● https://launchpad.net/murano ● https://wiki.openstack.org/wiki/Murano ● http://docs.openstack.org/developer/murano/ ● MailingList: ● openstack-dev@openstack.org with [Murano] in the subject Weekly IRC meeting is held on Tuesdays at 17:00 UTC on #openstack-meeting-alt at Freenode.

89. Copyright © 2016 Mirantis, Inc. All rights reserved Credits ● Alexander Tivelkov & Evgeniy Mashkin ● for amazing articles and developing Plone app for Murano ● David Flanders & Philip Bauer ● for opportunity being here and present this training

Production Plone on OpenStack Cloud

Serg Melikyan

Production Plone on OpenStack Cloud