This document proposes using RPM packages to deploy Java applications to Red Hat Linux systems in a more automated and standardized way. Currently, deployment is a manual multi-step process that is slow, error-prone, and requires detailed application knowledge. The proposal suggests using Maven and Jenkins to build Java applications into RPM packages. These packages can then be installed, upgraded, and rolled back easily using common Linux tools like YUM. This approach simplifies deployment, improves speed, enables easy auditing of versions, and allows for faster rollbacks compared to the current process.
IT Infrastructure Through The Public Network Challenges And SolutionsMartin Jackson
Identifying the challenges that companies face when they wish to adopt Infrastructure as a Service like those from Amazon and Rackspace and possible solutions to those problems. This presentation seeks to provide insight and possible solutions, covering the areas of security, availability, cloud standards, interoperability, vendor lock in and performance management.
How to create a secured cloudera clusterTiago Simões
This presentation, it’s for everyone that is curious with Big Data and does have the know how to start learning...
With this, you will be able to create quickly a Kerberos secured Cloudera Cluster.
How to configure a hive high availability connection with zeppelinTiago Simões
With this presentation, you not only should be able to configure a Hive Interpreter on Zeppelin but also with a High Availability, Load balancing and Concurrency architecture.
It will be created a JDBC connection with kerberos authentication that will communicate with your Zookeeper on the cluster.
IT Infrastructure Through The Public Network Challenges And SolutionsMartin Jackson
Identifying the challenges that companies face when they wish to adopt Infrastructure as a Service like those from Amazon and Rackspace and possible solutions to those problems. This presentation seeks to provide insight and possible solutions, covering the areas of security, availability, cloud standards, interoperability, vendor lock in and performance management.
How to create a secured cloudera clusterTiago Simões
This presentation, it’s for everyone that is curious with Big Data and does have the know how to start learning...
With this, you will be able to create quickly a Kerberos secured Cloudera Cluster.
How to configure a hive high availability connection with zeppelinTiago Simões
With this presentation, you not only should be able to configure a Hive Interpreter on Zeppelin but also with a High Availability, Load balancing and Concurrency architecture.
It will be created a JDBC connection with kerberos authentication that will communicate with your Zookeeper on the cluster.
This session will quickly show you how to describe the security configuration of your Kafka cluster in an AsyncAPI document. And if you've been given an AsyncAPI document, this session will show you how to use that to configure a Kafka client or application to connect to the cluster, using the details in the AsyncAPI spec.
Puppet is a configuration management tool which allows easy deployment and configuration ranging from 1 to 1 thousand servers (and even more). Even though its common knowledge for devops, puppet is still a strange piece of software for developers. How does it work and what can it do for you as a developer?
Running High Performance and Fault Tolerant Elasticsearch Clusters on DockerSematext Group, Inc.
Sematext engineer Rafal Kuc (@kucrafal) walks through the details of running high-performance, fault tolerant Elasticsearch clusters on Docker. Topics include: Containers vs. Virtual Machines, running the official Elasticsearch container, container constraints, good network practices, dealing with storage, data-only Docker volumes, scaling, time-based data, multiple tiers and tenants, indexing with and without routing, querying with and without routing, routing vs. no routing, and monitoring. Talk was delivered at DevOps Days Warsaw 2015.
kubernetes install and practice
* Environment (bare metal installation, not using cloud service)
- VM 1 : Mater node, 30GB, 2 vCPU, 4GB Mem
- VM 2 : Worker node, 30GB, 2 vCPU, 4GB Mem
* Practice
- deploying pod, make a deployment and service
- expose service using ingress(nginx-ingress)
PuppetCamp SEA 1 - Puppet Deployment at OnAppWalter Heck
Wai Keen Woon, CTO CDN Division OnApp Malaysia, gave an interesting overview of what the Puppet architecture at OnApp looks like. The CDN division at OnApp is a large provider of CDN services, and as such makes a very interesting candidate for a case study.
Walter Heck, founder of OlinData, presented a step-by-step guide on how to set up a proper puppet repository, complete with the brand new PuppetDB, exported resources and usage of open source modules.
EFK Stack이란 ElasticSearch, Fluentd, Kibana라는 오픈소스의 조합으로, 방대한 양의 데이터를 신속하고 실시간으로 수집/저장/분석/시각화 할 수 있는 솔루션입니다. 특히 컨테이너 환경에서 로그 수집을 위해 주로 사용되는 기술 스택입니다.
Elasitc Stack에 대한 소개와 EFK Stack 설치 방법에 대해 설명합니다.
Ansible is an open source automation platform, written in Python, that can be used for configuration-management, application deployment, cloud provisioning, ad-hoc task-execution, multinode orchestration and so on. This talk is an introduction to Ansible for beginners, including tips like how to use containers to mimic multiple machines while iteratively automating some tasks or testing.
->Introduction
->>What is Ansible?
->>Ansible history
->Basic concepts
->>Inventory
->>Playbook
->>Role
->>Module
->>Plugin
->Diving into Ansible roles
->>Getting started
->>Create a role
->>Roles under the hood
->>How to use roles?
This session will quickly show you how to describe the security configuration of your Kafka cluster in an AsyncAPI document. And if you've been given an AsyncAPI document, this session will show you how to use that to configure a Kafka client or application to connect to the cluster, using the details in the AsyncAPI spec.
Puppet is a configuration management tool which allows easy deployment and configuration ranging from 1 to 1 thousand servers (and even more). Even though its common knowledge for devops, puppet is still a strange piece of software for developers. How does it work and what can it do for you as a developer?
Running High Performance and Fault Tolerant Elasticsearch Clusters on DockerSematext Group, Inc.
Sematext engineer Rafal Kuc (@kucrafal) walks through the details of running high-performance, fault tolerant Elasticsearch clusters on Docker. Topics include: Containers vs. Virtual Machines, running the official Elasticsearch container, container constraints, good network practices, dealing with storage, data-only Docker volumes, scaling, time-based data, multiple tiers and tenants, indexing with and without routing, querying with and without routing, routing vs. no routing, and monitoring. Talk was delivered at DevOps Days Warsaw 2015.
kubernetes install and practice
* Environment (bare metal installation, not using cloud service)
- VM 1 : Mater node, 30GB, 2 vCPU, 4GB Mem
- VM 2 : Worker node, 30GB, 2 vCPU, 4GB Mem
* Practice
- deploying pod, make a deployment and service
- expose service using ingress(nginx-ingress)
PuppetCamp SEA 1 - Puppet Deployment at OnAppWalter Heck
Wai Keen Woon, CTO CDN Division OnApp Malaysia, gave an interesting overview of what the Puppet architecture at OnApp looks like. The CDN division at OnApp is a large provider of CDN services, and as such makes a very interesting candidate for a case study.
Walter Heck, founder of OlinData, presented a step-by-step guide on how to set up a proper puppet repository, complete with the brand new PuppetDB, exported resources and usage of open source modules.
EFK Stack이란 ElasticSearch, Fluentd, Kibana라는 오픈소스의 조합으로, 방대한 양의 데이터를 신속하고 실시간으로 수집/저장/분석/시각화 할 수 있는 솔루션입니다. 특히 컨테이너 환경에서 로그 수집을 위해 주로 사용되는 기술 스택입니다.
Elasitc Stack에 대한 소개와 EFK Stack 설치 방법에 대해 설명합니다.
Ansible is an open source automation platform, written in Python, that can be used for configuration-management, application deployment, cloud provisioning, ad-hoc task-execution, multinode orchestration and so on. This talk is an introduction to Ansible for beginners, including tips like how to use containers to mimic multiple machines while iteratively automating some tasks or testing.
->Introduction
->>What is Ansible?
->>Ansible history
->Basic concepts
->>Inventory
->>Playbook
->>Role
->>Module
->>Plugin
->Diving into Ansible roles
->>Getting started
->>Create a role
->>Roles under the hood
->>How to use roles?
Minimum Viable Docker: our journey towards orchestrationOutlyer
While Kubernetes and Mesos are all the rage, you don't necessarily need a complex orchestration layer to start using and benefiting from Docker. We will present how Babylon Health is running its dockerised AI microservices in production, pros and cons, and what we have in store for the future.
PuppetConf 2017: Use Puppet to Tame the Dockerfile Monster- Bryan Belanger, A...Puppet
You want to create an application? Great. Download a Docker image and install all your stuff. Sounds like a lot of work, huh? Wait, you also need to be able to patch your container too? That Dockerfile will become a Frankenfile! Well, guess what: Puppet has a an answer for you. Using Docker, Puppet and Jenkins we will show you how you can: 1. Put all your code in an easy to use project. 2. Give yourself a powerful toolkit for configuration 3. Automate your builds 4. Allow your project to automate security updates / patches
Code testing and Continuous Integration are just the first step in a source code to production process. Combined with infrastructure-as-code tools such as Puppet the whole process can be automated, and tested!
Instrumentación de entrega continua con GitlabSoftware Guru
Mostraremos el caso real de cómo tenemos implementado en nuestra empresa el flujo de desarrollo para integración y entrega continua, instrumentado con GitLab.
Sesión presentada por David Padilla en SG Next 2017
distribute und pip als Ersatz für setuptools und easy_install bieten im Zusammenspiel mit virtualenv viele neue Möglichkeiten bei der Entwicklung und dem Deployment von Python-Applikationen. In diesem Vortrag stelle ich alle Werkzeuge kurz vor und zeige, wie man sie zusammen einsetzen kann.
This talk will give you tips and tricks to get better build time performance and smaller images. The most important take-away is: you should be using multi-stage Dockerfiles and enable BuildKit.
1. Automated RPM based Java Artefact Deployments
With Puppet for Red Hat Based Linux Systems
2. Code that is written and not deployed
is money wasted.”
Jesse Robbins, Opscode
3. My Current Problem
Tomcat Application Deployment where:
• Artefacts are supplied as a drop in WAR/JAR file
built by Maven
• Environment specific configuration items are not
externalized to the WAR/JAR e.g. hardcoded
hostnames and databases in hibernate.xml
• Environment variables are not being used to define
different environments e.g. java –Denv=dev
• Due to Rapid development there are many releases
and files types and their uses are in flux e.g. changing
formats in hibernate.xml
4. Anti-patterns
The way we currently deploy our Java
applications is an anti-pattern
(ConfigurationBirdsNest*)
Example deploy of mytomcatapp application:
$ service tomcat6 stop
$ rm /var/lib/tomcat6/webapps/mytomcatapp.war
$ rm –rf /var/lib/tomcat6/webapps/mytomcatapp
$ rm –rf /var/cache/tomcat6/work/Catalina/localhost/mytomcatapp
$ service tomcat6 start
$ wget –O /var/lib/tomcat6/webapps/mytomcatapp.war
http://path/to/mytomcatapp.war
$ sleep 30
$ service tomcat6 stop
$ vi /var/lib/tomcat6/webapps/mytomcatapp/META-INF/context.xml
$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-
INF/classes/application.properties
$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-
INF/classes/hibernate.cfg.xml
$ vi /etc/tomcat6/Catalina/localhost/mytomcatapp.xml
$ service tomcat6 start
*http://code.google.com/p/devops-toolchain/wiki/ConfigurationBirdNest
5. Why are these anti-patterns?
It’s slow
It’s risky (typing errors anyone?)
It requires detailed knowledge of the application
Configuration files and their purpose are in flux
and require constant updating of puppet (or insert
configuration management tool of choice here)
code
7. A problem shared is a problem halved…
Communicating to our Developers why this causes us pain
Why we should either:
• Externalize our configuration with overrides
• Or use environment variables to separate them
• Or tokenize them
Why War files don’t fully meet our needs in Ops
• Version querying
• Audibility
• Speed
• Fast rollback
8. Design Pattern
RPM Packaged Artifact*
based delivery for Java
*http://code.google.com/p/devops-toolchain/wiki/PackagedArtifact
9. Proposal
Using RPM-Maven-Plugin*
• Automated Red Hat Package creation of Java applications and
deploy them to your software artefact repository
Leveraging Jenkins** (or insert CI tool of choice here) to
automatically build them as artefacts alongside the war/jar’s
Using Jenkins ssh plugin*** to copy rpm to cobbler repo
and perform a “cobbler reposync”
*http://mojo.codehaus.org/rpm-maven-plugin/
**http://jenkins-ci.org/
*** https://wiki.jenkins-ci.org/display/JENKINS/SSH+plugin &
https://wiki.jenkins-ci.org/display/JENKINS/Publish+Over+SSH+Plugin
10. Why?
Just by adding something like this to your maven
pom.xml build files:
<properties>
<app.home>/var/lib/tomcat6/webapps/mytomcatapp</app.home>
</properties>
...
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>rpm-maven-plugin</artifactId>
<version>2.0.1</version>
<executions>
<execution>
<goals>
<goal>rpm</goal>
</goals>
</execution>
</executions>
<configuration>
<copyright>2011, Uncommon Sense Consulting</copyright>
<group>Development</group>
<description>Maven Recipe: RPM Package.</description>
<mappings>
<mapping>
<directory>${app.home}</directory>
<sources>
<source>
<location>target/mytomcatapp</location>
</source>
</mapping>
</mappings>
</configuration>
</plugin>
11. Why?
We can get a Red Hat Package
mytomcatapp-$version.rpm – contains
/var/lib/tomcat6/webapps/mytomcatapp
/var/lib/tomcat6/webapps/mytomcatapp/META-INF
/var/lib/tomcat6/webapps/mytomcatapp/META-INF/maven
/var/lib/tomcat6/webapps/mytomcatapp/WEB-INF
/var/lib/tomcat6/webapps/mytomcatapp/WEB-INF/classes
/var/lib/tomcat6/webapps/mytomcatapp/WEB-INF/lib
12. Why?
Need to install a fully functioning mytomcatapp in
Development?
$ yum install mytomcatapp
Dependencies Resolved
================================================================================
Package Arch Version
================================================================================
Installing:
mytomcatapp noarch 1.2.3-1
Installing for dependencies:
oracle-instantclient11.2-jdbc x86_64 11.2.0.2.0-1
oracle-instantclient11.2-basic x86_64 11.2.0.2.0-1
tomcat6 x86_64 6.0.26-1
tomcat6-servlet x86_64 6.0.26-1
tomcat6-jsp-2.1-api x86_64 6.0.26-1
tomcat6-el x86_64 6.0.26-1
tomcat6-lib x86_64 6.0.26-1
Transaction Summary
================================================================================
Install 8 Package(s)
Upgrade 0 Package(s)
Total download size: 58 M
Is this ok [y/N]:
13. Puppet
Current Puppet Module
class mytomcatapp::install {
$packagelist = ["oracle-instantclient11.2-jdbc"]
package{ $packagelist: ensure => installed }
file{"/var/lib/tomcat6/webapps/mytomcatapp.war":
source => ["puppet:///modules/mytomcatapp/var/lib/tomcat6/webapps/${fqdn}.mytomcatapp.war",
"puppet:///modules/mytomcatapp/var/lib/tomcat6/webapps/${custom_env}.mytomcatapp.war",
"puppet:///modules/mytomcatapp/var/lib/tomcat6/webapps/default.mytomcatapp.war"],
owner => "tomcat",
group => "tomcat",
notify => Class["tomcat6::service"],
mode => 644,
}
}
...100 lines later...
class mytomcatapp::portal {
include mytomcatapp::common
include mytomcatapp::install
include mytomcatapp::config
include mytomcatapp::backup
include mytomcatapp::monitor
}
14. Impact on Puppet
New Puppet Module
class mytomcatapp::install {
$packagelist = [$mytomcatapp::common::package]
package{ $packagelist: ensure => installed }
}
class mytomcatapp::monitor {
#noop
}
class mytomcatapp::portal {
include mytomcatapp::common
include mytomcatapp::common::environment
include mytomcatapp::install
include mytomcatapp::monitor
}
15. Value Proposition
Simplicity in application deployment – lower overhead,
puppet friendly runtime changes
Massive speed improvement in time to deploy and upgrades
• Bare metal deploy estimate is 5 minutes down from 8 minutes
• Application upgrade is 60 seconds down from 60 minutes
Auditability – upgrades are logged and version’s easily and
programmatically checked:
$ rpm –qi mytomcatapp
Rollback is as simple as:
$ rpm –e mytomcatapp
$ yum install mytomcatapp-$previousversion
16. Simplicity
Pop Quiz! What’s easier?
This?
$ service tomcat6 stop
$ rm /var/lib/tomcat6/webapps/mytomcatapp.war
$ rm –rf /var/lib/tomcat6/webapps/mytomcatapp
$ rm –rf /var/cache/tomcat6/work/Catalina/localhost/mytomcatapp
$ service tomcat6 start
$ wget –O /var/lib/tomcat6/webapps/mytomcatapp.war
http://path/to/mytomcatapp.war
$ sleep 30
$ service tomcat6 stop
$ vi /var/lib/tomcat6/webapps/mytomcatapp/META-INF/context.xml
$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-
INF/classes/application.properties
$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-
INF/classes/hibernate.cfg.xml
$ vi /etc/tomcat6/Catalina/localhost/mytomcatapp.xml
$ service tomcat6 start
Or this? Bonus points if you
guess what’s faster!
$ yum upgrade mytomcatapp
17. Auditability
Question – What version of mytomcatapp is in
X environment?
[actionjack@rasengan]$ rpm -qi mytomcatapp
Name : mytomcatapp Relocations: (not relocatable)
Version : 1.2.4 Vendor: Uncommon Sense
Release : 1 Build Date: Fri 07 Oct 2011 06:21:10 PM BST
Install Date: Fri 07 Oct 2011 06:31:10 PM Build Host: rasengan.uncommonsense.local
Group : Development Source RPM: mytomcatapp-1.2.4-1.src.rpm
Size : 17922480 License: 2011, Uncommon Sense
Signature : (none)
Packager : Martin Jackson <martin@uncommonsense-uk.com>
Summary : Unnamed - com.uncommonsense-uk:mytomcatapp:war:1.2.4
Description :
My Tomcat App.
Question – Has anybody changed anything?
[actionjack@rasengan]$ rpm --verify mytomcatapp
[actionjack@rasengan]$ echo I guess not...
18. Rolling Back
Rapid recovery needed?
[actionjack@rasengan]$ sudo yum upgrade mytomcatapp
[actionjack@rasengan]$ sudo tail /var/log/tomcat6/catalina.out
mytomcatappv1.3.1-alpha starting...
Err something isn’t quite right..
I feel sick...
I’m gonna puke..
[actionjack@rasengan]$ sudo rpm –e mytomcatapp
[actionjack@rasengan]$ sudo yum install mytomcatapp-1.2.4
[actionjack@rasengan]$ sudo /etc/init.d/tomcat6 restart
[actionjack@rasengan]$ sudo tail /var/log/tomcat6/catalina.out
mytomcatappv1.2.4-alpha starting...
Top of the morning to you!
Ready for business.
[actionjack@rasengan]$