JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave)

•

1 like•519 views

By Denis Chernilevskiy, Yandex Attendees will learn about the concept of using several nodes as a single Jenkins slave. At Yandex, we’re deploying software environments consisting of many nodes from the cloud and we can leverage existing Jenkins slave management mechanisms - such as scheduling, automated provisioning and load-balancing - via this single-slave abstraction. This greatly decreases the time needed to support lots of separate slave nodes. At Yandex, we have created a plugin for these purposes and it will be presented in this session.

Technology

Multi-node environment as
a Jenkins slave
aka “compound-slave”
Denis Chernilevskiy
Head of Media Advertising Department

The image cannot be displayed. Your computer may not have
enough memory to open the image, or the image may have
been corrupted. Restart your computer, and then open the file
again. If the red x still appears, you may have to delete the
image and then insert it again.
Common use-case
the “prehistory”
Multi-node environment as a Jenkins slave

Common use-case
4
He’s famous The “slave” The “job”

The quest
6
Nodes Cloud Deploy Get resultsTest

The solution
▌  Cloud – Amazon/OpenStack/Docker/…
▌  Deploy – JClouds/DockerCloud + Chef/Salt/Ansible/…
▌  Test – PyTest/TestNG/…
▌  Results – Allure Framework/xUnit/…
7

How it works
▌  The common job concept
›  Take 1 slave
›  Deploy the system under test to the slave
›  Deploy and run tests on the same slave
›  Clean the slave
›  Return the slave to the pool
8

Process requirements
▌  Auto build, deploy, test
▌  Continuous Integration
▌  “Green trunk”
15

System components
▌  Front-end UI and API
▌  Ad.Management
▌  Ad.Selection
▌  Ad.Storage
▌  SocDem prediction & machine learning
16

The obvious “not a solution” J
▌  Use Jenkins
▌  Get a slave for the job
▌  Get some more cloud nodes (somehow)
▌  Use the slave as a launchpad (run deploy, tests remotely)
18

Why not?
▌  Laziness. The less new code, the better!
▌  Separate nodes management
▌  JClouds not reused
▌  Lack of visibility
20

Features
▌  Group several nodes as a compound-slave
›  Existing nodes grouping
›  Get & group from the cloud
25

Features
▌  Cloud provisioning
›  Plugins with a “Cloud” interface
›  Using labels from the cloud plugin
›  Number of each label to provision
26

Features
▌  Roles and labelling
›  Distinct nodes
›  Default labels
27

Features
▌  General
›  Built-in slave management
›  Built-in job-slave provisioning
▌  Visibility
›  Built-in dashboards
›  Built-in job steps configuration
29

Links
▌  wiki.jenkins-ci.org/display/JENKINS/Compound+Slaves
▌  github.com/jenkinsci/compound-slaves-plugin
31

Contacts
dchr@yandex-team.ru
+79250351311 @dchernilevskiy
denis.chernilevskiy
Denis Chernilevskiy
Head of Yandex MediaAd Department

This document summarizes how Twitter used DTrace to investigate performance issues. DTrace helped Twitter identify that 40% of CPU time was spent in backtrace() calls in Ruby code, due to unnecessary exception handling. Removing these checks led to a 30% drop in CPU usage. DTrace also found that rewriting a CRC32 computation from Ruby to C could yield a further 15% CPU reduction. In general, DTrace allows identifying performance bottlenecks and improving application efficiency.

Aegir Cpanel for Drupal Sites

Damjan Cvetan

Cpanel for Drupal sites discusses the benefits of using Ægir, an open source hosting system, for managing multiple Drupal sites. Ægir sits alongside a LAMP server and allows users to create, deploy, and manage Drupal sites through its entities including servers, platforms, and sites. Key features of Ægir include automating file permission management, database provisioning, backups and recovery for Drupal multisite installations.

Docker in production: problems, fixes and beyond

Miguel A. Guillén Tarazaga

This document discusses some problems encountered when using Docker in production and provides recommendations to address them. It recommends: (1) avoiding links and instead using service discovery like Consul; (2) avoiding port redirection due to potential memory leaks and compatibility issues, and using host networking instead; (3) not using the default storage driver and instead creating a partition with a thin pool to address storage issues; (4) avoiding large images to reduce deployment time and disk usage; and (5) not using Docker for databases or storage services due to data persistence challenges. The document aims to help users address common problems when adopting Docker in a production environment.

Commit to excellence - Java in containers

Red Hat Developers

Java in 2019 was predicted to be business as usual by many. We have seen new Java releases coming out as planned, AdoptOpenJDK became the main trustful source of binaries and Oracle fighting for the trademark again by preventing the use of javax as namespace. Everything looks like it would be a silent year for Java. But one thing seems obvious. Java's popularity is not gaining any more traction. New language features keep it up to date but people are getting more selective when it comes to implementation choices. Especially in the age of containers and cloud infrastructures. This talk walks you through the why and how of Java in containers. We will talk about image size and development and deployment processes.

Os Django

oscon2007

This document provides an outline for a Django tutorial given at OSCON in 2007. The tutorial covers various topics related to Django including unit testing, middleware, signals, forms and AJAX, template tags, custom fields, OpenID integration, scaling techniques, and geospatial data (GIS). For each section, the document provides an overview and examples. It emphasizes the importance of unit testing and provides testing code examples.

Run alone: a standalone application attempt by Gabriel Sor

FAST

Олександр Лінивий — Multisite platform with continuous delivery process for m...

LEDC 2016

Headless Android allows running the full Android system software stack without a graphical user interface. It is achieved by disabling SurfaceFlinger and the WindowManager system services. This removes the display and window management functionality while retaining all other Android frameworks and APIs. The result is a headless version of Android that can be used for building embedded and specialized devices without screen output.

Novices guide to docker

Alec Clews

This document provides an introduction to Docker, including basic Docker concepts and terms. It discusses how Docker solves issues like scalability, resilience and portability by allowing processes to be isolated in lightweight containers. Key terms are explained, such as images, which contain the files and metadata to start a container, and containers, which are running instances of images. The document demonstrates Docker functionality by running example Alpine Linux containers and modifying and removing containers. It also discusses persisting container data using volumes and mounts.

DCSF19 Docker Containers & Java: What I Wish I Had Been Told

Docker, Inc.

Mohammed Aboullaite, xHub Nowadays, containers are more than a cool new tool to play with - they've revolutionized the way we develop, architect and ship software and have become part of our day-to-day operations. On the other hand, Java has been around for more than 2 decades, dominating the entreprise world. Both advocate the "RUN anywhere" principal, but, is it that easy? No! Your perfect working local container will most probably fail on production because of MEMORY &/or CPU issues, since jvm apps assume they OWN the server they are running on. This session will look at the gotchas of running JVM apps in containers & how to avoid costly pitfalls. We will cover: _ what updates Java 10 brought to the table to improve container awareness and the issues related to prior versions, including how to address each one of them to avoid the mess. - tips and tricks to obtain smaller images & best practices while writing your DockerFiles - some plugins to helps non-containers experts integrate docker in their development process; and - the concept of "Distroless" Docker Images and how to benefit from it.

Drupal Presentation for CapitalCamp 2011: Features Driven Development

Mediacurrent

This document provides an overview of feature driven development in Drupal. It discusses how features allow encapsulation of items like views, content types, and configurations to make deployment and version control easier. It provides instructions for creating basic features and exporting non-exportable items. Feature driven development is presented as creating a feature module for each content type along with related components. Examples of real world feature lists are shown to demonstrate how features are used to package common site elements.

Hadoop: Big Data Stacks validation w/ iTest How to tame the elephant?

Dmitri Shiryaev

Java and Container - Make it Awesome !

Dinakar Guniguntala

More and more companies are adopting some form of cloud for hosting their applications. Cloud also increasingly means Docker containers and Kubernetes to orchestrate them. Cloud is a different beast when it comes to running Java applications. It focuses on fast provisioning, ephemeral containers that startup and scale quickly and use minimum resources when idle. Correspondingly the JVM has undergone a number of changes that help it to adapt to cloud environments. This talk looks closely at each of the cloud specific aspects and explores the changes in Java that a developer needs to be aware of to make the most of running a Java app in the cloud and provides a github based tutorial for reference.

Running JVM in Docker

Uri Savelchev

This document discusses running Java applications in Docker containers. It begins with a minimal example of dockerizing a Java HTTP server application and shows that the default Docker image is quite large at 726 MB. Alternative base images and techniques are explored to significantly reduce the image size, such as using Alpine Linux, Java 9+ modules, and building a native Java runtime. Memory limits are also discussed, noting that the default Docker memory settings are not optimal and cgroups values or Java ergonomics should be used instead. With these techniques, the example application's Docker image size was reduced to as small as 35 MB.

Django Deployment with Fabric

Jonas Nockert

This document discusses using Fabric, a Python library, to help with deploying Django applications to different environments. It provides an overview of Fabric's core functionality like running commands remotely, copying files, and asking users for input. It also discusses using Fabric to implement common deployment tasks like installing dependencies, applying database migrations, and reloading application servers. The document suggests Fabric can help standardize and simplify deployments but notes deployment processes can be complex and require additional considerations around testing, database handling, and managing a live site during upgrades.

Eclipse RCP outside of Eclipse IDE - Gradle to the rescue!

Michał Ćmil

Wuff is a Gradle plugin that allows building Eclipse RCP applications outside of the Eclipse IDE using Gradle. It addresses issues with the standard Eclipse build system like unstable builds, difficulty sharing code between client and server modules, and tight coupling with the IDE. Wuff treats Eclipse plugins as normal Gradle dependencies, generates manifests, and bundles non-OSGi dependencies to create executable Eclipse applications with a Maven-like structure and dependency management. This provides a more stable, IDE-independent build system for Eclipse RCP projects.

DockerCon EU 2015: Trading Bitcoin with Docker

Docker, Inc.

Presented by Sebastien Goasguen, VP, Apache CloudStack and Mathieu Buffenoir, co-founder, SBEX Bity is an internet money gateway built by Swiss Bitcoin Exchange ( SBEX ). To trade bitcoin the entire infrastructure of Bity is running in Docker containers. All the components of the infrastructure are using Docker, from the frontend applications and load balancer, the Django based backend, replicated Postgres database, Bitcoin daemon and remittance engine. All software goes through a CI pipeline that starts with Docker images being built on private repositories in Docker hub. Developers take also advantage of a docker-compose definition that allows them to run the entire infrastructure on a single laptop. Finally the production deployments happen thanks to the Ansible Docker module on a CloudStack based public cloud. Everything has been automated to ease re-deployment and operations. This presentation will go through every component and how Docker has enabled us to go production in 4 months.

Hacking Jenkins

Miro Cupak

This document outlines a workshop on hacking Jenkins using various techniques like the REST API, CLI, and Groovy scripting. It includes tasks to trigger builds via REST, update configurations, safely shutdown the master, create nodes via CLI, execute scripts on slaves to check resources, and add labels to nodes. It also discusses creating system Groovy jobs that run on the master to monitor slave status and send emails if no slaves are online.

November 15 cloud bees clusterhq meetup fli, flockerhub, and jenkins

Ryan Wallner

Hands on Exploration of Page Objects and Abstraction Layers with Selenium Web...

Alan Richardson

This document provides an overview of a half-day tutorial on building abstraction layers for web automation tests using Selenium WebDriver. The tutorial will cover common approaches to modeling web applications and constructing page object and other abstraction layers. Attendees will participate in exercises to experiment with different abstraction techniques using a sample todo list application. The session will discuss topics like page object design, element abstractions, domain objects, and component abstractions. Code examples demonstrating various abstraction implementations will also be presented and analyzed.

Deploy Mediawiki Using FIWARE Lab Facilities

FIWARE

Infrastructure Engineering

Bryan McLellan

The document discusses Bryan McLellan's role as a Systems Administrator and his focus on infrastructure engineering through automation of tasks using tools like Puppet, Facter, Capistrano and iClassify. It provides examples of how automation is used at Widemile to build new servers through PXE boot and configuration via Puppet as well as monitoring with Munin. The document also references Bryan's goal of leveraging his technical skills to provide scalable solutions through adaptive use of tools and frameworks.

Rick Blalock: Your Apps are Leaking - Controlling Memory Leaks

Axway Appcelerator

Developing large-scale Titanium apps can be tricky. Although Titanium manages memory on the native level for developers, care and attention on memory management and profiling techniques must be used in order to ensure their javascript is optimial. Attendees will learn how to detect memory leak / retention issues caused by client-side code. They will learn how to utilize tools and techniques such as Instruments / DDMS, proper architectural patterns, and more to produce performant apps. Appcelerator's Rick Blalock leads this session for developers who have a a strong grasp of Titanium but want to learn how to profile and improve the apps they build.

Js tacktalk team dev js testing performance

Артем Захарченко

This document provides an overview of using Gradle to automate JavaScript application assembly, minification, and deployment tasks. It discusses installing Gradle and relevant plugins, writing a Gradle script to define tasks for combining JS files, minifying with Closure Compiler, and deploying the minified file to FTP. Examples are given for setting up the Gradle script with the necessary inputs, outputs, and dependencies between tasks. The benefits of automating these repetitive tasks with Gradle are highlighted.

Deploy MediaWiki usgin Fiware Lab Facilities

José Ignacio Carretero Guarde

Building a JavaScript Library

jeresig

JUC Europe 2015: Scaling Your Jenkins Master with Docker

CloudBees

This document discusses scaling a Jenkins master using Docker. It describes how the company Wyplay was using a single Jenkins master to manage over 500 jobs, which led to performance and reliability issues. It then explains how Wyplay set up multiple Docker containers running Jenkins masters to address these problems. It provides details on installing and configuring Jenkins in Docker containers, managing plugins, backups, and monitoring across multiple containerized masters. Finally, it discusses future directions like using orchestration tools and programmatically configuring aspects of Jenkins.

JUC Europe 2015: Plugin Development with Gradle and Groovy

CloudBees

Similar to JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave)

Urs Köster - Convolutional and Recurrent Neural Networks

Intel Nervana

Headless Android

Opersys inc.

Novices guide to docker

Alec Clews

DCSF19 Docker Containers & Java: What I Wish I Had Been Told

Docker, Inc.

Drupal Presentation for CapitalCamp 2011: Features Driven Development

Mediacurrent

Hadoop: Big Data Stacks validation w/ iTest How to tame the elephant?

Dmitri Shiryaev

Java and Container - Make it Awesome !

Dinakar Guniguntala

Running JVM in Docker

Uri Savelchev

Django Deployment with Fabric

Jonas Nockert

Eclipse RCP outside of Eclipse IDE - Gradle to the rescue!

Michał Ćmil

DockerCon EU 2015: Trading Bitcoin with Docker

Docker, Inc.

Hacking Jenkins

Miro Cupak

November 15 cloud bees clusterhq meetup fli, flockerhub, and jenkins

Ryan Wallner

Hands on Exploration of Page Objects and Abstraction Layers with Selenium Web...

Alan Richardson

Deploy Mediawiki Using FIWARE Lab Facilities

FIWARE

Infrastructure Engineering

Bryan McLellan

Rick Blalock: Your Apps are Leaking - Controlling Memory Leaks

Axway Appcelerator

Js tacktalk team dev js testing performance

Артем Захарченко

Deploy MediaWiki usgin Fiware Lab Facilities

José Ignacio Carretero Guarde

Building a JavaScript Library

jeresig

Similar to JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave) (20)

Urs Köster - Convolutional and Recurrent Neural Networks

Headless Android

Novices guide to docker

DCSF19 Docker Containers & Java: What I Wish I Had Been Told

Drupal Presentation for CapitalCamp 2011: Features Driven Development

Hadoop: Big Data Stacks validation w/ iTest How to tame the elephant?

Java and Container - Make it Awesome !

Running JVM in Docker

Django Deployment with Fabric

Eclipse RCP outside of Eclipse IDE - Gradle to the rescue!

DockerCon EU 2015: Trading Bitcoin with Docker

Hacking Jenkins

November 15 cloud bees clusterhq meetup fli, flockerhub, and jenkins

Hands on Exploration of Page Objects and Abstraction Layers with Selenium Web...

Deploy Mediawiki Using FIWARE Lab Facilities

Infrastructure Engineering

Rick Blalock: Your Apps are Leaking - Controlling Memory Leaks

Js tacktalk team dev js testing performance

Deploy MediaWiki usgin Fiware Lab Facilities

Building a JavaScript Library

More from CloudBees

JUC Europe 2015: Scaling Your Jenkins Master with Docker

CloudBees

JUC Europe 2015: Plugin Development with Gradle and Groovy

CloudBees

JUC Europe 2015: The Famous Cows of Cambridge: A Non-Standard Use Case for Je...

CloudBees

By Sarah Woodall, NXP Semiconductors LPCXpresso is a multi-platform IDE for developers of embedded software to run on NXP Semiconductor's ARM-based microcontrollers. NXP needs to test that the debugger can execute programs on numerous different development boards that connect to the USB ports of host computers. Besides building a complex software product, the Jenkins installation drives an automated test farm consisting of home-built software-controlled USB switches ("cows") that control a huge array of combinations of test board, debug probe and host platform. This talk will give a tour of the NXP farm, including video of the cows in action, and will describe the features of Jenkins that are used to make it work, with particular emphasis on dynamic selection of combinations within matrix jobs, parameterized triggers and the Summary Display plugin. Finally, plans to migrate to the new Workflow plugin will be discussed. NXP believes the Workflow plugin will simplify the structure and make it more maintainable.

JUC Europe 2015: Jenkins Made Easy

CloudBees

By Nobuaki Ogawa, EFI DirectSmile In this session, you will learn how you can easily utilize continuous delivery practices with Jenkins CI. From build, deploy, test, maintenance and monitoring, lots of processes can be easily orchestrated with Jenkins. Nobuaki will cover a very simple case for which he implemented continuous delivery with Jenkins CI, Azure and Selenium. This project is a basic case of continuous delivery. Especially noteworthy, for a Windows program. :-)

JUC Europe 2015: Scaling of Jenkins Pipeline Creation and Maintenance

CloudBees

By Damien Coraboeuf, Clear2Pay In a large company where several dozens of projects and branches will need their own pipelines, you cannot afford to maintain all the jobs, manually. For security reasons and knowledge limitations, Clear2Pay does not want to open Jenkins job configurations in the centralised master. Instead, the Clear2Pay team offers project teams a "shopping list" that they can use to automatically generate their own pipelines for all branches, without requiring the Jenkins administration team to intervene. Projects just update a jenkins.properties in the SCM branch and the pipeline for this branch is updated accordingly. This allows the number of projects to scale, each getting their own pipeline in the Jenkins master without having to worry about administering hundreds of jobs.

JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...

CloudBees

By Christian Lipphardt, Camunda Services Camunda is an open source, Java-based framework process/business process automation. As a middleware technology, Camunda integrates with six different Java application servers (in different versions) and supports six different database products. The team at Camunda maintains five supported versions of Camunda itself, adding two versions every year. Maintaining the necessary continuous integration (CI) infrastructure based on virtual machines became increasingly problematic, with poor build reproducibility and limited scalability. Feedback cycles for developers were unacceptable. Recently Camunda switched from the virtual machine model to a container model based on Docker. The Camunda team now develops infrastructure as code and applies microservice-like separation of concerns. In the talk, Daniel will share the new CI architecture and present lessons learned.

JUC Europe 2015: Hey! What Did We Just Release?

CloudBees

By Robert McNulty, Experian Marketing Services Have you ever completed a build and wondered what exactly changed? Typically, output logs and parameterized build input data such as SCM branches, bug tracking issues and notes entered by the developer are lost once the build has completed. At best, Jenkins keeps this information for a limited time. The only historical reference is, perhaps, in a report. Consequently, this data cannot easily be reused for future builds or reviewed during the auditing process. Using Jenkins, Groovy and Neo4j, this data can persist for the life of a project. This presentation will describe the simple steps taken to save this information for posterity.

JUC Europe 2015: Orchestrating Your Pipelines with Jenkins, Python and the Je...

CloudBees

By Pradeepto K. Bhattacharya, MSys-Clogeny Do you have complex pipeline scripts that you run for your continuous integration (CI)? Do you want to access Jenkins from those scripts? What about searching and accessing the artifacts? Or would you like to block a job until something has finished correctly? Interact with slaves? Would you like to have some kind of synchronization between your jobs and pipeline scripts and yet keep the your job configurations as simple as possible? This talk is about you getting more programmatic control of your Jenkins instance from your pipeline scripts using the Python API. We will look at the power of the API by showing working code examples and demoing the results. This talk will be very friendly to Jenkins beginners and intermediates alike. We will walk through the concepts and actual code during the presentation.

JUC Europe 2015: Jenkins Pipeline for Continuous Delivery of Big Data Projects

CloudBees

By: Luca Milanesio, GerritForge Big data is now everywhere, from mobile media analytics, banking, industry, avionics and even in medicine to monitor expansion of epidemics. In this session, Luca will show how continuous integration and continuous delivery is applied to a big data scenario that poses new challenges to the existing Jenkins framework. He will present the implementation of an agile build and deployment process used in big data software development projects for media and financial organizations in London. The talk will start with a presentation of the workflow and then will explain how existing Jenkins plugins were leveraged, as well as how integration with Docker, Mesos and the Hadoop ecosystem was achieved.

JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...

CloudBees

By Oleg Nenashev, CloudBees, Inc. This talk will address Jenkins-based continuous integration (CI) in the area of embedded systems, which include both hardware and software components. An overview of common automation cases, challenges and their solutions based on Jenkins CI services will be presented. The specifics of Jenkins usage in the hardware area (available plugins and workarounds, environment and desired high availability features) will also be discussed. The session will cover several automation examples and case studies.

JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkin...

CloudBees

By Jonathann Zenou, Ravello Systems Ravello Systems has relied heavily on Jenkins since the early days. During the company's first four years, their Jenkins setup changed and evolved to the point that it was out of control. Ravello decided to take all the experience from those years and create a new and improved Jenkins setup. This session will share the lessons that were learned the hard way - and explain how the CI process was optimized. The process of re-doing the CI mechanism for a large always developing group will be discussed, as well as mechanisms for revision control in Jenkins, job infrastructure and architecture guidelines for maximal flexibility and reuse and various other considerations. The overall continuous integration and testing strategies - which are completely cloud-based and cover all our varied components (from a hypervisor through networking and storage layers, distributed backend systems and all the way to an HTML5-based UI and a cross platform client side application) - will be described.

JUC Europe 2015: Enabling Continuous Delivery for Major Retailers

CloudBees

By Masood Jan, Mazataz Masood will illustrate the achievements and challenges faced whilst implementing a continuous delivery (CD) framework for a major retailer by using a rigorous but simple development process, integrated with Jenkins build pipelines. The pipelines have been carefully architected to orchestrate the various build, deployment, testing and release stages of e-commerce applications. The presenter will conclude with future goals regarding a cloud-based CD process using Jenkins.

JUC Europe 2015: Bringing CD at Cloud-Scale with Jenkins, Docker and "Tiger"

CloudBees

JUC Europe 2015: Making Strides towards Enterprise-Scale DevOps...with Jenkin...

CloudBees

By Ilkka Turunen, Sonatype Moving to DevOps in large complex enterprise-IT environments is an incremental process. One that requires culture, process and technology. Technology like Jenkins, Nexus, Puppet, Docker and more. In this session, you’ll hear about first-hand experiences building successful enterprise-scale DevOps practices and specifically look at the role of Jenkins working with other key technologies in the continuous tool chain. Learn about additional practices to support the goal of driving down cycle times. And no DevOps practice is complete without accounting for compliance and security requirements Jenkins can play a key role there too. Learn more.

JUC Europe 2015: Evolving the Jenkins UI

CloudBees

Gus Reiber and Tom Fennelly presented on evolving the Jenkins user interface. They discussed the history of the Jenkins GUI from its origins in 2005 to recent updates. They highlighted issues with CSS, page structure, and JavaScript modularization. Their solutions included using LESS for modular CSS theming, supporting GUI themes, and implementing JavaScript modularization best practices to improve the user experience. They demonstrated a prototype Jenkins interface and discussed continuing to enhance the visual design and functionality.

JUC Europe 2015: Using Infrastructure Nodes Wisely With Jenkins and Apache Mesos

CloudBees

By Pradeepto K. Bhattacharya, MSys-Clogeny So it is release crunch time and your developers are hacking away. Your Jenkins CI build queue is getting longer because of lack of free slaves. What do you do? Allocate more dedicated hardware? OK, but what happens when the crunch time is over? Those slave instances will then be sitting idle. Not good! Do you wish that nodes in your infrastructure could be better utilized? Is your infrastructure a victim of "static partitioning"? How awesome would it be if there was a way to scale to multiple Jenkins slaves automatically, whenever needed - and then also scale back down when the work is done? This talk is about solving these problems. Pradeepto will cover all the concepts necessary to understand Apache Mesos, its architecture, what it does and how it does it. He will then demo Jenkins running over Apache Mesos and demonstrate the ability to scale up and down as needed. Code will be shared for better understanding of the solution and the concepts behind it.

JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...

CloudBees

By Viktor Clerc, XebiaLabs If you are taking the quality of your software seriously, you'll have numerous automated tests across many different Jenkins jobs. But getting a grip on all of your automated tests -- and then figuring out whether your software is good enough to go live -- becomes harder and harder as you speed up software delivery. Viktor will share tips on how naming conventions, partitioning of testware and mirroring the application's structure in the test code help you best handle automated testing with Jenkins. Viktor will also provide insight into how to keep this setup manageable and will share practical experiences of managing a large portfolio of automated tests. Finally, he will showcase several practices that help you manage all your results, plus add aggregation, trend analysis and qualification capabilities to your Jenkins setup. These practices will help you draw the right conclusions from your tests and deliver code faster, with the confidence that your systems won't fail in production.

JUC 2015 - Keynote Address and Opening Remarks by Kohsuke Kawaguchi, Founder,...

CloudBees

This document outlines Kohsuke Kawaguchi's Jenkins World Tour 2015 presentation. Some key points from the presentation include statistics showing growth in Jenkins usage and installations, new features like Jenkins Workflow and Continuous Delivery Pipelines to improve automation and integration, and initiatives to promote Jenkins adoption and certification. The presentation also discusses how Jenkins is expanding its role in areas like containerization with Docker and promoting the concept of infrastructure and deployments as code.

JUC Europe 2015: Continuous Integration and Distribution in the Cloud with DE...

CloudBees

By Mark Galpin, JFrog Correct this if it's wrong, but as a software developer you have two main dreams - to enjoy your coding and to not have to care about anything else but code. Setting up an environment and maintaining a CI/CD cycle for your software can be complicated and painful. The good news is, it doesn't have to be! In this talk, Mark will demo some of the most popular alternatives for a cloud-based development life cycle: from CI builds with DEV@cloud, through artifact deployment to a binary repository and finally, rolling out your release on a truly modern distribution platform.

JUC Europe 2015: Configuration as Code: The Job DSL Plugin

CloudBees

More from CloudBees (20)

JUC Europe 2015: Scaling Your Jenkins Master with Docker

JUC Europe 2015: Plugin Development with Gradle and Groovy

JUC Europe 2015: The Famous Cows of Cambridge: A Non-Standard Use Case for Je...

JUC Europe 2015: Jenkins Made Easy

JUC Europe 2015: Scaling of Jenkins Pipeline Creation and Maintenance

JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...

JUC Europe 2015: Hey! What Did We Just Release?

JUC Europe 2015: Orchestrating Your Pipelines with Jenkins, Python and the Je...

JUC Europe 2015: Jenkins Pipeline for Continuous Delivery of Big Data Projects

JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...

JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkin...

JUC Europe 2015: Enabling Continuous Delivery for Major Retailers

JUC Europe 2015: Bringing CD at Cloud-Scale with Jenkins, Docker and "Tiger"

JUC Europe 2015: Making Strides towards Enterprise-Scale DevOps...with Jenkin...

JUC Europe 2015: Evolving the Jenkins UI

JUC Europe 2015: Using Infrastructure Nodes Wisely With Jenkins and Apache Mesos

JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...

JUC 2015 - Keynote Address and Opening Remarks by Kohsuke Kawaguchi, Founder,...

JUC Europe 2015: Continuous Integration and Distribution in the Cloud with DE...

JUC Europe 2015: Configuration as Code: The Job DSL Plugin

Recently uploaded

June Patch Tuesday

Ivanti

Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.

AWS Cloud Cost Optimization Presentation.pptx

HarisZaheer8

This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.

Choosing The Best AWS Service For Your Website + API.pptx

Brandon Minnick, MBA

Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API? Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose? Which one is cheapest? Which one is fastest? Which one will scale to meet our needs? Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!

5th LF Energy Power Grid Model Meet-up Slides

DanBrown980551

5th Power Grid Model Meet-up It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology. Power Grid Model The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services. Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability. Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization. What to expect For the upcoming meetup we are organizing, we have an exciting lineup of activities planned: -Insightful presentations covering two practical applications of the Power Grid Model. -An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024. -An interactive brainstorming session to discuss and propose new feature requests. -An opportunity to connect with fellow Power Grid Model enthusiasts and users.

Taking AI to the Next Level in Manufacturing.pdf

ssuserfac0301

Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as: 1. How quickly AI is being implemented in manufacturing. 2. Which barriers stand in the way of AI adoption. 3. How data quality and governance form the backbone of AI. 4. Organizational processes and structures that may inhibit effective AI adoption. 6. Ideas and approaches to help build your organization's AI strategy.

TrustArc Webinar - 2024 Global Privacy Survey

TrustArc

How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024? In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores. See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe. This webinar will review: - The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey - The top challenges for privacy leaders, practitioners, and organizations in 2024 - Key themes to consider in developing and maintaining your privacy program

JavaLand 2024: Application Development Green Masterplan

Miro Wengner

HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU

panagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/ DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen! Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell. Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten. Diese Themen werden behandelt - Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten - Wie funktionieren CCB- und CCX-Lizenzen wirklich? - Verstehen des DLAU-Tools und wie man es am besten nutzt - Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw. - Praxisbeispiele und Best Practices zum sofortigen Umsetzen

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on integration of Salesforce with Bonterra Impact Management. Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...

Tatiana Kojar

Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI. With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.

Main news related to the CCS TSI 2023 (2023/1695)

Jakub Marek

An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers. The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 . The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .

FREE A4 Cyber Security Awareness Posters-Social Engineering part 3

Data Hops

Your One-Stop Shop for Python Success: Top 10 US Python Development Providers

akankshawande

Energy Efficient Video Encoding for Cloud and Edge Computing Instances

Alpen-Adria-Universität

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack

shyamraj55

Best 20 SEO Techniques To Improve Website Visibility In SERP

Pixlogix Infotech

Trusted Execution Environment for Decentralized Process Mining

LucaBarbaro3

How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf

Chart Kalyan

dbms calicut university B. sc Cs 4th sem.pdf

Shinana2

GNSS spoofing via SDR (Criptored Talks 2024)

Javier Junquera

In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security. This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing. The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.

Recently uploaded (20)

June Patch Tuesday

AWS Cloud Cost Optimization Presentation.pptx

Choosing The Best AWS Service For Your Website + API.pptx

5th LF Energy Power Grid Model Meet-up Slides

Taking AI to the Next Level in Manufacturing.pdf

TrustArc Webinar - 2024 Global Privacy Survey

JavaLand 2024: Application Development Green Masterplan

HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...

Main news related to the CCS TSI 2023 (2023/1695)

FREE A4 Cyber Security Awareness Posters-Social Engineering part 3

Your One-Stop Shop for Python Success: Top 10 US Python Development Providers

Energy Efficient Video Encoding for Cloud and Edge Computing Instances

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack

Best 20 SEO Techniques To Improve Website Visibility In SERP

Trusted Execution Environment for Decentralized Process Mining

How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf

dbms calicut university B. sc Cs 4th sem.pdf

GNSS spoofing via SDR (Criptored Talks 2024)

JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave)

1. Multi-node environment as a Jenkins slave aka “compound-slave” Denis Chernilevskiy Head of Media Advertising Department

2. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. Common use-case the “prehistory” Multi-node environment as a Jenkins slave

3. Common use-case 4 He’s famous The “slave” The “job”

4. │ Test this please!

5. The quest 6 Nodes Cloud Deploy Get resultsTest

6. The solution ▌  Cloud – Amazon/OpenStack/Docker/… ▌  Deploy – JClouds/DockerCloud + Chef/Salt/Ansible/… ▌  Test – PyTest/TestNG/… ▌  Results – Allure Framework/xUnit/… 7

7. How it works ▌  The common job concept ›  Take 1 slave ›  Deploy the system under test to the slave ›  Deploy and run tests on the same slave ›  Clean the slave ›  Return the slave to the pool 8

8. │ Perfectly fits with 1 slave

9. │ But what if…

10. 11

11. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. The “real life” of QA/DevOps the problem Multi-node environment as a Jenkins slave

12. │ Yandex MediaAd platform

13. │ Test this please!

14. Process requirements ▌  Auto build, deploy, test ▌  Continuous Integration ▌  “Green trunk” 15

15. System components ▌  Front-end UI and API ▌  Ad.Management ▌  Ad.Selection ▌  Ad.Storage ▌  SocDem prediction & machine learning 16

16. │ None fits into 1 slave…

17. The obvious “not a solution” J ▌  Use Jenkins ▌  Get a slave for the job ▌  Get some more cloud nodes (somehow) ▌  Use the slave as a launchpad (run deploy, tests remotely) 18

18. │ Not for us J

19. Why not? ▌  Laziness. The less new code, the better! ▌  Separate nodes management ▌  JClouds not reused ▌  Lack of visibility 20

20. │ Several slaves per job?

21. │ Need to refactor core…

22. │ Several nodes as a slave!

23. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. “Compound-slaves” plugin the solution Multi-node environment as a Jenkins slave

24. Features ▌  Group several nodes as a compound-slave ›  Existing nodes grouping ›  Get & group from the cloud 25

25. Features ▌  Cloud provisioning ›  Plugins with a “Cloud” interface ›  Using labels from the cloud plugin ›  Number of each label to provision 26

26. Features ▌  Roles and labelling ›  Distinct nodes ›  Default labels 27

27. The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again. Here comes the DEMO! J

28. Features ▌  General ›  Built-in slave management ›  Built-in job-slave provisioning ▌  Visibility ›  Built-in dashboards ›  Built-in job steps configuration 29

29. │ Profit! J

30. Links ▌  wiki.jenkins-ci.org/display/JENKINS/Compound+Slaves ▌  github.com/jenkinsci/compound-slaves-plugin 31

31. Contacts dchr@yandex-team.ru +79250351311 @dchernilevskiy denis.chernilevskiy Denis Chernilevskiy Head of Yandex MediaAd Department

JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave)

Recommended

Recommended

More Related Content

Similar to JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave)

Similar to JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave) (20)

More from CloudBees

More from CloudBees (20)

Recently uploaded

Recently uploaded (20)

JUC Europe 2015: Multi-Node Environment as a Jenkins Slave (Compound-Slave)