Jumbo the Hadoop cluster bootstrapper

•

0 likes•140 views

Jumbo helps you deploy development environments for Big Data technologies. It takes a few minutes to get a custom virtualized Hadoop cluster up and running, with all the technologies you needed installed.

Jumbo
Presentation and future of
the Hadoop cluster bootstrapper
Marrakech
October 2018
2

What is Jumbo?
● Hadoop cluster bootstrapper
1. Describe your cluster in minutes
2. Let Jumbo provision everything
3. Start using your cluster
Marrakech
October 2018
3

What is Jumbo?
● Python 3 CLI tool
● Script generator
● Leverages trusted DevOps tools:
○ Vagrant
○ Ansible
○ Ambari
Marrakech
October 2018
4

Why Jumbo?
Need for multiple development environments:
● Fast and easy definition
● Fast provisioning
● Reusable code
Marrakech
October 2018
5

Jumbo
User Interface
Script generation and tools control
Marrakech
October 2018
6
Hadoop
installation
Node
configuration
VM provisioning
VBox | KVM

Jumbo internals
Marrakech
October 2018
7
Script generation:
Vagrantfile Jinja templating
Ansible playbooks Python dict → YAML (Ansible group_vars)
Ambari blueprint Python dict → JSON

● service
● component
● cluster
● node
Abstraction layer
Marrakech
October 2018
8
mycluster
HDFS
node01 node02 node03
NN
SNN
DN
DN

Abstraction layer
Marrakech
October 2018
9
node types
master master components of master-slave services
sidemaster key components of non master-slave services
worker slave components of master-slave services
edge server components
ldap security components

Available features
● Nodes (VM) definition
● Services topology
○ Automatic components placement
○ Automatic installation of service dependencies
○ Manual tuning with compatibility control
○ High Availability support
Marrakech
October 2018
10

Hadoop ecosystem
Database
SecurityProvisioning and
monitoring
Available features
Supported services:
Marrakech
October 2018
11

● Cluster templating
● Cluster security (Kerberos)
● Centralized versions and repositories management
● VM control within Jumbo
Available features
Marrakech
October 2018
12

Jumbo in action
Marrakech
October 2018
13

What’s next for Jumbo?
Marrakech
October 2018
14
● HDP - Hortonworks Data Platform
○ Support for all the services of the stack
○ HA support for all the services
● Smart cluster topology based on available resources

What’s next for Jumbo?
Marrakech
October 2018
15
● Create a bundle model for services to ease contribution
○ Service definition
○ Ansible playbooks
● Support other stacks
○ HDF - Hortonworks DataFlow
○ CDH - Cloudera Distribution for Hadoop

Get involved!
Facing problems using Jumbo?
Missing a service for your development environment?
github.com/adaltas/jumbo
jumbo.adaltas.com
Marrakech
October 2018
16

Building a minimum viable product in 3 months is easy. Scaling it towards a reactive system that can handle thousands of requests per second and deploying new versions without causing a denial of service is another challenge. Find out how at Crobox we scaled from a single machine (and point of failures) towards the high-available server cluster we are now running. On this journey you can also learn how we solved challenges with monitoring, logging and deployments.

Why we need open systems, and how to create them in the era of the Cloud (Ops...

Igalia

SFScon18 - Gerhard Sulzberger - Jason Tevnan - gitops with gitlab + terraform

South Tyrol Free Software Conference

For a lot of companies it is a challenge to automate their development pipeline. We would like to talk about one possible solution based on gitlab and terraform. The infrastructure and development process is created around git repositories. With Terraform it is possible to code also parts of the infrastructure. So each change in the application and also in the infrastrucure can be tracked within git repositorities. This is a great effort also for the CI process. So it is possible to automate the whole testing and integration processes very easy.

Nyc storm meetup_robdoherty

Robert Doherty

DevOps Fest 2019. Дмитрий Лагоза. CD for StartUp, cheap and furious

DevOps_Fest

Кто сказал, что у стартапов всегда много денег? Как быть если вам важна быстрая разработка и внедрение, но на это нет средств? Мы поговорим о способе построить Continuous Delivery процесс с ChatOps "за дешево" используя популярные инструменты и сервисы, такие как: Bitbucket, Kubernetes, Terraform, Scaleway и др. Я покажу вам как автоматизировать без дополнительных расходов.

Defrag 2014 - Blend Web IDEs, Open Source and PaaS to Create and Deploy APIs

Restlet

This session will explain how to craft an API using a dedicated Web IDE, implement the API in Java using an Open Source Framework, host and scale the API using generic PaaS, manage access to this API, including documentation, client SDKs, access management, firewall and analytics, using a dedicated PaaS. We will highlight how to combine the best of open source and cloud tools such as web IDEs, open source frameworks and PaaS to manage a web API project in a modern and effective way.

Integracia security do ci cd pipelines

Juraj Hantak

The document discusses integrating security into continuous integration and continuous delivery (CI/CD) pipelines. It covers topics like vulnerability management, application security testing methodologies, and integrating security checks into the phases of a CI/CD pipeline. The presenter demonstrates some security tools and their limitations. He emphasizes the importance of automating security, handling vulnerabilities like defects, and rolling out security testing incrementally to avoid disruption.

BLOCKCHAIN DELIVERY ENGINE (HYPERLEDGER)

Augusta HiTech

Augusta is a blockchain technology company, focused on building innovative products using blockchain, AI, Blockchain Testing as a Service and smart contracts. The company designs and develops independent intellectual property blockchain powered enterprise solutions in auditing, supply chain, healthcare, gaming, human identity, financial, medical marijuana and other industries. We have built/ building end-to-end blockchain solutions, including audit, healthcare, supply chain and restricted agricultural industries.

This document provides an overview of the Oslo project in OpenStack. The mission of Oslo is to produce Python libraries with shared code for OpenStack projects. The APIs are intended to be high quality, stable, documented and generally applicable. The team is made up of generalist code reviewers and specialist API maintainers. There are around 40 projects covering areas like messaging, databases, configuration and more. The team meets weekly on IRC and holds meetings on Mondays at 1500 UTC. Independent contributors are encouraged to get involved by picking a project, reviewing code, and fixing bugs.

Yipit - AWS Start-Up Customer

Amazon Web Services

Yipit is the largest daily deal aggregator in the US with 20 employees and 40+ servers hosted on AWS. It was built using Python, Django, Nginx, Gunicorn, Celery, MySQL, MongoDB, Redis and DynamoDB. AWS services currently used include S3, EC2, CloudWatch, CloudFront, RDS, ElastiCache, SQS, IAM, Route53 and DynamoDB. AWS provides benefits like speed of iteration, infrastructure as code, leveraging AWS expertise, and automation which helps Yipit as a startup. Yipit monitors its infrastructure using New Relic, Django Sentry, Papertrail, AppFirst and Stathat.

How we build Videome

Kane Zhu

This document discusses how to build scalable and stable web applications using VME. It focuses on measures for scalability including using distributed databases like MongoDB, message queues for logic decoupling, and stateless microservices for frontends and backends. It also covers high availability through techniques like MongoDB replication, RabbitMQ HA, and web server clusters. Additionally, it outlines the importance of automation through testing, DevOps, and containers, as well as comprehensive monitoring of availability, performance, errors, and alerts.

Hyperledger composer (from zero to hero )

EmanHerawy

Hyperledger Composer is a development toolset that makes it easier to build blockchain applications by allowing users to model business networks and integrate existing systems and data. It supports the Hyperledger Fabric blockchain infrastructure and runtime. The presentation introduces Hyperledger Composer, explains the relationship between Composer and Fabric, demonstrates how to develop a business network by modeling languages, access control, and transactions, and how to generate front-end and REST API applications and deploy networks locally or on IBM Bluemix.

Last Month in PHP - September 2016

Eric Poe

This document summarizes notable PHP news from last month. It discusses recent security releases for PHP, Drupal, WordPress, and other frameworks. It also mentions an upcoming Argon2 password hashing feature in PHP 7.2 and several PHP conferences happening in the coming months. Security vulnerabilities are noted in ImageMagick and CMS platforms, and users are encouraged to upgrade. The PHP right way documentation was also updated on several topics. Finally, the document advertises the Hactoberfest event and upcoming Nomad PHP online talks.

Going Enterprise QGIS and PostGIS

QGIS UK

Go lambda-presentation

Steven White

Steven White gave a presentation on using Go (golang) to build serverless applications targeting AWS Lambda. He defined serverless computing and Function as a Service (FaaS). The presentation included setting up a Go project to target Lambda, integrating with AWS API Gateway, and using DynamoDB as a backend database for a simple REST API, with live coding demonstrations. Resources for further learning about serverless architectures using Go and AWS Lambda were also provided.

[WSO2Con EU 2018] Deploying Applications in K8S and Docker

WSO2

Within the last four years container technologies have become very popular. A lot of companies and developers are now using containers to ship their applications. Docker provides an easy-to-use packaging model to bundle the application. However in many cases, a single container is not enough to run an application. It requires multiple containers, scaled into multiple host machines to become a production grade deployment. Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. This presentation discusses best practices of deploying application in Docker and Kubernetes while discussing Docker and Kubernetes concepts.

Modern Monitoring and processing logs

Dmitry Lavrinenko

This document discusses modern approaches to monitoring systems and processing logs compared to older methods. It outlines what types of metrics and data are collected like host metrics and events. It then lists common tools used for collecting data like CollectD and Sensu, storing data in databases like InfluxDB and Graphite, and visualizing data with Grafana. It also discusses log management tools like Logstash, ElasticSearch, Kafka and Splunk for processing logs.

Introduction to Modern DevOps Technologies

Kriangkrai Chaonithi

This document provides an introduction to modern DevOps technologies. It discusses DevOps concepts like source code management using Git, different methods of deploying programs including using bare metal servers, virtualization, containers, and cloud functions. Specific container and container orchestration technologies like Docker and Kubernetes are explained. Continuous integration and continuous delivery (CI/CD) practices are also introduced. The presentation includes an agenda with slides on these topics and ends with a question and answer section and announcement of a Docker workshop to deploy an HTTP server container.

Introduction to DevOps and the Practical Use Cases at Credit OK

Kriangkrai Chaonithi

The document provides an introduction to DevOps and practical use cases. It discusses what DevOps is, why it is popular, the skills required of DevOps engineers, and common DevOps technologies like version control, CI/CD pipelines, containers, and monitoring. It also summarizes Credit OK's use of DevOps practices like Docker, Kubernetes, and GitLab CI/CD pipelines for their credit scoring platform. Finally, it outlines some modern obstacles in software development and concludes that DevOps can help ensure quality, improve productivity, and automate infrastructure through practices like continuous integration, containerization, and logging/monitoring.

RealTime Recommendations @Netflix - Spark

Nitin S

This document discusses Netflix's use of near real-time recommendations using Spark streaming. It provides examples of use cases like video insights and billboard recommendations that require processing data with low latency. The infrastructure for handling terabytes of daily data across regions at Netflix's scale is also described, along with challenges of scaling streaming workloads and ensuring reliability.

DevOps: Coding Defines Monitoring

Opsta

SAIS2018 - Fact Store At Netflix Scale

Nitin S

This document discusses recommendations and machine learning at Netflix. It provides an overview of: - How Netflix provides personalized recommendations on member homepages to help them find content to watch. - Netflix's experimentation cycle of designing experiments, collecting data, generating features, training models, and doing A/B testing. - How Netflix handles "facts" or input data for recommendations, including how facts change over time and how they are logged and stored at scale. - The challenges of logging and accessing facts at Netflix's scale, and how they are addressing issues like deduplication, performance, and supporting different access patterns.

Head in the clouds @ bol.com

Maarten Dirkse

QGIS UK: Developing with QGIS - What is possible (Lutra Consulting)

Ross McDonald

QGIS is a modular and extensible open source geographic information system. It has a Python API that allows developers to build plugins that interact with QGIS functionality and libraries. Plugins can be built with C++ or Python and interface with QGIS through the QgisInterface class. Example plugins include InaSAFE for hazard impact scenarios and Crayfish for visualizing flood modeling results. Developers can also create standalone applications using the QGIS libraries for tasks like automating GIS processes, building custom editing tools, or developing specialized mapping applications.

Building Scalable Backends with Go

Shiju Varghese

Drupal Brisbane Meetup :: Drupal in late 2017-2018

Vladimir Roudakov

The document summarizes Drupal developments in late 2017 and plans for 2018. It discusses the releases of Drupal 8.4 and various PHP frameworks in 2017. For 2018, it outlines the expected releases of Drupal 8.5 and 8.6 as well as upcoming events like Drupal Global Training Days, DrupalCamp Sydney, DrupalSouth in Canberra, and DrupalCon in Nashville. Experimental Drupal modules are also reviewed, noting their progress toward stability between versions 8.3 and 8.4.

MongoDB .local London 2019: Nationwide Building Society: Building Mobile Appl...

MongoDB

Nationwide Building Society has invested £4.1 billion in technology and is creating 750 new digital roles. They need a "Speed Layer" to support increased mobile and digital activation, open banking, and enhanced customer propositions. The Speed Layer uses Kafka as an event hub, MongoDB as an operational data store, and stream processing to aggregate and enrich data. It provides pre-populated caches and introduces an event-based architecture. To ensure high resilience across two data centers, Nationwide uses independent Kafka, stream processing and MongoDB clusters in each rather than a stretched MongoDB cluster. Nationwide loaded 15 billion transactions into MongoDB by bucketing documents by account and month to improve performance for reads. They conducted proof-of-concepts to

Monitoring to the Nth tier: The state of distributed tracing in 2016

AppNeta

This document discusses distributed tracing and monitoring of distributed systems. It provides examples of common distributed architectures and outlines challenges with instrumentation, trace ID propagation, and extracting valuable monitoring information. Distributed tracing allows understanding request flows, latencies, errors and more across tiers. While initially for performance, tracing provides a rich data set for analyzing individual components and custom metrics. Future directions include context propagation for things beyond performance like authentication and flow control.

LINE's Private Cloud - Meet Cloud Native World

LINE Corporation

This document discusses LINE's private cloud platform Verda and two new services: Verda Kubernetes as a Service (KaaS) and Verda Event Handler. Verda KaaS provides managed Kubernetes clusters to developers. It is built using Rancher and aims to simplify Kubernetes usage. Verda Event Handler aims to improve automation by defining operations as functions that are triggered by events. It will utilize Knative to provide a functions-as-a-service platform and improve visibility, operability, and maintenance of automation scripts. The status and future plans of these new services are also outlined.

IDEALIZE 2023 - NodeJS & Firebase Session

Brion Mario

What's hot

Oslo Vancouver Onboarding

Ben Nemec

Yipit - AWS Start-Up Customer

Amazon Web Services

How we build Videome

Kane Zhu

Hyperledger composer (from zero to hero )

EmanHerawy

Last Month in PHP - September 2016

Eric Poe

Going Enterprise QGIS and PostGIS

QGIS UK

Go lambda-presentation

Steven White

[WSO2Con EU 2018] Deploying Applications in K8S and Docker

WSO2

Modern Monitoring and processing logs

Dmitry Lavrinenko

Introduction to Modern DevOps Technologies

Kriangkrai Chaonithi

Introduction to DevOps and the Practical Use Cases at Credit OK

Kriangkrai Chaonithi

RealTime Recommendations @Netflix - Spark

Nitin S

DevOps: Coding Defines Monitoring

Opsta

SAIS2018 - Fact Store At Netflix Scale

Nitin S

Head in the clouds @ bol.com

Maarten Dirkse

QGIS UK: Developing with QGIS - What is possible (Lutra Consulting)

Ross McDonald

Building Scalable Backends with Go

Shiju Varghese

Drupal Brisbane Meetup :: Drupal in late 2017-2018

Vladimir Roudakov

MongoDB .local London 2019: Nationwide Building Society: Building Mobile Appl...

MongoDB

Monitoring to the Nth tier: The state of distributed tracing in 2016

AppNeta

What's hot (20)

Oslo Vancouver Onboarding

Yipit - AWS Start-Up Customer

How we build Videome

Hyperledger composer (from zero to hero )

Last Month in PHP - September 2016

Going Enterprise QGIS and PostGIS

Go lambda-presentation

[WSO2Con EU 2018] Deploying Applications in K8S and Docker

Modern Monitoring and processing logs

Introduction to Modern DevOps Technologies

Introduction to DevOps and the Practical Use Cases at Credit OK

RealTime Recommendations @Netflix - Spark

DevOps: Coding Defines Monitoring

SAIS2018 - Fact Store At Netflix Scale

Head in the clouds @ bol.com

QGIS UK: Developing with QGIS - What is possible (Lutra Consulting)

Building Scalable Backends with Go

Drupal Brisbane Meetup :: Drupal in late 2017-2018

MongoDB .local London 2019: Nationwide Building Society: Building Mobile Appl...

Monitoring to the Nth tier: The state of distributed tracing in 2016

Similar to Jumbo the Hadoop cluster bootstrapper

LINE's Private Cloud - Meet Cloud Native World

LINE Corporation

IDEALIZE 2023 - NodeJS & Firebase Session

Brion Mario

DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way

smalltown

Angular2 - A story from the trenches

Johannes Rudolph

This document discusses Angular (2), including its history, architecture, and use at Meshcloud. It notes that Angular has undergone significant changes through beta releases. Key aspects highlighted include its simplified component-based structure, TypeScript integration, and use of Observables and services. Meshcloud's single page application was built with Angular CLI and uses lazy loading, AOT compilation, and HATEOAS for its REST APIs. Challenges discussed include debugging, performance analysis, and CORS configuration complexities.

State of serverless

Anurag Saran

Serverless/Frugal Architecture describes the benefits of serverless computing including continuous scaling, developer productivity, and fully managed operations. It discusses AWS Lambda's programming model of handlers, contexts, events, and asynchronous exceptions. Lambda supports various languages and has resource limits. Serverless computing is gaining adoption with Amazon Lambda as the pioneer, and other cloud providers like IBM, Microsoft, and Google developing their own serverless offerings. Challenges of serverless include testing, state management, and lack of observability. Open source projects are also emerging in this space like OpenWhisk.

Intro to creating kubernetes operators

Juraj Hantak

This document provides an introduction to Kubernetes operators and the Java Operator SDK framework. It begins with an overview of operators and how they extend the Kubernetes API using custom resource definitions (CRDs). A simple web server example is demonstrated. The document then discusses how the Java Operator SDK addresses common problems like concurrency handling and retries. It also covers features like finalizers, event sources, and integration with Java frameworks. In comparison to Terraform, operators are meant to manage resources, while Terraform provisions resources managed by cloud providers. The status of custom resources in operators is analogous to state in Terraform.

AWS Lambda and Serverless framework: lessons learned while building a serverl...

Luciano Mammino

The document discusses lessons learned from building a serverless company. It introduces Planet 9 Energy and their use of AWS Lambda and the Serverless framework. Key topics covered include security, quality assurance, developer experience, costs, and lessons learned. Some challenges discussed are debugging, API Gateway custom domains, and Lambda limitations. The document emphasizes that serverless architectures provide infinite scalability at low cost but also have some limitations that require management.

Rancher 2.0 Technical Deep Dive

LINE Corporation

Traefik on Kubernetes at MySocialApp (CNCF Paris Meetup)

Pierre Mavro

This document discusses using Traefik as an ingress controller on Kubernetes to route traffic to applications running on a Kubernetes cluster. It provides details on Traefik's features like load balancing, SSL termination, and integration with Let's Encrypt for automatic SSL certificate management. It also describes how the presenter's company MySocialApp uses Traefik on their Kubernetes infrastructure, including configuration with annotations, support for high availability, and using Consul for storage and distributed locking. The document offers advice on dealing with Let's Encrypt rate limiting and using a CDN like Cloudflare to help mitigate those issues in a production environment.

Openshift serverless Solution

Ryan ZhangCheng

Unifying Frontend and Backend Development with Scala - ScalaCon 2021

Taro L. Saito

Scala can be used for developing both frontend (Scala.js) and backend (Scala JVM) applications. A missing piece has been bridging these two worlds using Scala. We built Airframe RPC, a framework that uses Scala traits as a unified RPC interface between servers and clients. With Airframe RPC, you can build HTTP/1 (Finagle) and HTTP/2 (gRPC) services just by defining Scala traits and case classes. It simplifies web application design as you only need to care about Scala interfaces without using existing web standards like REST, ProtocolBuffers, OpenAPI, etc. Scala.js support of Airframe also enables building interactive Web applications that can dynamically render DOM elements while talking with Scala-based RPC servers. With Airframe RPC, the value of Scala developers will be much higher both for frontend and backend areas.

Cloud native IPC for Microservices Workshop @ Containerdays 2022

QAware GmbH

This document provides an agenda and overview for a workshop on migrating from REST to gRPC. The agenda covers exercises on using Protocol Buffers with Quarkus and JAX-RS, building a gRPC API with Quarkus, implementing a gRPC REST gateway, and using a gRPC web client with Envoy. Additional sections provide background on tools like Protocol Buffers, gRPC, and the gRPC ecosystem. The document is intended to guide participants through hands-on exercises demonstrating techniques for migrating a REST API to a gRPC API.

Design Summit - Technology Vision - Oleg Barenboim and Jason Frey

ManageIQ

Node.js scaling in highload

Timur Shemsedinov

This document outlines different steps in scaling Node.js applications from 2012 to 2019. It begins with using Node.js in cluster mode with Nginx as a reverse proxy. It progresses to using CDNs for static files, in-memory databases, and eventually custom protocols for real-time data synchronization across servers and clients. Key aspects discussed include data synchronization, offline capabilities, interactivity, scalability, and high connectivity. Alternative approaches and bad practices are also addressed.

How Sysbee Manages Infrastructures and Provides Advanced Monitoring by Using ...

InfluxData

Discover how Sysbee helps organizations bring DevOps culture to small and medium enterprises. Their team helps their customers by improving stability, security, scalability — by providing cost-effective IT infrastructure. Learn how monitoring everything can improve your processes and simplify debugging! Sysbee’s introspection on monitoring tools over the years How TSDB’s, and specifically InfluxDB, fits into improving observability Their approach to using the TICK Stack to improve the web hosting industry

Openstack India May Meetup

Deepak Garg

Openstack is open source software that allows users to create an Infrastructure as a Service (IaaS) cloud by pooling physical compute, storage, and network resources. It provides on-demand, scalable computing and storage through components like Nova (compute), Swift (object storage), Glance (images), Keystone (identity), and Quantum (networking). The presentation covers the architecture and components of Openstack, how it works from a user perspective, its history and motivation, partners, open development model, and the Openstack community in India.

Advanced deeplearning4j features

Adam Gibson

TSC BoF: OSS Toolchain Discussion - SFO17-409

Linaro

Session ID: SFO17-409 Session Name: TSC BoF: OSS Toolchain Discussion - SFO17-409 Speaker: Ryan Arnold Track: Toolchain ★ Session Summary ★ Discussion on the various OSS Tools efforts in Linaro. What are we doing now and how should this evolve? --------------------------------------------------- ★ Resources ★ Event Page: http://connect.linaro.org/resource/sfo17/sfo17-409/ Presentation: Video: https://www.youtube.com/watch?v=XIhuNcSAVXU --------------------------------------------------- ★ Event Details ★ Linaro Connect San Francisco 2017 (SFO17) 25-29 September 2017 Hyatt Regency San Francisco Airport --------------------------------------------------- Keyword: 'http://www.linaro.org' 'http://connect.linaro.org' --------------------------------------------------- Follow us on Social Media https://www.facebook.com/LinaroOrg https://twitter.com/linaroorg https://www.youtube.com/user/linaroorg?sub_confirmation=1 https://www.linkedin.com/company/1026961

Livnat Peer & Arthur Berezin, Red Hat - Neutron High Availability - OpenStack...

Cloud Native Day Tel Aviv

This document discusses techniques for achieving high availability in OpenStack Neutron. It describes enabling technologies like Pacemaker and HAProxy that provide redundancy. It also outlines several built-in Neutron mechanisms for high availability, including keeping DHCP and metadata agents highly available, and using Virtual Router Redundancy Protocol (VRRP) for L3 agent HA. Distributed Virtual Routing (DVR) is also discussed as a way to improve availability by distributing routing functionality across compute nodes.

Neutron high availability open stack architecture openstack israel event 2015

Arthur Berezin

Similar to Jumbo the Hadoop cluster bootstrapper (20)

LINE's Private Cloud - Meet Cloud Native World

IDEALIZE 2023 - NodeJS & Firebase Session

DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way

Angular2 - A story from the trenches

State of serverless

Intro to creating kubernetes operators

AWS Lambda and Serverless framework: lessons learned while building a serverl...

Rancher 2.0 Technical Deep Dive

Traefik on Kubernetes at MySocialApp (CNCF Paris Meetup)

Openshift serverless Solution

Unifying Frontend and Backend Development with Scala - ScalaCon 2021

Cloud native IPC for Microservices Workshop @ Containerdays 2022

Design Summit - Technology Vision - Oleg Barenboim and Jason Frey

Node.js scaling in highload

How Sysbee Manages Infrastructures and Provides Advanced Monitoring by Using ...

Openstack India May Meetup

Advanced deeplearning4j features

TSC BoF: OSS Toolchain Discussion - SFO17-409

Livnat Peer & Arthur Berezin, Red Hat - Neutron High Availability - OpenStack...

Neutron high availability open stack architecture openstack israel event 2015

Recently uploaded

leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...

alexjohnson7307

dbms calicut university B. sc Cs 4th sem.pdf

Shinana2

Fueling AI with Great Data with Airbyte Webinar

Zilliz

GenAI Pilot Implementation in the organizations

kumardaparthi1024

Building Production Ready Search Pipelines with Spark and Milvus

Zilliz

HCL Notes and Domino License Cost Reduction in the World of DLAU

panagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/ The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this! We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model. Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward. These topics will be covered - Reducing license cost by finding and fixing misconfigurations and superfluous accounts - How do CCB and CCX licenses really work? - Understanding the DLAU tool and how to best utilize it - Tips for common problem areas, like team mailboxes, functional/test users, etc - Practical examples and best practices to implement right away

Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf

Malak Abu Hammad

Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers: * What is Vector Search? * Importance and benefits of vector search * Practical use cases across various industries * Step-by-step implementation guide * Live demos with code snippets * Enhancing LLM capabilities with vector search * Best practices and optimization strategies Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications. #MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology

Skybuffer SAM4U tool for SAP license adoption

Tatiana Kojar

Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool. SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.

Driving Business Innovation: Latest Generative AI Advancements & Success Story

Safe Software

Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency. During the hour, we’ll take you through: Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board. Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes. Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI. We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI. This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack

shyamraj55

GraphRAG for Life Science to increase LLM accuracy

Tomaz Bratanic

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on integration of Salesforce with Bonterra Impact Management. Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Columbus Data & Analytics Wednesdays - June 2024

Jason Packer

Finale of the Year: Apply for Next One!

GDSC PJATK

System Design Case Study: Building a Scalable E-Commerce Platform - Hiike

Hiike

Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf

flufftailshop

When it comes to unit testing in the .NET ecosystem, developers have a wide range of options available. Among the most popular choices are NUnit, XUnit, and MSTest. These unit testing frameworks provide essential tools and features to help ensure the quality and reliability of code. However, understanding the differences between these frameworks is crucial for selecting the most suitable one for your projects.

Serial Arm Control in Real Time Presentation

tolgahangng

Energy Efficient Video Encoding for Cloud and Edge Computing Instances

Alpen-Adria-Universität

Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...

saastr

Ocean lotus Threat actors project by John Sitima 2024 (1).pptx

SitimaJohn

Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.

Recently uploaded (20)

leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...

dbms calicut university B. sc Cs 4th sem.pdf

Fueling AI with Great Data with Airbyte Webinar

GenAI Pilot Implementation in the organizations

Building Production Ready Search Pipelines with Spark and Milvus

HCL Notes and Domino License Cost Reduction in the World of DLAU

Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf

Skybuffer SAM4U tool for SAP license adoption

Driving Business Innovation: Latest Generative AI Advancements & Success Story

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack

GraphRAG for Life Science to increase LLM accuracy

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

Columbus Data & Analytics Wednesdays - June 2024

Finale of the Year: Apply for Next One!

System Design Case Study: Building a Scalable E-Commerce Platform - Hiike

Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf

Serial Arm Control in Real Time Presentation

Energy Efficient Video Encoding for Cloud and Edge Computing Instances

Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...

Ocean lotus Threat actors project by John Sitima 2024 (1).pptx

Jumbo the Hadoop cluster bootstrapper

1. Marrakech October 2018

2. Jumbo Presentation and future of the Hadoop cluster bootstrapper Marrakech October 2018 2

3. What is Jumbo? ● Hadoop cluster bootstrapper 1. Describe your cluster in minutes 2. Let Jumbo provision everything 3. Start using your cluster Marrakech October 2018 3

4. What is Jumbo? ● Python 3 CLI tool ● Script generator ● Leverages trusted DevOps tools: ○ Vagrant ○ Ansible ○ Ambari Marrakech October 2018 4

5. Why Jumbo? Need for multiple development environments: ● Fast and easy definition ● Fast provisioning ● Reusable code Marrakech October 2018 5

6. Jumbo User Interface Script generation and tools control Marrakech October 2018 6 Hadoop installation Node configuration VM provisioning VBox | KVM

7. Jumbo internals Marrakech October 2018 7 Script generation: Vagrantfile Jinja templating Ansible playbooks Python dict → YAML (Ansible group_vars) Ambari blueprint Python dict → JSON

8. ● service ● component ● cluster ● node Abstraction layer Marrakech October 2018 8 mycluster HDFS node01 node02 node03 NN SNN DN DN

9. Abstraction layer Marrakech October 2018 9 node types master master components of master-slave services sidemaster key components of non master-slave services worker slave components of master-slave services edge server components ldap security components

10. Available features ● Nodes (VM) definition ● Services topology ○ Automatic components placement ○ Automatic installation of service dependencies ○ Manual tuning with compatibility control ○ High Availability support Marrakech October 2018 10

11. Hadoop ecosystem Database SecurityProvisioning and monitoring Available features Supported services: Marrakech October 2018 11

12. ● Cluster templating ● Cluster security (Kerberos) ● Centralized versions and repositories management ● VM control within Jumbo Available features Marrakech October 2018 12

13. Jumbo in action Marrakech October 2018 13

14. What’s next for Jumbo? Marrakech October 2018 14 ● HDP - Hortonworks Data Platform ○ Support for all the services of the stack ○ HA support for all the services ● Smart cluster topology based on available resources

15. What’s next for Jumbo? Marrakech October 2018 15 ● Create a bundle model for services to ease contribution ○ Service definition ○ Ansible playbooks ● Support other stacks ○ HDF - Hortonworks DataFlow ○ CDH - Cloudera Distribution for Hadoop

16. Get involved! Facing problems using Jumbo? Missing a service for your development environment? github.com/adaltas/jumbo jumbo.adaltas.com Marrakech October 2018 16

17. Questions? Marrakech October 2018 17

Jumbo the Hadoop cluster bootstrapper

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Jumbo the Hadoop cluster bootstrapper

Similar to Jumbo the Hadoop cluster bootstrapper (20)

Recently uploaded

Recently uploaded (20)

Jumbo the Hadoop cluster bootstrapper