Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle)

•Download as ODP, PDF•

13 likes•9,415 views

This document introduces Pentaho Data Integration (KETL), an open source extract-transform-load (ETL) tool. It is part of Pentaho's full stack business intelligence platform. The document discusses KETL's capabilities for extracting, transforming and loading data from various sources through jobs and transformations. It also provides an overview of Pentaho's community and resources for using and contributing to its open source software.

Technology

Moving and Transforming Data with

Pentaho Data Integration
a.k.a. KETTLE

Welcome!
• Software engineer
• rbouman@pentaho.com

Pentaho
• Business Intelligence & Analytics
• Full stack
• Open core
–

GPLv2, Apache 2.0

–

Enterprise and OEM licenses

• Java-based
• Web front-ends

The Pentaho Stack
•
•
•
•
•
•
•
•
•
•

Data Integration / ETL
Big Data / NoSQL
Data Modeling
Reporting
OLAP / Analysis
Data Visualization
Dashboarding
Data Mining / Predictive Analysis
(Mobile) Delivery
Bursting, Scheduling, Self Service

Full Stack BI & BA

Sources

Reports

OLAP

Blending

Visualization
Data Warehouse
ETL

T

T

T

T

Models
D
D

Instant Analytics

Dashboards

D

F
D

Mining
D

Information:

A well-prepared, well-presented meal

Extraction:

Catching the right data and hauling it in

Transformation

Disgusting job of validating & cleaning data

Loading:

Store manageable units for later use

Pentaho Data Integration
• Kettle
– Extract,

Transform, Load

– Blending
– Instant Analytics

• Changing input to desired output

Kettle Architecture
Data Integration Engine
Job
Engine

Call

Job

Transformation
Engine

Transformation

Tools and Utilities
Launch:
Kitchen, Carte

Launch:
Pan

Develop: Spoon

Repository
(RDBMS)
.kjb

.ktr

Jobs & Transformations
• Jobs
– Synchronous

workflow of job entries (tasks)

• Transformations
– Stepwise

parallel & asynchronous

processing of a recordstream
• Distributed

Sources and Destinations
•
•
•
•
•
•
•

RDBMS (> 40)
NoSQL / Big Data
OLAP (Mondrian, Palo, XML/A)
Web (REST, SOAP, XML, JSON .)
Files (CSV, Fixed, Excel …)
ERP (SAP, Salesforce, OpenERP)
...way Too Many To Mention™!

Transformations
•
•
•
•
•
•
•
•

String & Date manipulation
Data Validation / Business Rules
Lookup / Join
Calculation, Statistics
Cryptography
Decisions, Flow control
Scripting
...> 150, excluding plugins

Demo
• NLUUG Program Webpage
Input

Output
dim_room

ETL

dim_track

fact_talk

dim_speaker

dim_company

Business Model
• Open core
– Majority
– Give

is open source

and take

• Enterprise Edition
– Extra

features, Support, Services

Community
• Plugins
– Pentaho

Marketplace

• Code contributions
• Applications & Solutions
• Tutorials, Support

Pentaho Software
• Community Edition Binaries
–

community.pentaho.com (release)

ci.pentaho.com (development)
• Source code
–

–

github.com/pentaho

• Enterprise Edition Evaluation
–

pentaho.com/testdrive

Resources
• Online documentation
– infocenter.pentaho.com
– wiki.pentaho.com

(manual)

(community wiki)

• Issue tracker
– jira.pentaho.com

• Community Support
– forums.pentaho.com
– Freenode

IRC: ##pentaho

Thank You
Join the conversation. You can find us on:
blog.pentaho.com
@Pentaho
Facebook.com/Pentaho
Pentaho Business Analytics
25

Fine-grained data protection at a column level in data lake environments has become a mandatory requirement to demonstrate compliance with multiple local and international regulations across many industries today. ORC is a self-describing type-aware columnar file format designed for Hadoop workloads that provides optimized streaming reads but with integrated support for finding required rows quickly. Owen O’Malley dives into the progress the Apache community made for adding fine-grained column-level encryption natively into ORC format, which also provides capabilities to mask or redact data on write while protecting sensitive column metadata such as statistics to avoid information leakage. The column encryption capabilities will be fully compatible with Hadoop Key Management Server (KMS) and use the KMS to manage master keys, providing the additional flexibility to use and manage keys per column centrally.

Get Savvy with Snowflake

Matillion

In this webinar you'll learn how to quickly and easily improve your business using Snowflake and Matillion ETL for Snowflake. Webinar presented by Solution Architects Craig Collier (Snowflake) adn Kalyan Arangam (Matillion). In this webinar: - Learn to optimize Snowflake and leverage Matillion ETL for Snowflake - Discover tips and tricks to improve performance - Get invaluable insights from data warehousing pros

The Marriage of the Data Lake and the Data Warehouse and Why You Need Both

Adaryl "Bob" Wakefield, MBA

In the past few years, the term "data lake" has leaked into our lexicon. But what exactly IS a data lake? Some IT managers confuse data lakes with data warehouses. Some people think data lakes replace data warehouses. Both of these conclusions are false. Their is room in your data architecture for both data lakes and data warehouses. They both have different use cases and those use cases can be complementary. Todd Reichmuth, Solutions Engineer with Snowflake Computing, has spent the past 18 years in the world of Data Warehousing and Big Data. He spent that time at Netezza and then later at IBM Data. Earlier in 2018 making the jump to the cloud at Snowflake Computing. Mike Myer, Sales Director with Snowflake Computing, has spent the past 6 years in the world of Security and looking to drive awareness to better Data Warehousing and Big Data solutions available! Was previously at local tech companies FireMon and Lockpath and decided to join Snowflake due to the disruptive technology that's truly helping folks in the Big Data world on a day to day basis.

Master the Multi-Clustered Data Warehouse - Snowflake

Matillion

Snowflake is one of the most powerful, efficient data warehouses on the market today—and we joined forces with the Snowflake team to show you how it works! In this webinar: - Learn how to optimize Snowflake - Hear insider tips and tricks on how to improve performance - Get expert insights from Craig Collier, Technical Architect from Snowflake, and Kalyan Arangam, Solution Architect from Matillion - Find out how leading brands like Converse, Duo Security, and Pets at Home use Snowflake and Matillion ETL to make data-driven decisions - Discover how Matillion ETL and Snowflake work together to modernize your data world - Learn how to utilize the impressive scalability of Snowflake and Matillion

Data ingestion and distribution with apache NiFi

Lev Brailovskiy

In this session, we will cover our experience working with Apache NiFi, an easy to use, powerful, and reliable system to process and distribute a large volume of data. The first part of the session will be an introduction to Apache NiFi. We will go over NiFi main components and building blocks and functionality. In the second part of the session, we will show our use case for Apache NiFi and how it's being used inside our Data Processing infrastructure.

Pentaho-BIEdureka!

Data Lakehouse, Data Mesh, and Data Fabric (r1)

James Serra

Kafka for Real-Time Replication between Edge and Hybrid Cloud

Kai Wähner

Not all workloads allow cloud computing. Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration. This session explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell and Confluent Cloud.

[EN] Building modern data pipeline with Snowflake + DBT + Airflow.pdf

Chris Hoyean Song

I'm posting the slide presented at the Snowflake user group meet up. NFT Bank has introduced DBT to rebuild and operate the entire data pipeline from scratch. Data quality control and monitoring are critical as data is at the core of the company. You can manage your numerous data validation tests in organized way. You can add one data validation test with just single line of yaml. You can build the data catalog and data lineage docs if you just implement your data pipeline on top of DBT without big effort. --- Session 1: Data Quality & Productivity Data Quality Data Quality Validation Data Catalog, Lineage Documentation DBT Introduction Session 2: Integrate DBT with Airflow DBT Cloud or Airflow? Astronomer Cosmos dbt deps Session 3: Cost Optimization Query Optimization Cost Monitoring

Automated Metadata Management in Data Lake – A CI/CD Driven Approach

Databricks

We as data engineers are aware of trade off’s between development speed, metadata governance and schema evolution (or restriction) in rapidly evolving organization. Our day to day activities involve adding/removing/updating tables, protecting PII Information, curating and exposing data to our consumers. While our data lake keeps growing exponentially, there is equal increase in our downstream consumers. Struggle is to maintain balance between quickly promoting metadata changes with robust validation for downstream systems stability. In relational world DDL, DML changes can be managed through numerous options available for every kind of database from the vendor or 3rd party. As engineers we developed a tool which uses centralized git managed repository of data schemas in yml structure with ci/cd capabilities which maintains stability of our data lake and downstream systems. In this presentation Northwestern Mutual Engineers, will discuss how they designed and developed new end-to-end ci/cd driven metadata management tool to make introduction of new tables/views, managing access requests etc in a more robust, maintainable and scalable way, all with only checking in yml files. This tool can be used by people who have no or minimal knowledge of spark. Key focus will be: Need for metadata management tool in a data lake Architecture and Design of the tool Maintaining information on databases/tables/views like schema, owner, PII, description etc in simple to understand yml structure Live demo of creating a new table with CI/CD promotion to production

ETL Using Informatica Power Center

Edureka!

Informatica provides the market's leading data integration platform. Tested on nearly 500,000 combinations of platforms and applications, the data integration platform inter operates with the broadest possible range of disparate standards, systems, and applications. This unbiased and universal view makes Informatica unique in today's market as a leader in the data integration platform. It also makes Informatica the ideal strategic platform for companies looking to solve data integration issues of any size.

Running Apache NiFi with Apache Spark : Integration Options

Timothy Spann

Introducing the Snowflake Computing Cloud Data Warehouse

Snowflake Computing

Etl overview training

Mondy Holten

1- Introduction of Azure data factory.pptx

BRIJESH KUMAR

Introduction To Pentaho

pentaho Content

Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...

Denodo

This is the first in a series of five webinars that look 'under the covers' of Denodo's industry leading Data Virtualization Platform. The webinar will provide an overview of the architecture and key modules of the Denodo Platform - subsequent webinars in the series will take a deeper look at some of the key modules and capabilities of the platform, including performance, scalability, security, and so on. More information and FREE registrations to this webinar: http://goo.gl/fLi2bC To learn more click to this link: http://go.denodo.com/a2a Join the conversation at #Architect2Architect Agenda: The Denodo Platform Platform Architecture Key Modules Connectors Data Services and APIs

Snowflake for Data Engineering

Harald Erb

Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose

Aldrin Piri

Apache Iceberg Presentation for the St. Louis Big Data IDEA

Adam Doyle

Pentaho bi suite overview presentationnvvrajesh

Business Intelligence and Big Data Analytics with Pentaho

Uday Kothari

This webinar gives an overview of the Pentaho technology stack and then delves deep into its features like ETL, Reporting, Dashboards, Analytics and Big Data. The webinar also facilitates a cross industry perspective and how Pentaho can be leveraged effectively for decision making. In the end, it also highlights how apart from strong technological features, low TCO is central to Pentaho’s value proposition. For BI technology enthusiasts, this webinar presents easiest ways to learn an end to end analytics tool. For those who are interested in developing a BI / Analytics toolset for their organization, this webinar presents an interesting option of leveraging low cost technology. For big data enthusiasts, this webinar presents overview of how Pentaho has come out as a leader in data integration space for Big data. Pentaho is one of the leading niche players in Business Intelligence and Big Data Analytics. It offers a comprehensive, end-to-end open source platform for Data Integration and Business Analytics. Pentaho’s leading product: Pentaho Business Analytics is a data integration, BI and analytics platform composed of ETL, OLAP, reporting, interactive dashboards, ad hoc analysis, data mining and predictive analytics.

What's hot

Protect your private data with ORC column encryption

Owen O'Malley

Get Savvy with Snowflake

Matillion

The Marriage of the Data Lake and the Data Warehouse and Why You Need Both

Adaryl "Bob" Wakefield, MBA

Master the Multi-Clustered Data Warehouse - Snowflake

Matillion

Data ingestion and distribution with apache NiFi

Lev Brailovskiy

Pentaho-BIEdureka!

Data Lakehouse, Data Mesh, and Data Fabric (r1)

James Serra

Kafka for Real-Time Replication between Edge and Hybrid Cloud

Kai Wähner

[EN] Building modern data pipeline with Snowflake + DBT + Airflow.pdf

Chris Hoyean Song

Automated Metadata Management in Data Lake – A CI/CD Driven Approach

Databricks

ETL Using Informatica Power Center

Edureka!

Running Apache NiFi with Apache Spark : Integration Options

Timothy Spann

Introducing the Snowflake Computing Cloud Data Warehouse

Snowflake Computing

Etl overview training

Mondy Holten

1- Introduction of Azure data factory.pptx

BRIJESH KUMAR

Introduction To Pentaho

pentaho Content

Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...

Denodo

Snowflake for Data Engineering

Harald Erb

Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose

Aldrin Piri

Apache Iceberg Presentation for the St. Louis Big Data IDEA

Adam Doyle

What's hot (20)

Protect your private data with ORC column encryption

Get Savvy with Snowflake

The Marriage of the Data Lake and the Data Warehouse and Why You Need Both

Master the Multi-Clustered Data Warehouse - Snowflake

Data ingestion and distribution with apache NiFi

Pentaho-BI

Data Lakehouse, Data Mesh, and Data Fabric (r1)

Kafka for Real-Time Replication between Edge and Hybrid Cloud

[EN] Building modern data pipeline with Snowflake + DBT + Airflow.pdf

Automated Metadata Management in Data Lake – A CI/CD Driven Approach

ETL Using Informatica Power Center

Running Apache NiFi with Apache Spark : Integration Options

Introducing the Snowflake Computing Cloud Data Warehouse

Etl overview training

1- Introduction of Azure data factory.pptx

Introduction To Pentaho

Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...

Snowflake for Data Engineering

Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose

Apache Iceberg Presentation for the St. Louis Big Data IDEA

Viewers also liked

Pentaho bi suite overview presentationnvvrajesh

Business Intelligence and Big Data Analytics with Pentaho

Uday Kothari

Advanced ETL2 Pentaho Sunny U Okoro

NGINX Plus PLATFORM For Flawless Application Delivery

Ashnikbiz

Flawless Application Delivery using Nginx Plus By leveraging these latest features: • Support for HTTP/2 standard • Thread pools and socket sharding and how it can help improve performance • NTLM support and new TCP security enhancements • Advanced NGINX Plus monitoring, management and visibility of health & load checks Catch this exclusive Google Hangout live! November 4th, 2015 | 2.00-2.30PM IST | 4.30-5.00PM SGT About the speaker: Sandeep Khuperkar, Director and CTO at Ashnik will be heading this session. He is an author, enthusiast and community moderator at opensource.com. He is also member of Open Source Initiative, Linux Foundation and Open Source Consortium Of India.

Docker Ecosystem - Part II - Compose

Mario IC

Building Data Integration and Transformations using Pentaho

Ashnikbiz

Indic threads pune12-accelerating computation in html 5

IndicThreads

Docker Ecosystem: Engine, Compose, Machine, Swarm, Registry

Mario IC

Introduction to docker swarm

Walid Ashraf

Scaling Jenkins with Docker: Swarm, Kubernetes or Mesos?

Carlos Sanchez

The Jenkins platform can be dynamically scaled by using several Docker cluster and orchestration platforms, using containers to run slaves and jobs and also isolating job execution. But which cluster technology should be used? Docker Swarm? Apache Mesos? Kubernetes? How do they compare? All of them can be used to dynamically run jobs inside containers. This talk will cover these main container clusters, outlining the pros and cons of each, the current state of the art of the technologies and Jenkins support.

Continuous Development with Jenkins - Stephen Connolly at PuppetCamp Dublin '12

Puppet

Migración de datos con OpenERP-Kettle

raimonesteve

Scaling Jenkins with Docker and Kubernetes

Carlos Sanchez

Docker is revolutionizing the way people think about applications and deployments. It provides a simple way to run and distribute Linux containers for a variety of use cases, from lightweight virtual machines to complex distributed micro-services architectures. Kubernetes is an open source project to manage a cluster of Linux containers as a single system, managing and running Docker containers across multiple Docker hosts, offering co-location of containers, service discovery and replication control. It was started by Google and now it is supported by Microsoft, RedHat, IBM and Docker Inc amongst others. Jenkins Continuous Integration environment can be dynamically scaled by using the Kubernetes and Docker plugins, using containers to run slaves and jobs, and also isolate job execution.

Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy

Giovanni Toraldo

Tao zhangharishk2

Jenkins Peru Meetup Docker Ecosystem

Mario IC

Elementos ETL - Kettle Pentaho valex_haro

Introduction to GPU ProgrammingChakkrit (Kla) Tantithamthavorn

Load Balancing Apps in Docker Swarm with NGINX

NGINX, Inc.

On-demand webinar recording: http://bit.ly/2mRjk2g Docker and other container technologies continue to gain in popularity. We recently surveyed the broad community of NGINX and NGINX Plus users and found that two-thirds of organizations are either investigating containers, using them in development, or using them in production. Why? Because abstracting your applications from the underlying infrastructure makes developing, distributing, and running software simpler, faster, and more robust than ever before. But when you move from running your app in a development environment to deploying containers in production, you face new challenges – such as how to effectively run and scale an application across multiple hosts with the performance and uptime that your customers demand. The latest Docker release, 1.12, supports multihost container orchestration, which simplifies deployment and management of containers across a cluster of Docker hosts. In a complex environment like this, load balancing plays an essential part in delivering your container-based application with reliability and high performance. Join us in this webinar to learn: * The basic built-in load balancing options available in Docker Swarm Mode * The pros and cons of moving to an advanced load balancer like NGINX * How to integrate NGINX and NGINX Plus with Swarm Mode to provide an advanced load-balancing solution for a cluster with orchestration * How to scale your Docker-based application with Swarm Mode and NGINX Plus

Building a data warehouse with Pentaho and Docker

Wellington Marinho

EDW CENIPA is a opensource project designed to enable analysis of aeronautical incidentes that occured in the brazilian civil aviation. The project uses techniques and BI tools that explore innovative low-cost technologies. Historically, Business Intelligence platforms are expensive and impracticable for small projects. BI projects require specialized skills and high development costs. This work aims to break this barrier.

Viewers also liked (20)

Pentaho bi suite overview presentation

Business Intelligence and Big Data Analytics with Pentaho

Advanced ETL2 Pentaho

NGINX Plus PLATFORM For Flawless Application Delivery

Docker Ecosystem - Part II - Compose

Building Data Integration and Transformations using Pentaho

Indic threads pune12-accelerating computation in html 5

Docker Ecosystem: Engine, Compose, Machine, Swarm, Registry

Introduction to docker swarm

Scaling Jenkins with Docker: Swarm, Kubernetes or Mesos?

Continuous Development with Jenkins - Stephen Connolly at PuppetCamp Dublin '12

Migración de datos con OpenERP-Kettle

Scaling Jenkins with Docker and Kubernetes

Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy

Tao zhang

Jenkins Peru Meetup Docker Ecosystem

Elementos ETL - Kettle Pentaho

Introduction to GPU Programming

Load Balancing Apps in Docker Swarm with NGINX

Building a data warehouse with Pentaho and Docker

Similar to Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle)

Querona Presentation 2018

Synergo!

Any data source becomes an SQL Query with all the power of Apache Spark. Querona is a virtual database that seamlessly connects any data source with Power BI, TARGIT, Qlik, Tableau, Microsoft Excel or others. It lets you build your own universal data model and share it among reporting tools. Querona does not create another copy of your data, unless you want to accelerate your reports and use build-in execution engine created for purpose of Big Data analytics. Just write standard SQL query and let Querona consolidate data on the fly, use one of execution engines and accelerate processing no matter what kind and how many sources you have.

Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...

Hortonworks

Today’s enterprises are challenged with capturing large amounts of data from a number of sources in a variety of formats, and then storing it in a cost-effective, timely manner. With your current data warehouse, this may seem overwhelming. It doesn’t have to be. With a Hadoop-based modern data warehouse, you can overcome these challenges and get meaningful insights from real-time data. Want to learn how? Join experts from Attunity, Hortonworks, and RCG Global Services for a live webinar - where we will be discussing enterprise data warehouse optimization. You will learn how to: •Rebalance your data warehouse by identifying unused data and resource-intensive workloads that can be moved to Hadoop. •Seamlessly integrate your current enterprise data warehouse with a Modern Data Architecture. •Better utilize data assets to reduce costs while realizing more value from your data. •Develop a roadmap for implementing the Hadoop-based Modern Data Architecture and Data Lake.

SQL Server Data Discovery with PowerPivot

Eduardo Castro

sandhya exp resume sandhya chamarthi

Amit Kumar_ResumeAmit Kumar

Pentaho etl-toolSreenivas Kappala

seven steps to dataops @ dataops.rocks conference Oct 2019

DataKitchen

Heavy Metal PowerPivot RemasteredJason Himmelstein

Building Scalable Big Data Infrastructure Using Open Source Software Presenta...

ssuserd3a367

rough-work.pptx

sharpan

Informatica to ODI Migration – What, Why and How | Informatica to Oracle Dat...

Jade Global

Learn about the First and Only Automated Solution for Informatica to Oracle Data Integrator (ODI) conversion Do you want to know: “What” is Informatica vs ODI? “Why” do you need to move to ODI? “How” is the migration from Informatica to ODI possible? Learn how you can achieve up to 90% automated conversion, up to 90% reduced implementation time, up to 50% cost savings and up to 5X productivity gain. Know more please visit: http://informaticatoodi.jadeglobal.com/

Pentaho

teza123

Summer Shorts: Big Data Integration

ibi

Today's organizations contend with more diverse applications, data, and systems than ever before – silos that are often fragmented and difficult to leverage together. iWay Big Data Integrator (BDI) simplifies the creation, management, and use of Hadoop-based data lakes. It provides a modern, native approach to Hadoop-based data integration and management that ensures high levels of capability, compatibility, and flexibility to help your organization. Join us to learn how you can simplify adoption of Apache Hadoop using iWay Big Data Integrator. Learn about our ability to streamline the deployment of ingestion, transformation, and extraction tasks. See the pre-recorded webcast online at: http://www.informationbuilders.com/webevents/online/24427#sthash.J0cRy1PG.dpuf

Apache CarbonData+Spark to realize data convergence and Unified high performa...

Tech Triveni

Challenges in Data Analytics: Different application scenarios need different storage solutions: HBASE is ideal for point query scenarios but unsuitable for multi-dimensional queries. MPP is suitable for data warehouse scenarios but engine and data are coupled together which hampers scalability. OLAP stores used in BI applications perform best for Aggregate queries but full scan queries perform at a sub-optimal performance. Moreover, they are not suitable for real-time analysis. These distinct systems lead to low resource sharing and need different pipelines for data and application management.

Analytics at the Speed of Thought: Actian Express Overview

Actian Corporation

Pentaho Roadmap 2011

Datalytics

SQL Server Konferenz 2014 - SSIS & HDInsightTillmann Eitelberg

Data Platform Architecture Principles and Evaluation Criteria

ScyllaDB

Resume (3)KUNAL YADAV

Modernizing Global Shared Data Analytics Platform and our Alluxio Journey

Alluxio, Inc.

Similar to Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle) (20)

Querona Presentation 2018

Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...

SQL Server Data Discovery with PowerPivot

sandhya exp resume

Amit Kumar_Resume

Pentaho etl-tool

seven steps to dataops @ dataops.rocks conference Oct 2019

Heavy Metal PowerPivot Remastered

Building Scalable Big Data Infrastructure Using Open Source Software Presenta...

rough-work.pptx

Informatica to ODI Migration – What, Why and How | Informatica to Oracle Dat...

Pentaho

Summer Shorts: Big Data Integration

Apache CarbonData+Spark to realize data convergence and Unified high performa...

Analytics at the Speed of Thought: Actian Express Overview

Pentaho Roadmap 2011

SQL Server Konferenz 2014 - SSIS & HDInsight

Data Platform Architecture Principles and Evaluation Criteria

Resume (3)

Modernizing Global Shared Data Analytics Platform and our Alluxio Journey

Recently uploaded

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

Tobias Schneck

As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other? Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.

The Future of Platform Engineering

Jemma Hussein Allen

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Inflectra

In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring. Learn about: • The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks. • Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective. • Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification. • Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process. Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...

Thierry Lestable

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams. Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

UiPath Test Automation using UiPath Test Suite series, part 4

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap. The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies. Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques What will you get from this session? 1. Insights into SAP testing best practices 2. Heatmap utilization for testing 3. Optimization of testing processes 4. Demo Topics covered: Execution from the test manager Orchestrator execution result Defect reporting SAP heatmap example with demo Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

DevOps and Testing slides at DASA Connect

Kari Kakkonen

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Product School

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Ramesh Iyer

In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Product School

Recently uploaded (20)

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

The Future of Platform Engineering

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Assuring Contact Center Experiences for Your Customers With ThousandEyes

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

GraphRAG is All You need? LLM & Knowledge Graph

UiPath Test Automation using UiPath Test Suite series, part 4

DevOps and Testing slides at DASA Connect

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

The Art of the Pitch: WordPress Relationships and Sales

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

PCI PIN Basics Webinar from the Controlcase Team

When stars align: studies in data quality, knowledge graphs, and machine lear...

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

Securing your Kubernetes cluster_ a step-by-step guide to success !

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle)

1. Moving and Transforming Data with Pentaho Data Integration a.k.a. KETTLE

2. Welcome! • Software engineer • rbouman@pentaho.com

3. Pentaho • Business Intelligence & Analytics • Full stack • Open core – GPLv2, Apache 2.0 – Enterprise and OEM licenses • Java-based • Web front-ends

4. The Pentaho Stack • • • • • • • • • • Data Integration / ETL Big Data / NoSQL Data Modeling Reporting OLAP / Analysis Data Visualization Dashboarding Data Mining / Predictive Analysis (Mobile) Delivery Bursting, Scheduling, Self Service

5. Full Stack BI & BA Sources Reports OLAP Blending Visualization Data Warehouse ETL T T T T Models D D Instant Analytics Dashboards D F D Mining D

6. Information: A well-prepared, well-presented meal

7. Extraction: Catching the right data and hauling it in

8. Transformation Disgusting job of validating & cleaning data

9. Loading: Store manageable units for later use

10. Loading: Store manageable units for later use

11. Pentaho Data Integration • Kettle – Extract, Transform, Load – Blending – Instant Analytics • Changing input to desired output

12. Kettle Architecture Data Integration Engine Job Engine Call Job Transformation Engine Transformation Tools and Utilities Launch: Kitchen, Carte Launch: Pan Develop: Spoon Repository (RDBMS) .kjb .ktr

13. Jobs & Transformations • Jobs – Synchronous workflow of job entries (tasks) • Transformations – Stepwise parallel & asynchronous processing of a recordstream • Distributed

14. Sources and Destinations • • • • • • • RDBMS (> 40) NoSQL / Big Data OLAP (Mondrian, Palo, XML/A) Web (REST, SOAP, XML, JSON .) Files (CSV, Fixed, Excel …) ERP (SAP, Salesforce, OpenERP) ...way Too Many To Mention™!

15. Transformations • • • • • • • • String & Date manipulation Data Validation / Business Rules Lookup / Join Calculation, Statistics Cryptography Decisions, Flow control Scripting ...> 150, excluding plugins

16. Demo • NLUUG Program Webpage Input Output dim_room ETL dim_track fact_talk dim_speaker dim_company

17. Demo Transformation

18. Business Model • Open core – Majority – Give is open source and take • Enterprise Edition – Extra features, Support, Services

19. Community • Plugins – Pentaho Marketplace • Code contributions • Applications & Solutions • Tutorials, Support

20. Marketplace

21. Community Meetups

22. Pentaho Software • Community Edition Binaries – community.pentaho.com (release) ci.pentaho.com (development) • Source code – – github.com/pentaho • Enterprise Edition Evaluation – pentaho.com/testdrive

23. Resources • Online documentation – infocenter.pentaho.com – wiki.pentaho.com (manual) (community wiki) • Issue tracker – jira.pentaho.com • Community Support – forums.pentaho.com – Freenode IRC: ##pentaho

24. Books

25. Thank You Join the conversation. You can find us on: blog.pentaho.com @Pentaho Facebook.com/Pentaho Pentaho Business Analytics 25

Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle)

Similar to Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle) (20)

More from Roland Bouman

More from Roland Bouman (12)

Recently uploaded

Recently uploaded (20)

Moving and Transforming Data with Pentaho Data Integration 5.0 CE (aka Kettle)