SlideShare a Scribd company logo
Ricardo Pires – Partner & BI Lead Xpand IT
Real Uses Cases
A SET OF INSPIRING USE CASES
USE CASE 1:
ALL TRANSACTIONS,
ONE DASHBOARD
• Dashboard providing a common view across sales transactions
• Multiple roles
• Top management
• Brand managers
• Channel managers
• Requiring to organize data in multiple ways
• Establish dynamic hierarchies based on multiple attributes
DYNAMIC VIEW ACROSS SALES
DYNAMIC HIERARCHIES
Holding
Brand
Channel
Shop
Ʃ
Ʃ
Ʃ
Ʃ ATTR = abc
• 3 Years historical data
• 7,2 billion transactions representing 4,5TB
• Wide group of users spread across the organization
• Intuitive User Interface with a great User Experience
• Detailed visualization
• Row level security
• Maximum dashboard load time 5s
CHALLENGES
THE SOLUTION
HDFS Hive Impala
Pentaho Data Integration (PDI)
PDI
HBase
Web
Application
Hadoop
• Impala on Cloudera Hadoop can be used as an interactive data
base
• Hadoop distributed nature allows implementing used cases that
wouldn’t be viable on other technologies
• We went from 7 days of data to 3 years
• Pentaho Data Integration implements and orchestrates the whole
ETL process, making it much easier
• From traditional data sources to summarized data on Hadoop
KEY TAKEAWAYS
USE CASE 2:
LOADING THE DATA LAKE
• Data lake goal is to make data available on a centralized location
• Requires dealing with
• Wide set of sources
• Disparate technologies
• In this case it is a repetitive batch loading process
DATA INGESTION
THE SMART SOLUTION
Configure
Metadata
Repository
Ingestion Engine
based on
Templates
Use Hadoop as
Data Repository
METADATA DRIVEN INGESTION
ARCHITECTURE
HDFS
Web UI
HadoopAny Datasource
PDI
PDI
PDI
{REST}
Ingestion Engine
Hive
• Pentaho Data Integration flexibility is a great match for Hadoop
semi-structured nature
• Cloudera Hadoop can be easily used to store data and make it
immediately available through a SQL interface
• Patterns and well defined workflows are essential to data
governance
KEY TAKEAWAYS
USE CASE 3:
FOSTERING TRANSPARENCY
• Government agencies have long collected data but that doesn’t
mean it can easily be perceived by citizens
• Challenge
• Create an intuitive UI to represent more than 100 KPIs across 308
municipalities
• Become a standard in terms of transparency
GOVERNMENT CHALLENGE
Architecture
BA
SERVER
Public Data
Service
PDI
ETL Web Application
• Pentaho Business Analytics is a comprehensive suite
• Pentaho Server components are really flexible and extensible
allowing creating custom UIs such as:
• Analytics portals
• Embed on existing products
KEY TAKEAWAYS
THANK YOU

More Related Content

What's hot

Pentaho Data Integration Introduction
Pentaho Data Integration IntroductionPentaho Data Integration Introduction
Pentaho Data Integration Introduction
mattcasters
 
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
Marcus Vinicius Miguel Pedro
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
Arnab Mitra
 
Architecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterArchitecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres Cluster
Ashnikbiz
 
YugaByte DB Internals - Storage Engine and Transactions
YugaByte DB Internals - Storage Engine and Transactions YugaByte DB Internals - Storage Engine and Transactions
YugaByte DB Internals - Storage Engine and Transactions
Yugabyte
 
Tag based policies using Apache Atlas and Ranger
Tag based policies using Apache Atlas and RangerTag based policies using Apache Atlas and Ranger
Tag based policies using Apache Atlas and Ranger
Vimal Sharma
 
Data Engineer's Lunch #54: dbt and Spark
Data Engineer's Lunch #54: dbt and SparkData Engineer's Lunch #54: dbt and Spark
Data Engineer's Lunch #54: dbt and Spark
Anant Corporation
 
RocksDB compaction
RocksDB compactionRocksDB compaction
RocksDB compaction
MIJIN AN
 
Hive on spark is blazing fast or is it final
Hive on spark is blazing fast or is it finalHive on spark is blazing fast or is it final
Hive on spark is blazing fast or is it final
Hortonworks
 
White Paper - Data Warehouse Documentation Roadmap
White Paper -  Data Warehouse Documentation RoadmapWhite Paper -  Data Warehouse Documentation Roadmap
White Paper - Data Warehouse Documentation Roadmap
David Walker
 
Impala presentation
Impala presentationImpala presentation
Impala presentation
trihug
 
Introduction to Git and Github
Introduction to Git and GithubIntroduction to Git and Github
Introduction to Git and Github
Roland Emmanuel Salunga
 
REST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using RESTREST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using REST
Christian Gohmann
 
Making Data Timelier and More Reliable with Lakehouse Technology
Making Data Timelier and More Reliable with Lakehouse TechnologyMaking Data Timelier and More Reliable with Lakehouse Technology
Making Data Timelier and More Reliable with Lakehouse Technology
Matei Zaharia
 
Machine Learning in the IoT with Apache NiFi
Machine Learning in the IoT with Apache NiFiMachine Learning in the IoT with Apache NiFi
Machine Learning in the IoT with Apache NiFi
DataWorks Summit/Hadoop Summit
 
Enterprise guide to building a Data Mesh
Enterprise guide to building a Data MeshEnterprise guide to building a Data Mesh
Enterprise guide to building a Data Mesh
Sion Smith
 
Apache Iceberg Presentation for the St. Louis Big Data IDEA
Apache Iceberg Presentation for the St. Louis Big Data IDEAApache Iceberg Presentation for the St. Louis Big Data IDEA
Apache Iceberg Presentation for the St. Louis Big Data IDEA
Adam Doyle
 
Business Data Lake Best Practices
Business Data Lake Best PracticesBusiness Data Lake Best Practices
Business Data Lake Best Practices
Capgemini
 
Git 101 for Beginners
Git 101 for Beginners Git 101 for Beginners
Git 101 for Beginners
Anurag Upadhaya
 
Modern Data Architecture
Modern Data ArchitectureModern Data Architecture
Modern Data Architecture
Alexey Grishchenko
 

What's hot (20)

Pentaho Data Integration Introduction
Pentaho Data Integration IntroductionPentaho Data Integration Introduction
Pentaho Data Integration Introduction
 
2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs2020 - OCI Key Concepts for Oracle DBAs
2020 - OCI Key Concepts for Oracle DBAs
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Architecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterArchitecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres Cluster
 
YugaByte DB Internals - Storage Engine and Transactions
YugaByte DB Internals - Storage Engine and Transactions YugaByte DB Internals - Storage Engine and Transactions
YugaByte DB Internals - Storage Engine and Transactions
 
Tag based policies using Apache Atlas and Ranger
Tag based policies using Apache Atlas and RangerTag based policies using Apache Atlas and Ranger
Tag based policies using Apache Atlas and Ranger
 
Data Engineer's Lunch #54: dbt and Spark
Data Engineer's Lunch #54: dbt and SparkData Engineer's Lunch #54: dbt and Spark
Data Engineer's Lunch #54: dbt and Spark
 
RocksDB compaction
RocksDB compactionRocksDB compaction
RocksDB compaction
 
Hive on spark is blazing fast or is it final
Hive on spark is blazing fast or is it finalHive on spark is blazing fast or is it final
Hive on spark is blazing fast or is it final
 
White Paper - Data Warehouse Documentation Roadmap
White Paper -  Data Warehouse Documentation RoadmapWhite Paper -  Data Warehouse Documentation Roadmap
White Paper - Data Warehouse Documentation Roadmap
 
Impala presentation
Impala presentationImpala presentation
Impala presentation
 
Introduction to Git and Github
Introduction to Git and GithubIntroduction to Git and Github
Introduction to Git and Github
 
REST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using RESTREST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using REST
 
Making Data Timelier and More Reliable with Lakehouse Technology
Making Data Timelier and More Reliable with Lakehouse TechnologyMaking Data Timelier and More Reliable with Lakehouse Technology
Making Data Timelier and More Reliable with Lakehouse Technology
 
Machine Learning in the IoT with Apache NiFi
Machine Learning in the IoT with Apache NiFiMachine Learning in the IoT with Apache NiFi
Machine Learning in the IoT with Apache NiFi
 
Enterprise guide to building a Data Mesh
Enterprise guide to building a Data MeshEnterprise guide to building a Data Mesh
Enterprise guide to building a Data Mesh
 
Apache Iceberg Presentation for the St. Louis Big Data IDEA
Apache Iceberg Presentation for the St. Louis Big Data IDEAApache Iceberg Presentation for the St. Louis Big Data IDEA
Apache Iceberg Presentation for the St. Louis Big Data IDEA
 
Business Data Lake Best Practices
Business Data Lake Best PracticesBusiness Data Lake Best Practices
Business Data Lake Best Practices
 
Git 101 for Beginners
Git 101 for Beginners Git 101 for Beginners
Git 101 for Beginners
 
Modern Data Architecture
Modern Data ArchitectureModern Data Architecture
Modern Data Architecture
 

Similar to Real Use Cases - Pentaho & Big Data Ecosystem

Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
DataWorks Summit/Hadoop Summit
 
Skilwise Big data
Skilwise Big dataSkilwise Big data
Skilwise Big data
Skillwise Group
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2
Skillwise Group
 
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
MapR Technologies
 
Hadoop and Your Enterprise Data Warehouse
Hadoop and Your Enterprise Data WarehouseHadoop and Your Enterprise Data Warehouse
Hadoop and Your Enterprise Data Warehouse
Edgar Alejandro Villegas
 
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio JourneyModernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Alluxio, Inc.
 
Hitachi Data Systems Hadoop Solution
Hitachi Data Systems Hadoop SolutionHitachi Data Systems Hadoop Solution
Hitachi Data Systems Hadoop Solution
Hitachi Vantara
 
Moving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from PivotalMoving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from Pivotal
VMware Tanzu Korea
 
Next Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data WarehouseNext Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data Warehouse
Denodo
 
Accelerating Data Warehouse Modernization
Accelerating Data Warehouse ModernizationAccelerating Data Warehouse Modernization
Accelerating Data Warehouse Modernization
DataWorks Summit/Hadoop Summit
 
Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which
DataWorks Summit
 
OPEN'17_4_Postgres: The Centerpiece for Modernising IT Infrastructures
OPEN'17_4_Postgres: The Centerpiece for Modernising IT InfrastructuresOPEN'17_4_Postgres: The Centerpiece for Modernising IT Infrastructures
OPEN'17_4_Postgres: The Centerpiece for Modernising IT Infrastructures
Kangaroot
 
Transforming Business in a Digital Era with Big Data and Microsoft
Transforming Business in a Digital Era with Big Data and MicrosoftTransforming Business in a Digital Era with Big Data and Microsoft
Transforming Business in a Digital Era with Big Data and Microsoft
Perficient, Inc.
 
Logical Data Fabric and Data Mesh – Driving Business Outcomes
Logical Data Fabric and Data Mesh – Driving Business OutcomesLogical Data Fabric and Data Mesh – Driving Business Outcomes
Logical Data Fabric and Data Mesh – Driving Business Outcomes
Denodo
 
IBM - Introduction to Cloudant
IBM - Introduction to CloudantIBM - Introduction to Cloudant
IBM - Introduction to Cloudant
Francisco González Jiménez
 
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part20812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
Raul Chong
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
Antonios Chatzipavlis
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
Antonios Chatzipavlis
 
Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)
Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)
Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)
Denodo
 
Webinar | Data Management for Hybrid and Multi-Cloud: A Four-Step Journey
Webinar | Data Management for Hybrid and Multi-Cloud: A Four-Step JourneyWebinar | Data Management for Hybrid and Multi-Cloud: A Four-Step Journey
Webinar | Data Management for Hybrid and Multi-Cloud: A Four-Step Journey
DataStax
 

Similar to Real Use Cases - Pentaho & Big Data Ecosystem (20)

Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
 
Skilwise Big data
Skilwise Big dataSkilwise Big data
Skilwise Big data
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2
 
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
 
Hadoop and Your Enterprise Data Warehouse
Hadoop and Your Enterprise Data WarehouseHadoop and Your Enterprise Data Warehouse
Hadoop and Your Enterprise Data Warehouse
 
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio JourneyModernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
 
Hitachi Data Systems Hadoop Solution
Hitachi Data Systems Hadoop SolutionHitachi Data Systems Hadoop Solution
Hitachi Data Systems Hadoop Solution
 
Moving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from PivotalMoving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from Pivotal
 
Next Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data WarehouseNext Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data Warehouse
 
Accelerating Data Warehouse Modernization
Accelerating Data Warehouse ModernizationAccelerating Data Warehouse Modernization
Accelerating Data Warehouse Modernization
 
Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which
 
OPEN'17_4_Postgres: The Centerpiece for Modernising IT Infrastructures
OPEN'17_4_Postgres: The Centerpiece for Modernising IT InfrastructuresOPEN'17_4_Postgres: The Centerpiece for Modernising IT Infrastructures
OPEN'17_4_Postgres: The Centerpiece for Modernising IT Infrastructures
 
Transforming Business in a Digital Era with Big Data and Microsoft
Transforming Business in a Digital Era with Big Data and MicrosoftTransforming Business in a Digital Era with Big Data and Microsoft
Transforming Business in a Digital Era with Big Data and Microsoft
 
Logical Data Fabric and Data Mesh – Driving Business Outcomes
Logical Data Fabric and Data Mesh – Driving Business OutcomesLogical Data Fabric and Data Mesh – Driving Business Outcomes
Logical Data Fabric and Data Mesh – Driving Business Outcomes
 
IBM - Introduction to Cloudant
IBM - Introduction to CloudantIBM - Introduction to Cloudant
IBM - Introduction to Cloudant
 
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part20812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)
Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)
Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)
 
Webinar | Data Management for Hybrid and Multi-Cloud: A Four-Step Journey
Webinar | Data Management for Hybrid and Multi-Cloud: A Four-Step JourneyWebinar | Data Management for Hybrid and Multi-Cloud: A Four-Step Journey
Webinar | Data Management for Hybrid and Multi-Cloud: A Four-Step Journey
 

More from Xpand IT

Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018
Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018
Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018
Xpand IT
 
Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017
Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017
Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017
Xpand IT
 
Xporter for Jira - Overview
Xporter for Jira - OverviewXporter for Jira - Overview
Xporter for Jira - Overview
Xpand IT
 
Xray for Jira - How to automate your QA process
Xray for Jira - How to automate your QA processXray for Jira - How to automate your QA process
Xray for Jira - How to automate your QA process
Xpand IT
 
Xpand Addons - Addon Discovery Day 2017
Xpand Addons - Addon Discovery Day 2017Xpand Addons - Addon Discovery Day 2017
Xpand Addons - Addon Discovery Day 2017
Xpand IT
 
Xray for Jira 3.0 - What's New?
Xray for Jira 3.0 - What's New?Xray for Jira 3.0 - What's New?
Xray for Jira 3.0 - What's New?
Xpand IT
 
Xray for Jira - Overview
Xray for Jira - OverviewXray for Jira - Overview
Xray for Jira - Overview
Xpand IT
 
Xporter for Jira - Advanced topics
Xporter for Jira  - Advanced topicsXporter for Jira  - Advanced topics
Xporter for Jira - Advanced topics
Xpand IT
 
Keynote - Xamarin Experience London 2017
Keynote - Xamarin Experience London 2017 Keynote - Xamarin Experience London 2017
Keynote - Xamarin Experience London 2017
Xpand IT
 
Welcome & Introduction – Xamarin Experience London 2017
Welcome & Introduction – Xamarin Experience London 2017 Welcome & Introduction – Xamarin Experience London 2017
Welcome & Introduction – Xamarin Experience London 2017
Xpand IT
 
Gathering Customer Insights with Sitecore - Xamarin Experience London 2017
Gathering Customer Insights with Sitecore - Xamarin Experience London 2017Gathering Customer Insights with Sitecore - Xamarin Experience London 2017
Gathering Customer Insights with Sitecore - Xamarin Experience London 2017
Xpand IT
 
Why Speed Matters in Mobile Apps – Xamarin Experience London 2017
Why Speed Matters in Mobile Apps – Xamarin Experience London 2017Why Speed Matters in Mobile Apps – Xamarin Experience London 2017
Why Speed Matters in Mobile Apps – Xamarin Experience London 2017
Xpand IT
 
Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...
Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...
Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...
Xpand IT
 
Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...
Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...
Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...
Xpand IT
 
The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017
The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017
The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017
Xpand IT
 
Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017
Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017
Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017
Xpand IT
 
Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017
Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017
Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017
Xpand IT
 
How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...
How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...
How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...
Xpand IT
 
Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017
Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017 Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017
Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017
Xpand IT
 
The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017
The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017 The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017
The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017
Xpand IT
 

More from Xpand IT (20)

Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018
Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018
Xray & Xporter were in Austria: Jira & Confluence Solutions Day 2018
 
Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017
Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017
Using Xamarin for your Mobile+ Apps – Xamarin Experience London 2017
 
Xporter for Jira - Overview
Xporter for Jira - OverviewXporter for Jira - Overview
Xporter for Jira - Overview
 
Xray for Jira - How to automate your QA process
Xray for Jira - How to automate your QA processXray for Jira - How to automate your QA process
Xray for Jira - How to automate your QA process
 
Xpand Addons - Addon Discovery Day 2017
Xpand Addons - Addon Discovery Day 2017Xpand Addons - Addon Discovery Day 2017
Xpand Addons - Addon Discovery Day 2017
 
Xray for Jira 3.0 - What's New?
Xray for Jira 3.0 - What's New?Xray for Jira 3.0 - What's New?
Xray for Jira 3.0 - What's New?
 
Xray for Jira - Overview
Xray for Jira - OverviewXray for Jira - Overview
Xray for Jira - Overview
 
Xporter for Jira - Advanced topics
Xporter for Jira  - Advanced topicsXporter for Jira  - Advanced topics
Xporter for Jira - Advanced topics
 
Keynote - Xamarin Experience London 2017
Keynote - Xamarin Experience London 2017 Keynote - Xamarin Experience London 2017
Keynote - Xamarin Experience London 2017
 
Welcome & Introduction – Xamarin Experience London 2017
Welcome & Introduction – Xamarin Experience London 2017 Welcome & Introduction – Xamarin Experience London 2017
Welcome & Introduction – Xamarin Experience London 2017
 
Gathering Customer Insights with Sitecore - Xamarin Experience London 2017
Gathering Customer Insights with Sitecore - Xamarin Experience London 2017Gathering Customer Insights with Sitecore - Xamarin Experience London 2017
Gathering Customer Insights with Sitecore - Xamarin Experience London 2017
 
Why Speed Matters in Mobile Apps – Xamarin Experience London 2017
Why Speed Matters in Mobile Apps – Xamarin Experience London 2017Why Speed Matters in Mobile Apps – Xamarin Experience London 2017
Why Speed Matters in Mobile Apps – Xamarin Experience London 2017
 
Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...
Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...
Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experienc...
 
Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...
Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...
Atlassian Tools in Practice: A Customer Success Story – Xpand IT & Atlassian ...
 
The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017
The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017
The Secret Sauce of Successful Teams - Xpand IT & Atlassian JAM Sessions 2017
 
Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017
Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017
Quality Assurance Made Easy in JIRA - Xpand IT & Atlassian JAM Sessions 2017
 
Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017
Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017
Improved Reporting with JIRA Add-ons - Xpand IT & Atlassian JAM Sessions 2017
 
How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...
How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...
How our Team Collaborates with Atlassian Tools - Xpand IT & Atlassian JAM Ses...
 
Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017
Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017 Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017
Welcome & Introduction - Xpand IT & Atlassian JAM Sessions 2017
 
The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017
The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017 The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017
The Real World with OpenShift - Red Hat DevOps & Microservices Conference 2017
 

Recently uploaded

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 

Recently uploaded (20)

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 

Real Use Cases - Pentaho & Big Data Ecosystem

  • 1. Ricardo Pires – Partner & BI Lead Xpand IT Real Uses Cases
  • 2. A SET OF INSPIRING USE CASES
  • 3. USE CASE 1: ALL TRANSACTIONS, ONE DASHBOARD
  • 4. • Dashboard providing a common view across sales transactions • Multiple roles • Top management • Brand managers • Channel managers • Requiring to organize data in multiple ways • Establish dynamic hierarchies based on multiple attributes DYNAMIC VIEW ACROSS SALES
  • 6. • 3 Years historical data • 7,2 billion transactions representing 4,5TB • Wide group of users spread across the organization • Intuitive User Interface with a great User Experience • Detailed visualization • Row level security • Maximum dashboard load time 5s CHALLENGES
  • 7. THE SOLUTION HDFS Hive Impala Pentaho Data Integration (PDI) PDI HBase Web Application Hadoop
  • 8.
  • 9.
  • 10.
  • 11.
  • 12. • Impala on Cloudera Hadoop can be used as an interactive data base • Hadoop distributed nature allows implementing used cases that wouldn’t be viable on other technologies • We went from 7 days of data to 3 years • Pentaho Data Integration implements and orchestrates the whole ETL process, making it much easier • From traditional data sources to summarized data on Hadoop KEY TAKEAWAYS
  • 13. USE CASE 2: LOADING THE DATA LAKE
  • 14. • Data lake goal is to make data available on a centralized location • Requires dealing with • Wide set of sources • Disparate technologies • In this case it is a repetitive batch loading process DATA INGESTION
  • 15. THE SMART SOLUTION Configure Metadata Repository Ingestion Engine based on Templates Use Hadoop as Data Repository METADATA DRIVEN INGESTION
  • 17. • Pentaho Data Integration flexibility is a great match for Hadoop semi-structured nature • Cloudera Hadoop can be easily used to store data and make it immediately available through a SQL interface • Patterns and well defined workflows are essential to data governance KEY TAKEAWAYS
  • 18. USE CASE 3: FOSTERING TRANSPARENCY
  • 19. • Government agencies have long collected data but that doesn’t mean it can easily be perceived by citizens • Challenge • Create an intuitive UI to represent more than 100 KPIs across 308 municipalities • Become a standard in terms of transparency GOVERNMENT CHALLENGE
  • 21.
  • 22.
  • 23.
  • 24. • Pentaho Business Analytics is a comprehensive suite • Pentaho Server components are really flexible and extensible allowing creating custom UIs such as: • Analytics portals • Embed on existing products KEY TAKEAWAYS

Editor's Notes

  1. Goal is: - to let know examples of what we have been doing - inspire you to use these technologies
  2. Structure was static Last 7 days to last 3 years
  3. Mutliple levels that defined drill down path Multiple elements where each has a criteria establishing the rows to aggregate Sales are sum on each element Multiple hierarchies like this can be created and processed overnigh
  4. 3 main components: PDI + Hadoop + Web App Sqoop from Oracle Process on Hive (formulas, pre aggregation) using conf from HBase Impala stores end result Data is summarized as much as possible allowing each chart is able to be visualized using only a couple of rows which are filtered based on security criteria
  5. Zoom, Pan e Play
  6. Single workflow/pattern across all data sources Promote reusability -> opposite to typical ETL Create a metadata repository Describes sources, destinations and simple processes required to ingest the data, can be done with automatic profiling Implement the ingestion process with PDI Flexible tool with meta data injection capabilities Open standards allowing creating transformations on the fly Use Hadoop as the data repository File system based and thus very flexible Additional layers can be placed on top to access data in multiple ways
  7. 3 main components: PDI + Hadoop + Web App Sqoop from Oracle Process on Hive (formulas, pre aggregation) using conf from HBase Impala stores end result
  8. can be easily understood by anyone be “cosy” and attractive