In this session Satoru Goto, Solutions Engineer at MariaDB, shows how the Pentaho connector for MariaDB ColumnStore can be used for both BI/reporting on MariaDB ColumnStore as well as loading data into MariaDB ColumnStore.

1. Using Pentaho & Tableau
with MariaDB ColumnStore
GOTO Satoru
Customer Solutions Engineer
MariaDB Corporation

2. Note
● we have ≈ 5 minutes for Q&A at the end of
the session
○ In order to get through all slides

3. Agenda
● Motivation
● What is MariaDB ColumnStore
● What is Tableau
● Analytics with Tableau & ColumnStore
● What is Pantaho
● MariaDB ColumnStore bulk load plugin for Pentaho Data
Integration(PDI)
● Using Pentaho EE Analyzer with MariaDB ColumnStore

4. Motivation

5. Motivation to use Tableau/Pentaho Analyzer
with MariaDB ColumnStore
● MariaDB Corp. do not provide GUI tool
● For interactive analytics, GUI front end needed

6. Motivation of MariaDB ColumnStore bulk
load plugin for PDI on Windows
● ColumnStore Server only supports Linux
● end BI users are not familiar with Linux, enterprise
end users use Windows
● GUI bulk loader needed

7. What is
MariaDB ColumnStore

8. MariaDB ColumnStore architecture
Columnar Distributed Data Storage
Local Storage | SAN | NAS | EBS | GlusterFS
BI Tool SQL Client Custom
Big Data App
Application
MariaDB
SQL Front
End - UM
Distributed
Query
Engine - PM
Data Storage

9. MariaDB ColumnStore architecture
• Massively parallel
architecture (MPP)
– Linear scalability
• Horizontal scaling
– Add new data nodes
as your data grows
– Continue read queries
when adding new nodes
Shared-Nothing Distributed Data Storage
Highly Compressed by default
User
Module
(UM)
Performance
Module
(PM)
Data Storage

10. Test environment

11. Test environment
Software OS
MariaDB ColumnStore 1.2.2 CentOS 7
MariaDB Connector/J 2.3.0 OS independent
MariaDB Connector/ODBC 3.0.8 Microsoft Windows 10 Pro
Tableau Desktop 2019.1 Microsoft Windows 10 Pro
MariaDB Pentaho Kettle bulk loader plugin 1.2.2 Microsoft Windows 10 Pro
Pentaho EE 8.2 (PDI included) Microsoft Windows 10 Pro

12. What is Tableau

13. What is Tableau Desktop
● Widely used interactive data visualization tool for
BI(Business Intelligence) / Analytics
● Tableau Desktop 2019.1 now officially supports
MariaDB Connector/ODBC

14. Place Image Here
Analytics with
Tableau Desktop
2019.1 & MariaDB
ColumnStore

15. Sample data set
● https://github.com/mariadb-corporation/mariadb-columnstore-samples
● ‘Flights’ sample
○ on-time performance dataset for US based flights by US
Department of Transportation

16. Connect to MariaDB ColumnStore w/
Tableau 2019.1
● For the 1st time, you
need to download
and install MariaDB
Connector/ODBC
disabled

17. Download MariaDB Connector/ODBC
https://mariadb.com/downloads/#connectors

18. Install MariaDB Connector/ODBC
mariadb-connector-odbc-3.0.8-
win64.msi

19. Connect to ColumnStore via Connector/ODBC
● Restart of Tableau is
needed
● After restart of Tableau
Desktop 2019.1, you can
select MariaDB Data
Source

20. Connect to ColumnStore via Connector/ODBC
● “Sign In” button will
be enabled

21. Connect to ColumnStore via Connector/ODBC
● Select database
● Drag & drop tables

22. Analytics with Tableau: JOINs
flights.carrier =
airlines.iata_code
flights.dest =
airports.iata_code
Sample data set : flights

23. Analytics with Tableau: average delay / airline
• Rows:
airline
• Marks:
AVG(Arr Delay)

24. Analytics with Tableau: average delay / airline

25. What is Pentaho

26. What is Pentaho
● Developed by Hitachi Vantara
(Pentaho Corporation was acquired by Hitachi in 2015)
● Pentaho Server : Business Analytics Platform
● PDI (Pentaho Data Integration) : ETL engine
○ a.k.a. Kettle
https://www.hitachivantara.com/go/pentaho.html

27. Install
Pentaho EE 8.2

28. Download Pentaho EE 8.2
from Hitachi Vantara
Click to install

29. Download MariaDB Connector/J 2.3.0
Pentaho :
Java
application

30. Copy MariaDB Connector/J
Copy mariadb-java-client-2.3.0.jar to
C:Pentahodesign-toolsdata-integrationlib

31. Download Pentaho Kettle bulk loader plugin

32. Extract Pentaho Kettle bulk loader plugin
Unzip mariadb-columnstore-kettle-bulk-exporter-
plugin-1.2.2-1-x64-win.zip to PDI 8.2 plugins folder

33. Windows 10 dependencies
● Visual C++ Redistributable for Visual Studio 2015 (x64)
is required to use the Bulk Write SDK
https://github.com/mariadb-corporation/mariadb-columnstore-data-
adapters/tree/master/kettle-columnstore-bulk-exporter-plugin

34. CSV bulk load into ColumnStore
with PDI 8.2 Kettle plugin

35. Start Pentaho Data Generation(PDI/Spoon)

36. Bulk load Kettle plugin : CSV file input step
● drag “CSV file input”
step into to the canvas
● Click “CSV file input”
https://help.pentaho.com/Documentation/8.2/Products/Data_Integration/Transformati
on_Step_Reference/CSV_File_Input

37. Bulk load Kettle plugin : CSV file input step
● Recommended to
have “header row”
in CSV
(for fields mapping)
● Otherwise you need
to map all fields by
hand
● Click “Get Fields”

38. Bulk load Kettle plugin : MCS Bulk loader step
canvas

39. Bulk load Kettle plugin : connect 2 steps

40. Bulk load Kettle plugin : MCS Bulk loader step

41. Bulk load Kettle plugin : DB connection

42. Bulk load Kettle plugin : Columnstore.xml
Replace all 127.0.0.1 to IP address of ColumnStore server in
Columnstore.xml :
<?xml version="1.0" encoding="UTF-8"?>
<!-- $Id: Columnstore.xml 3713 2013-08-16 20:04:47Z wweeks $ -->
<!-- WARNING: Do not make changes to this file unless directed to do so by
MariaDB service engineers. Incorrect settings can render your system
unusable and will require a service call to correct.
-->
<Columnstore Version="V1.0.0">
<ExeMgr1>
<IPAddr>192.168.2.12</IPAddr>
<Port>8601</Port>
<Module>pm1</Module>
</ExeMgr1>

43. Bulk load Kettle plugin : fields mapping
• Choose “Field
mapping” tab
• Click “Custom
mapping”

44. Bulk load Kettle plugin : fields mapping

45. Bulk load Kettle plugin : Run Transformation

46. Bulk load Kettle plugin : bulk load performance
445,827 rows
loaded
in 52.8 sec.

47. Import InnoDB table data into
ColumnStore w/ PDI Kettle plugin

48. Import from MariaDB Server InnoDB table
● Import with
‘Table input’
step

49. Import from MariaDB Server InnoDB table
● DB connection
● Connection
type: MariaDB
● Acess:
Native(JDBC)

50. Import from MariaDB Server InnoDB table

51. Place Image Here
Using Pentaho
Server w/
ColumnStore
via MariaDB Connector/J

52. Install MariaDB Connector/J to
Pentaho EE 8.2

53. Login to Pentaho User Console
● Browse http://localhost:8080
● Select “Login as an Evaluator”
● GO

54. DB connection setting w/o MariaDB Connector/J
cannot connect to MariaDB ColumnStore server
No MariaDB
No MySQL
Database Type

55. Install MariaDB Connector/J
Copy mariadb-java-client-2.3.0.jar into
C:Pentahoserverpentaho-servertomcatlib

56. Restart Pentaho server
compmgmt.msc
restart Pentaho Server
Re-login to Pentaho User
Console

57. Pentaho EE data source and
connection setting

58. Sample data set
● https://github.com/mariadb-corporation/mariadb-columnstore-samples
● ‘Flights’ sample
○ US Department of Transportation on-time performance
dataset for US based flights

59. Manage Data Sources
● Pentaho User Console
● Select
Manage Data Sources

60. Manage Data Sources
● Click
New Data Source

61. Manage Data Sources
● Source Type:
Database Tables(s)
● Click + icon to add new source

62. Manage Data Sources : connection setting
● Now MariaDB can
be selected
● Click Test to test
connection

63. Manage Data Sources
● Select the new
connection
(MCS_flights)
● Click Next

64. Manage Data Sources : table selection

65. Manage Data Sources : fact table selection
● Fact Table:
flights.flights
● Click Next

66. Manage Data Sources: tables joins

67. Place Image Here
Pentaho EE
Analysis Report w/
ColumnStore

68. Pentaho EE Analysis Report
● Create New -
Analysis Report

69. Analysis Report : select Data Source
● Select created
ColumnStore
Data Source
● Click OK

70. Analysis Report

71. Analysis Report : average arrival delay

72. Analysis Report : average arrival delay
● Drag ‘Airline’ to
‘Rows’
● Drag ‘Arr Delay’ to
‘Measures’

73. Analysis Report : average arrival delay table
● Switch to Chart
format by clicking
● or Ctrl+Alt+C

74. Analysis Report : average arrival delay chart

75. Summary

76. Summary
● Interactive Analytics using Tableau Desktop
connecting to ColumnStore via MariaDB
Connector/ODBC
● Data bulk load to MariaDB ColumnStore using PDI
Kettle plugin
● Interactive Analytics using Pentaho EE connecting
to ColumnStore via MariaDB Connector/J

77. Thank you