In this session Alejandro Infanzon, Solutions Engineer, introduces the linear regression and statistical functions that debuted in MariaDB ColumnStore 1.2, and how you can use them to support powerful analytics. He explains how to perform even-more-powerful analytics by writing multi-parameter user-defined functions (UDFs) – also new in MariaDB ColumnStore 1.2.

1. Applying Linear
Regression and
Predictive Analytics
Alejandro Infanzon
Solutions Architect
MariaDB Corporation

2. Topics
● Need for Analytics / SQL
● MariaDB Platform Introduction.
● Demo Statistical Functions
● Demo “Hello World” UDF

3. Why Analytics ?
Extract value form your data
Get the most
value of your
data assets
Improve the
quality of
decision making
Improve
planning and
forecasting
Introduce new
products and
services

4. Analytics Data Challenge
Data must be managed!

5. Types of Analytics
Uncover correlations and patterns
Descriptive Analytics
What happened?
1 Diagnostic Analytics
How or why did it happen?
2
Predictive Analytics
What is likely to happen next?
34
Prescriptive Analytics
What should I do about it?

6. Top Tools Used for Analytics
● Jupyter (Julia, Python and R) notebooks
are popular with Data Scientists.
● SQL is important in data science.
● Prebuilt analytical functions / aggregations
* https://www.kdnuggets.com/2017/05/poll-analytics-data-science-machine-learning-
software-leaders.html

7. MariaDB Platform X3
Why MariaDB Platform X3 ?

8. MariaDB Platform: use cases
Unified Platform
MariaDB Server
Row storage
OLTP
MariaDB Server
HTAP
Row storage Columnar storage
MariaDB Server
Columnar storage
OLAP
Transactions Transactions + Analytics Analytics

9. MariaDB Platform X3 High Level

10. MariaDB Platform X3 Solution
Encompassing applications, tools, database and support services
MariaDB Server
Enterprise analytics
Enterprise high availability Enterprise scalability
Enterprise performance Enterprise integration
Enterprise security
MariaDB connectors
Enterprise tools Enterprise support
- Management
- Monitoring
- Backup/restore
- Technical
- Consultative
- Notifications
- Alerts

11. MariaDB Platform X3: Pluggable Storage
Right storage for different workloads
MariaDB Server
tbl_products tbl_purchases tbl_carts tbl_clicks
InnoDB MyRocks Spider ColumnStore
General purpose Write optimized Sharded Columnar
Product catalog Purchases Shopping carts Clickstream events

12. Jupyter Notebook
Interaction
Literate programing interaction ?
Available analytical functions / aggregations ?

13. The Demo Environment
Data Layer
Docker ColumnStore Container
Application Layer
Connectivity Layer pymysql connector
Python Jupyter DBeaver

14. Demo Tables
Using different storage engines

15. Database Connection
Using PYMSQL module

16. Basic Statistical Functions
Top 10 records from the breeds table

17. Basic Statistical Functions (Cont.)
Using PYMSQL module

18. Using Basic Statistics
Find out extra-small and extra-large dog breeds

19. Advanced Statistical Functions
Top 10 rows of the employees table

20. Advanced Statistical Functions ( Cont. )
Covariance between days since hired and salary

21. Advanced Statistical Functions ( Cont. )
Calculate the correlation between hire date and salary

22. Advanced Statistical Functions ( Cont. )
Calculate Linear Regression between hire date and salary

23. Advanced Statistical Functions ( Cont. )
Plotting the linear regression

24. Custom Analytical
Functions
Can I build my own analytical functions ?

25. User Defined
Functions (UDF)
• Extend MariaDB with a
new functions.
• UDFs Work like a native
(built-in) MariaDB
functions.
Alternative ways
• Modifying and compiling
the server source code.
• Writing a stored function.

26. “Hello World” UDF Example
Four key steps
Write UDF
code
Compile to
Shared Library
Move library
to Plugin
directory
Register,
verify and Run
the UDF

27. “Hello World” UDF Anatomy
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <my_global.h>
#include <my_sys.h>
#include <mysql.h>
#include <ctype.h>
#include <math.h>
Main UDF structure
my_bool radius_init(
UDF_INIT *initid
, UDF_ARGS *args
, char *message)
{
if ((args->arg_count != 1)) {
strcpy(message, "AICA: Message");
return 1;
}
args->arg_type[0] = REAL_RESULT;
return 0;
}
double radius(
UDF_INIT *initid
, UDF_ARGS *args
, char *result
, unsigned long *length
, char *is_null
, char *error)
{
double r = *((double*) (args->args[0]));
double pi = 3.14;
return pi * 2 * (r);
}
void radius_deinit(
UDF_INIT * initid)
{
//nothing
}
Libraries
Function
Initialization
Function
Body
De-allocate
resources

28. Compile and Move
Place .so in the Plugin Directory

29. Register, Verify and Run
Place .so in the Plugin Directory

30. THANK YOU!

31. Challenges Faced:
Take 2+ Days to Load
Data
Unsustainable Oracle
LIcensing Model
Historically would need to
create/manage indexes
to get query performance
Health population data analysis to equip epidemiologist, public health
officers, and insurance providers with evidence based outcome and efficacy
of treatment. The insurance providers further use this insight to measure &
predict treatment costs and population health over several years.
Benefits Realized:
Data Loads now take minutes
Can query 200+ columns with fast performance
Easy administration - no indexes
High Performance Data Visualization with Tableau
Center for Information
Management (CIM)

32. Genus provides
farmers with superior
genetics that enable
them to produce
higher-quality animal
protein more efficiently,
in the form of meat and
milk.
Challenges Faced:
Leveraging Oracle was
Cost Prohibitive
Existing process had
slow data loads
Data Scientist wanted to
use SQL as primary
interface
Didn’t want to have to
provide heavy database
administration
Benefits Realized:
Fast loading of raw data. A few Gigs to 20 Gig data per load
Leverage Known Interfaces:
Easy to use SQL Front End
Python bulk data adapters allows them to directly publish results from
machine learning data models instead of scheduling cpimport jobs
Fast query results
Easy to maintain
Much more affordable cost structure
Genus PLC

33. Industry: Telco
Data: call and text logs
Use case: Mobile app
use analytics
Details:
30 million text and 3
million phone call per day
1.5 billion rows of logs
per day
The text and call volume
rate will continue to grow
InnoDB backend hit the scale limit of 6TB and it requires lot of
performance tuning and index management
Migrated to MariaDB AX
Able to process 24 month - 24TB vs 6 months limitation of InnoDB
Same BI tools and client applications worked with MariaDB AX
seamlessly
Customer Use Case: Pinger