Dive into the world of Server-Side Extensions with Qlik, exploring examples and architectures with Python and R. This session includes examples of sentiment analysis, time-series forecasting, churn predictions, real-time routing, and much more. Whether you are a data scientist or a data analyst, this session will be useful for both sides of the house.

1. Advanced Analytics
Integration with
Python & R
Feb 26th, 2019
Daniel Pilla
Sr. Enterprise Architect

2. 2
Introduction

3. 3
Common Use Cases
Manufacturing & High Tech
• Demand Forecasting
• Quality & Machine Breakdown
• Warranty Analytics
Healthcare
• Patient Admission Forecasting
• Readmission Analysis
• Clinical Outcomes
Communications
• Customer Segmentation/Churn
• Quality/Process Control
• Next Best Offer/Action
Financial Services
• Credit Risk Analysis
• Loan Default
• Fraud Detection
Public Sector
• Crime Prevention
• Transportation Optimization
• Recidivism
Retail & Consumer
• Market Basket Analysis
• Retail Store Selection
• Product Recommendation
Energy & Utilities
• Customer Churn
• Proactive Failure
• Peak Demand
Life Sciences
• Territory Alignment
• Campaign Management
• Drug Discovery & Effectiveness

4. 4
Data Acquisition
Data Preparation
Data Selection
Model Selection
Result Generation
Result Presentation
Action Steps
Model Creation
Business AnalystData Scientist Business User
Results Interpretation
The old way

5. 5
Model Creation
Governance & Oversight
Data Selection
Basic Parameterization
Result Generation
Result Presentation
Action Steps
Data Acquisition
Data Preparation
Advanced Parameterization
Automated Model Selection
Roles are changing
Business AnalystData Scientist Business User

6. 6
Advanced
Analytics Tracks

7. 7
Advanced Analytics Tracks
Native Functions Augmented
Intelligence
Existing functions within
Qlik’s engine
Cognitive engine that enhances
the user experience
Server-Side
Extensions
Integration with any third-party
calculation engine

8. 8
Cognitive Engine

9. 9
The Qlik Cognitive Engine
Augmenting user intuition with machine intelligence
• A new augmented intelligence engine in
Qlik Sense
• Supports insight suggestions and
automation
• Leverages Qlik Associative Engine
• Machine learning for improved
recommendations over time
• Will be available via APIs

10. 10
Accelerated Self-Service
• New set of capabilities to speed users from
data to insight
- Smart data prep
- Automated chart generation
- Alternative chart suggestions
• Speeds creation of initial visualizations,
drives expanded self-service

11. 11
Insight Advisor
• Auto-generates and suggests the best
insights for users to explore further
• Performs advanced analysis to generate
and recommend insights
- Based on the overall data set
- Based on dimensions
- Based on search criteria
• Auto-generates visualizations and
analytics
• Allows users to add auto-generated
insights directly to sheets
Available
Today

12. 12
Native Functionality &
Server-Side Extensions

13. 13
Native & Third Party Capabilities
• Native capabilities include statistical functions, interactive visualizations, and scripting for many
advanced analytics use cases
• Server-Side Extensions offer engine-level sharing between Qlik Sense and third party tools such as
R and Python, for incorporating advanced calculation and machine learning into analyses
Corporate
Reporting
Guided
Analytics
Ad-hoc
Analysis
Data
Visualization
Advanced
Calculations
Statistics/
Algorithms
Modeling /
Mining
Scenario
Analysis
+ Server-Side Extensions

14. 14
Server-Side Extensions

15. 15
Server-Side Extensions
• Direct integration with 3rd party advanced analytics
engines through server-side extension APIs
• Allows data to be directly exchanged between the QIX
engine and external tools during analysis
- Leverages Qlik’s Associative Model to pass relevant data
based on user context
• Full integration with Qlik Sense expressions and libraries
• Connectors can be built for any external engines
• Open source connectors to be made available by Qlik for
R and Python
Leverage the power of advanced analytics calculations in Qlik Sense
Etc..

16. 16
1
User interacts with app,
making a selection or a
search
2
Hypercube recalculated
by the Qlik Associative
Engine to the new context
3
In-context data and
script sent to external
engine
4
External engine runs
and sends results to the
Qlik Associative engine
5
The Qlik Associative
Engine combines
hypercube with new data
6
Combined hypercube
is visualized for the
user in the app
How it works (chart expressions)

17. 17
Functional Buckets
Triggers from ETL
Triggering not only
advanced analytic
scripts/calls, but
potentially any call out to
an external system.
Think the ‘execute’
function.
Integrating functions in
ETL
Using advanced analytic
functions as a part of the
data load, e.g. sentiment
analysis or geocoding
within a record by record
load.
Front-end
This is what we primarily
market, where a user
makes a selection, the
associated data is then
sent, processed, and
returned through the
analytic connection in
real time.

18. 19
SSE Overview
Qlik Client
Qlik QMC
SSE Plugins
Qlik Engine

19. 20
Qlik Client
Qlik QMC
Qlik Engine
SSE with R Overview
Rserve

20. 21
Qlik Sense: QMC -> Analytic Connections Qlik Sense Desktop: settings.ini
Analytic Connection Configuration

21. 22
R Code Example

22. 23
R Code Example
The alias for the analytic connection

23. 24
R Code Example
One of the default base functions – passes R code as text to be executed

24. 25
R Code Example
Aggregation Functions
• ScriptAggr()
• ScrtiptAggrStr()
• ScriptAggrEx()
• ScriptAggrExStr()
-- Number in, Number out
-- String in, String out
-- Dual in, Number out
-- Dual in, String out
Scalar/Tensor Functions
• ScriptEval()
• ScrtiptEvalStr()
• ScriptEvalEx()
• ScriptEvalExStr()

25. 26
R Code Example
R script
• kmeans() is a clustering function within R
• q is the dataframe including the data from Qlik that is being sent to R
• $(vClusters) is a variable within Qlik, passing the amount of desired clusters
• nstart option creates multiple initial configurations and chooses the best one
• $cluster extracts the column named cluster from the dataframe

26. 27
R Code Example
The data being passed from Qlik as the dataframe q

27. 28
Python
Custom Functions

28. 29
Python Custom Function Example
The alias for the analytic connection

29. 30
Python Custom Function Example
The custom defined python function

30. 31
Python Custom Function Example

31. 32
• Scalar function with multiple arguments, the second being static
• Logged output below
Example Output from Qlik in Python

32. 33
• Tensor function with multiple arguments
• Argument 1 is a number from a Qlik field
• Argument 2 is a static input, and is
repeated with each call
• Batch processed due to Tensor
Example Output from Qlik in Python

33. 34
Load Script

34. 35
How it works – Load Script
• Mode 1: As a function

35. 36
How it works – Load Script
• Mode 2: Table load

36. 37
Demos

37. 38
Optimization &
Best Practices

38. 39
Known Limitations
• Syntax warnings and errors
• Syntax highlighting in the ETL might highlight in red for certain operations, even if correct
• Returning data to Qlik Sense
• Order of data sent from the connector might not match the way in which it is displayed
• There is no support for returning several columns or higher-order data, to Qlik Sense than was sent to the
plugin. The cardinality of the response from the plugin must be the same as sent from Qlik Sense.
• If less rows, or lower cardinality, is sent back from the plugin, Sense will add null values to match the
number of rows sent from Sense to the plugin originally. Note that the mapping may not work as intended if
less data is sent back.
• If higher-order data is sent back, either too many rows or more than one column, Qlik Sense will neglect
the additional data and only take the first n rows in the first column, assuming n is the number of rows sent
to the plugin. Note that the mapping may not work as intended if more data is sent back.

39. 40
Additional details
• Integration with advanced analytics can be executed during analysis or data loading.
• When executed during analysis, results are stored in an individual object (hypercube) and dynamically
change after each interaction
• When executed in the load script, results are stored in the overall data model for the app and remain static
until the next data refresh
• Qlik Sense does not natively support all the possible visualization types that might be needed
for advanced analytics calculations
• Extension objects can be easily built to handle more advanced scenarios
• Server-side extension APIs supporting advanced analytics integration are part of Qlik Sense
and fully supported by Qlik
• Custom extensions and connectors built by third parties are not supported by Qlik
• Qlik has open sourced two initial connectors, one for R and one for Python
• These extensions are open source community supported
• This aligns with how R and Python are typically supported – R and Python are both open source products
• We expect community to create other new extensions as open source projects, with a growing collection of
open-source capabilities available to customers

40. 41
Q&A