Do you have a true Big Data Analytics platform? What's a true Big Data Analytics platform? How can it help capitalize big data? What's needed to build one? This short introductory presentation can help understand what's a true Big Data Analytics platform and how it really helps building Big Data Analytics applications.

1. What are you building?
A Platform or an IDE
Two approaches in big data analytics

2. Intro: Big Data Analytics
is In
The biggest difference between traditional enterprises
and internet-age enterprises lies in the data, everything
is being converted to data. The competition between
companies is really the competition between their
utilization of data.

3. Data as Capital
In the age of everything is being digitalized, data is
everywhere and winning in the virtual world directly
translates into competitive advantage in real world
How to properly value and utilize data is the key to
capitalizing on big data

4. Steps to Achievement
Data Collection
•Identify datasets
•Collect data
•Joining data
Basic Analytics
•Understand data
•Clean, enrich, transform
data
•Simple discovery
Experimentation
•Apply data science
•Learn more about data
•Build and test models
Model Application
•Apply various models
•Get feedback, refine models
How to win at Big Data Analytics

5. Types of Users
Decision
makers
Line-of-
business
workers
Business
analysts
Data
scientists /
engineers
Domain
experts

6. What need be built?
Data Information Knowledge Wisdom

7. Two approaches
Applications
Analytic
Platform
Integrated
Development
Environment

8. Do you have a big data
analytics platform?
Many companies claim they have one
Do they really have a true platform, or they are
building custom applications every time?

9. What is a Platform
In general, a piece of computer software designed to support
applications, with fundamental functions provided, obeying its
constraints, and making use of its facilities
Different abstraction levels: hardware, OS, Runtime, Web,
Cloud, Analytics …
A big data analytics platform allows people to build apps out of
components that are hosted or provided by the providers with
specific protocols linking them together

10. Features of a Computing
Platform
Enables quick development of custom apps by providing
prebuilt functionalities (not tools or usability
enhancements)
Components can be independently applied and can
communicate with each other, often with proprietary
semantics and protocols
Usually result in a lock-in due to data or protocol
specifications, hard to move apps away

11. Examples of a Platform
Intel, Microsoft Windows (Wintel)
Adobe AIR, Apple iOS, …
Java Platform (J2EE etc.), .Net Framework
Facebook/Twitter …
WordPress

12. What is an IDE
An Integrated Development Environment (IDE) is a software
application that provides comprehensive facilities to developers for
software development. IDE normally contains a code/script editor,
build automation, file/item browser, debugger (profiler/monitor).
IDE normally offers features like GUI, MDI, RAD, and support code
generation, automation of execution (deployment), and revision
control…
Modern features include intelligent code completion, visual
browser, workflow manager and other productivity features

13. Examples of IDE
Microsoft Visual Studio, Delphi
Eclipse, IntelliJ IDEA, PyCharm
Xcode
WebStorm
Cloud9
…

14. Similarities Between a
Platform and IDE
Both are software providing facilities to its users
Both can enable faster application development

15. Differences between
Platform and IDE
Platform
Providing facilities in the form of
functional components
Faster development speed via pre-
packaged functionalities
Allowing users to build
applications only with its
functions*, can use multiple IDEs
Resulting in lock-in of applications
IDE
Providing facilities in the forms of
usability improvements
Faster development speed via
stream-lined operations
Allowing users develop only within
its environment, can support
multiple platforms
Resulting in lock-in of project files
* Most platforms allow calling external components, but still need fit into its own platform constraints

16. Why Not IDE
IDE can help one type of user, most likely Data
scientists or software engineers
These users are usually not the majority users in the
company
The ROI is mainly usability: low
Vs. platform that produces applications which can
multiply productivity

17. Why Platform
High reusability  Decreased time and cost to market
Supporting more customers  Higher value for
customers
Built-in flexibility  Faster application development time
Component Marketplace (AppStore)  Lower support
cost, enable third-party contributions

18. Five-star Data
Accessible
Parse-able (structurizable)
With shared metadata
Identifiable
Connected with relations

19. Four Pillars
Knowledge-base: domain expertise, rules, data, metadata, etc.
Semantic data management system: manage all software
artifacts including data sources, datasets, projects, users…
Function modules: parsers, algorithms, visualization modules,
transformers, models, … to build apps
Infrastructure support: connect to proper infrastructure to run
all the things

20. Custom Application
Development Workflow
Example Application: HR Insights
Start with requirements and goals:
Overview of whole company’s employees’ hours, times on
which app/sites, sentiments, average time of responding
email/requests, models to predict performance or attrition
The goals contain specific details on standards, conditions,
environment, resources, and even methodologies

21. HR Insights Workflow Step
#1
Find or Create Goals
Find similar goals
If not, specifying details such as working hours,
email/request response time, mapping out natural
workgroups via communication patterns …
Collect data
Select from list of known Sources and Datasets
Or create new sources or datasets if necessary

22. HR Insights Workflow
Step #2
Performing Ingestion, Pre-processing (Parsing), and
Instant Analytics (basic stats and other quick insights)
To help understand the data better for further steps
Perform transformation to get more targeted datasets
SMEs can run a set of existing tools (apps, models,
transformations) to get more insights
Including enriching, filtering, linking to other data sets and
do it over again

23. HR Insights Workflow Step
#3
Create Analytic Pipeline Requests to solicit data science
experts
Data scientists can start doing experimentation and build
models
Models reviewed and published as applications
End users can benefit from new models/capabilities

24. 24
Thank you