Trends in Enterprise
Advanced Analytics
Presented by: William McKnight
President, McKnight Consulting Group
williammcknight
www.mcknightcg.com
(214) 514-1444
#AdvAnalytics
William McKnight
President, McKnight Consulting Group
• Frequent keynote speaker and trainer internationally
• Consulted to Pfizer, Scotiabank, Fidelity, TD Ameritrade, Teva
Pharmaceuticals, Verizon, and many other Global 1000
companies
• Hundreds of articles, blogs and white papers in publication
• Focused on delivering business value and solving business
problems utilizing proven, streamlined approaches to
information management
• Former Database Engineer, Fortune 50 Information Technology
executive and Ernst&Young Entrepreneur of Year Finalist
• Owner/consultant: 2018 and 2017 Inc. 5000 strategy &
implementation consulting firm
• 30 years of information management and DBMS experience
2
McKnight Consulting Group Offerings
Strategy
Training
Strategy
 Trusted Advisor
 Action Plans
 Roadmaps
 Tool Selections
 Program Management
Training
 Classes
 Workshops
Implementation
 Data/Data Warehousing/Business
Intelligence/Analytics
 Master Data Management
 Governance/Quality
 Big Data
Implementation
3
Why Are Trends Important?
“Beyond the Mountain is
another mountain.”
We Are in the Business of Data
 Our information is exploding
 Our business is real-time, all the time
 Our information differentiates us from our
competitors
 Our information quality impacts our clients, our
Associates, and our shareholders
 Our information is used and reused; information
usage drives data value
 Information is a key business asset
Data Maturity is Highly Correlated to Business
Success
Data
Maturity
Business
Success
7
Maturity Modeling
• Should give a sense of priority
• You Can’t Skip Levels – in any category
• Maturity Levels tend to move in harmony
• Midsize and smaller companies can +1
• All must be at Level 3 (some need to be at 4) this year
• Momentum is paramount!
Raise the Foundation of Your
Company
Raise the Foundation of Your
Company
The Money Tree Doesn’t Exist
Hitch your Architecture and Maturity Efforts to an
Application Budget
10
Data Professional Success Measurement
User Satisfaction
Business ROI and
growth instigated
Data Maturity
(Long-term User Sat
and Bus ROI)
Misc.
Top Trends in Enterprise Analytics for
2019 and Beyond
Data
Lake
Usage Understanding by the Builders
D
a
t
a
C
u
l
t
i
v
a
t
i
o
n
Data
Warehouse
Data
Mart
Sensible Divisions of Analytic Platforms
Cloud Storage overtakes HDFS
• Cloud Storage is more scalable, persistent and
available, and less expensive
• Public Cloud Providers back up Cloud Storage
and support compression, making the cost of
big data less
• HDFS has better query performance
• HDFS has storage formats Parquet & ORC that
cannot be used on Cloud Storage
• Cloud Storage object size limits and PUT size
limits
14
Multi-Cloud
Becomes the
Norm
• Disparate data-related
objectives difficult to
pursue with agility in a
single-cloud strategy
• Kubernetes emerging
15
2019: The Year of Master Data Management
Source #1
SSN_NO X(9)
Claim_NO X(10)
Div_eff_dt X(10)
Source #2
Pol_ID 9(9)
Clm_NO X(10)
Stt_dt X(8
Source #3
Cust_ID X(10)
Claim_ID 9(9)
Beg_dt 9(8)
MDM
CLM_IDDec(15)SUB_ID Dec(13) EFF_DT DATE
MEMBER CLAIM GROUP
16
Data Virtualization Provides the Enterprise
Data Fabric
Consistent and timely access to right-placed
data
Data Warehouses
Marts & Cubes Operational
Data Stores Transactional
Sources
File Systems
Big Data
Enterprise Data Virtualization
17
2019: The Year of the Graph
• Stores entities and relationships
• Entities are “nodes”
• Relationships are “edges”
• Nodes and edges have properties
• Queries traverse the graph
• Nodes can be homogenous or heterogenous
• Consistent execution times not dependent on number
of nodes
Stream Processing
• ETL is Insufficient for this combination:
– Data platforms operating at an enterprise-wide scale
– A high variety of data sources
– Real-time/streaming data
• Enter Message-Oriented Middleware aka Streaming and message queuing
technology
19
Streaming
Platform
Streaming
Platform
Change logsChange logs
Streaming data pipelinesStreaming data pipelines
Messaging or
Stream processing
Messaging or
Stream processing
Request - ResponseRequest - Response
DWDW HadoopHadoop
Streaming
Platform
Change logs
Streaming data pipelines
Messaging or
Stream processing
Request - Response
DW Hadoop
AI is disruptive
Data is the Foundation
Data’s New Highest Use Will Be Training AI
Algorithms
Data Visualization Footprint Escalates
Treemaps Background Mapping InfoGraphics
Sparklines Bullet Graphs Scatter Plots
Self-Service Takes Off
• Technology delivers right-curated data, i.e., with
• Metadata
• Data Quality
• Performance
• An understanding of usage
• Technology can focus on more value-added activities
– Developing new applications
– Expanding data in data warehouse and improving its quality
– Incorporating new technologies to improve performance
• Technology becomes more of a partner rather than a
roadblock to business users
– Business users more responsible for BI capabilities
– Technology more supportive of business needs
Chief Data Officer Goes Mainstream
• Objectives
– Manage the project portfolio
– Create accountability
– Protect the company
• Data & Analytics Business Executive
• Data Strategy
• Data Maturity
23
Organizations Acknowledge Chief Information
Architect/Chief Analytics Officer
• Leads the process in every organization to vet practices and
ideas that accumulate in the industry and the enterprise and
assess their applicability to the architecture
• Looks “out and ahead” at unfulfilled, and often unspoken,
information management requirements and, as importantly, at
what the vendor marketplace is offering
• A job without boundaries of budget and deadlines, yet still
grounded in the reality that ultimately these factors will be in
place
• Solves tactical issues, but does so with the strategic needs of
the organization in mind
• Ensures there is a true architecture in place and followed
Data Science Pioneers Lock In
• Data Science Pioneers
– Let the Data Speak
– Use of Statistical Models
– Machine Learning
– Deep Business Implications to Work
– Deal in Algorithm Management
• Some fake-it-till-you-make-it Data Scientists
make it
• First wave of Data Science leaders emerge
– And reap the exponential benefits
25
Data Team Dynamics
• Business departments have clearly staked a claim in building their
architectures
– Still need dedicated technology professionals to do the work
– The notion of an "IT professional" is alive and well
– The reporting structure is more complicated than ever
• Acknowledgement of the need for data deployments to be near the
business unit in organization charts
• Strategists and implementors are seeing a reduction in the challenges
posed by internal grist and resistance to change
– Dependence on certain individuals is lessened with the cloud, and
in 2019, many will declare their organization unshackled from
resistance to progress
– Acceleration of acceptance and some challenging personnel
moments inside the data apparatus in organizations
26
Analytics Skills Go into the Operational
Environment
Data Lake
DW
DM
DM
27
Operational Big Data Platform Selection
SQL
Data
Size
Workload Complexity
Key-Value
Document
Column Store
Graph
SQL
A New Dataset: Bio Data
29
 There’s more maturity
in moving imperfectly
than in merely
perfectly defining the
shortcomings
 Build credibility
 Don’t be afraid to fail
 Don’t talk yourself out
of having a new
beginning
Have an open mind
No plateaus are
comfortable for long
That resistance is not
about making
progress, it’s the
journey
Second Thursday of Every
Month, at 2:00 ET
Presented by: William McKnight
President, McKnight Consulting Group
www.mcknightcg.com (214) 514-1444
#AdvAnalytics

Trends in Enterprise Advanced Analytics

  • 1.
    Trends in Enterprise AdvancedAnalytics Presented by: William McKnight President, McKnight Consulting Group williammcknight www.mcknightcg.com (214) 514-1444 #AdvAnalytics
  • 2.
    William McKnight President, McKnightConsulting Group • Frequent keynote speaker and trainer internationally • Consulted to Pfizer, Scotiabank, Fidelity, TD Ameritrade, Teva Pharmaceuticals, Verizon, and many other Global 1000 companies • Hundreds of articles, blogs and white papers in publication • Focused on delivering business value and solving business problems utilizing proven, streamlined approaches to information management • Former Database Engineer, Fortune 50 Information Technology executive and Ernst&Young Entrepreneur of Year Finalist • Owner/consultant: 2018 and 2017 Inc. 5000 strategy & implementation consulting firm • 30 years of information management and DBMS experience 2
  • 3.
    McKnight Consulting GroupOfferings Strategy Training Strategy  Trusted Advisor  Action Plans  Roadmaps  Tool Selections  Program Management Training  Classes  Workshops Implementation  Data/Data Warehousing/Business Intelligence/Analytics  Master Data Management  Governance/Quality  Big Data Implementation 3
  • 4.
    Why Are TrendsImportant?
  • 5.
    “Beyond the Mountainis another mountain.”
  • 6.
    We Are inthe Business of Data  Our information is exploding  Our business is real-time, all the time  Our information differentiates us from our competitors  Our information quality impacts our clients, our Associates, and our shareholders  Our information is used and reused; information usage drives data value  Information is a key business asset
  • 7.
    Data Maturity isHighly Correlated to Business Success Data Maturity Business Success 7
  • 8.
    Maturity Modeling • Shouldgive a sense of priority • You Can’t Skip Levels – in any category • Maturity Levels tend to move in harmony • Midsize and smaller companies can +1 • All must be at Level 3 (some need to be at 4) this year • Momentum is paramount!
  • 9.
    Raise the Foundationof Your Company Raise the Foundation of Your Company
  • 10.
    The Money TreeDoesn’t Exist Hitch your Architecture and Maturity Efforts to an Application Budget 10
  • 11.
    Data Professional SuccessMeasurement User Satisfaction Business ROI and growth instigated Data Maturity (Long-term User Sat and Bus ROI) Misc.
  • 12.
    Top Trends inEnterprise Analytics for 2019 and Beyond
  • 13.
    Data Lake Usage Understanding bythe Builders D a t a C u l t i v a t i o n Data Warehouse Data Mart Sensible Divisions of Analytic Platforms
  • 14.
    Cloud Storage overtakesHDFS • Cloud Storage is more scalable, persistent and available, and less expensive • Public Cloud Providers back up Cloud Storage and support compression, making the cost of big data less • HDFS has better query performance • HDFS has storage formats Parquet & ORC that cannot be used on Cloud Storage • Cloud Storage object size limits and PUT size limits 14
  • 15.
    Multi-Cloud Becomes the Norm • Disparatedata-related objectives difficult to pursue with agility in a single-cloud strategy • Kubernetes emerging 15
  • 16.
    2019: The Yearof Master Data Management Source #1 SSN_NO X(9) Claim_NO X(10) Div_eff_dt X(10) Source #2 Pol_ID 9(9) Clm_NO X(10) Stt_dt X(8 Source #3 Cust_ID X(10) Claim_ID 9(9) Beg_dt 9(8) MDM CLM_IDDec(15)SUB_ID Dec(13) EFF_DT DATE MEMBER CLAIM GROUP 16
  • 17.
    Data Virtualization Providesthe Enterprise Data Fabric Consistent and timely access to right-placed data Data Warehouses Marts & Cubes Operational Data Stores Transactional Sources File Systems Big Data Enterprise Data Virtualization 17
  • 18.
    2019: The Yearof the Graph • Stores entities and relationships • Entities are “nodes” • Relationships are “edges” • Nodes and edges have properties • Queries traverse the graph • Nodes can be homogenous or heterogenous • Consistent execution times not dependent on number of nodes
  • 19.
    Stream Processing • ETLis Insufficient for this combination: – Data platforms operating at an enterprise-wide scale – A high variety of data sources – Real-time/streaming data • Enter Message-Oriented Middleware aka Streaming and message queuing technology 19 Streaming Platform Streaming Platform Change logsChange logs Streaming data pipelinesStreaming data pipelines Messaging or Stream processing Messaging or Stream processing Request - ResponseRequest - Response DWDW HadoopHadoop Streaming Platform Change logs Streaming data pipelines Messaging or Stream processing Request - Response DW Hadoop
  • 20.
    AI is disruptive Datais the Foundation Data’s New Highest Use Will Be Training AI Algorithms
  • 21.
    Data Visualization FootprintEscalates Treemaps Background Mapping InfoGraphics Sparklines Bullet Graphs Scatter Plots
  • 22.
    Self-Service Takes Off •Technology delivers right-curated data, i.e., with • Metadata • Data Quality • Performance • An understanding of usage • Technology can focus on more value-added activities – Developing new applications – Expanding data in data warehouse and improving its quality – Incorporating new technologies to improve performance • Technology becomes more of a partner rather than a roadblock to business users – Business users more responsible for BI capabilities – Technology more supportive of business needs
  • 23.
    Chief Data OfficerGoes Mainstream • Objectives – Manage the project portfolio – Create accountability – Protect the company • Data & Analytics Business Executive • Data Strategy • Data Maturity 23
  • 24.
    Organizations Acknowledge ChiefInformation Architect/Chief Analytics Officer • Leads the process in every organization to vet practices and ideas that accumulate in the industry and the enterprise and assess their applicability to the architecture • Looks “out and ahead” at unfulfilled, and often unspoken, information management requirements and, as importantly, at what the vendor marketplace is offering • A job without boundaries of budget and deadlines, yet still grounded in the reality that ultimately these factors will be in place • Solves tactical issues, but does so with the strategic needs of the organization in mind • Ensures there is a true architecture in place and followed
  • 25.
    Data Science PioneersLock In • Data Science Pioneers – Let the Data Speak – Use of Statistical Models – Machine Learning – Deep Business Implications to Work – Deal in Algorithm Management • Some fake-it-till-you-make-it Data Scientists make it • First wave of Data Science leaders emerge – And reap the exponential benefits 25
  • 26.
    Data Team Dynamics •Business departments have clearly staked a claim in building their architectures – Still need dedicated technology professionals to do the work – The notion of an "IT professional" is alive and well – The reporting structure is more complicated than ever • Acknowledgement of the need for data deployments to be near the business unit in organization charts • Strategists and implementors are seeing a reduction in the challenges posed by internal grist and resistance to change – Dependence on certain individuals is lessened with the cloud, and in 2019, many will declare their organization unshackled from resistance to progress – Acceleration of acceptance and some challenging personnel moments inside the data apparatus in organizations 26
  • 27.
    Analytics Skills Gointo the Operational Environment Data Lake DW DM DM 27
  • 28.
    Operational Big DataPlatform Selection SQL Data Size Workload Complexity Key-Value Document Column Store Graph SQL
  • 29.
    A New Dataset:Bio Data 29
  • 30.
     There’s morematurity in moving imperfectly than in merely perfectly defining the shortcomings  Build credibility  Don’t be afraid to fail  Don’t talk yourself out of having a new beginning Have an open mind No plateaus are comfortable for long That resistance is not about making progress, it’s the journey
  • 31.
    Second Thursday ofEvery Month, at 2:00 ET Presented by: William McKnight President, McKnight Consulting Group www.mcknightcg.com (214) 514-1444 #AdvAnalytics