SlideShare a Scribd company logo
1 of 20
Applying Data Science
to
Government Services
Data Science
• Extracting knowledge or insights from data
– in various forms, structured or unstructured
• Utilizes data preparation, statistics, predictive
modeling and Machine Learning
• Applied to various domains
– Discovering new cures, Improving science research
– Optimizing supply chains and delivery routes
– Reducing traffic congestions, Optimizing energy grids
– Forecasting weather, Improving sports performance
– Improving security and reducing spam
– Targeted marketing, personalization, churn prediction
© Harbinger Systems | www.harbinger-systems.com
8-Levels of Analytics (SAS)
© Harbinger Systems | www.harbinger-systems.com
Information Strategy (Gartner)
• Enterprise Information Management
– Information is everywhere & growing
– Volume, Variety & Velocity
– Drive innovation in rapid information processing
• Information Strategy
– Harness the power of information assets
– Drive growth, improve efficiency
• Data Analytics – Strategic decision making
– Insights from your large and complex datasets
– Predict future behaviors, trends and outcomes
© Harbinger Systems | www.harbinger-systems.com
Machine Learning (ML)
A type of Artificial Intelligence that provides computers with
ability to learn without being explicitly programmed.
– Computer can infer rules inherent in data
– Computer adapts when exposed to new data
• (Tom Mitchell ) - A computer program is set to learn from an
experience E with respect to some task T and some performance
measure P if its performance on T as measured by P improves
with experience E
• Automating Automata
© Harbinger Systems | www.harbinger-systems.com
What’s a Machine Learning Problem?
© Harbinger Systems | www.harbinger-systems.com
Emphasis of machine learning is on
automatic methods
Devise learning algorithms that do the
learning automatically without human
intervention
Program by example: we don't care what
the machine does, as long as it does it
right
Result-oriented rather than process-
oriented
How can Machine Learning Add Value?
© Harbinger Systems | www.harbinger-systems.com
ML is a data driven approach
• Business knowledge isn’t necessary
ML is domain independent
• Same algorithms can be used across domains and in different use cases
ML creates flexible decision systems
• Creates robust systems that can adjust for changing systems without
human intervention
ML and Big Data
ML thrives with big data!
– Accuracy of algorithms increases with size of data
– Statistical approaches can treat big datasets much better than
traditional paradigms
– Decision making using ML can adapt to transactional data much better
© Harbinger Systems | www.harbinger-systems.com
Machine Learning Big Data
Fraud Detection: Did the user really do this login/make this purchase?
Product Recommendation: Will the user like this product?
Stock Trading: Will the stock go up or down?
Medical Diagnosis: Given some symptoms, what is the patient
suffering from
© Harbinger Systems | www.harbinger-systems.com
Machine Learning Applications
© Harbinger Systems | www.harbinger-systems.com
How to Categorize the Problem?
Generally, machine learning problems looks to:
Identify a Value
Assign data points to a category
Discover similarities between two data points
© Harbinger Systems | www.harbinger-systems.com
Flowchart
Start
Sufficient
Data?
Sort into
category?
Predict a
value?
Define Problem!
Labeled
Data
Clustering
Classification
Get more!
Regression
© Harbinger Systems | www.harbinger-systems.com
What to look for in algorithms:
Flexible across many use cases
Able to handle several input types
Accurate
Resistant to over-fitting/noise/error
Machine Learning Algorithms
© Harbinger Systems | www.harbinger-systems.com
Random Forest
Used for classification and regression
Works on small subsets of data and combines the result into the best estimate
XGBoost
Works on classification and regression
Starts off with a weak learner that improves over successive iterations
K-Means
Works on classification and clustering
Tries to find boundaries between data points for each individual variable
Machine Learning Algorithms
© Harbinger Systems | www.harbinger-systems.com
Tools and Technologies
Emphasis on tools which
Can integrate with existing data architecture
Have a smooth learning curve
Simplify the process of analysis and prediction
Have an active community
© Harbinger Systems | www.harbinger-systems.com
Popular Machine Learning Tools
Python
Free, open-source, widely popular
Consolidates many important libraries in python, C
Has an active community
Disclaimer: Brand names, logos and trademarks used herein remain the property of their respective owners.
© Harbinger Systems | www.harbinger-systems.com
Popular Machine Learning Tools
R
Statistical computing language that simplifies complex
statistical operations
Large number of libraries available for extending
functionality (DB connectors, algorithm, visualization)
Disclaimer: Brand names, logos and trademarks used herein remain the property of their respective owners.
Open Data and Gov Services
© Harbinger Systems | www.harbinger-systems.com
• Open data, tools and resources available
• ~181K datasets
Sample Applications
• City-Data provides detailed profiles of all U.S. cities -
demographics, crime rates, home values, cost of
living, etc.
• Farmers can use Climate Corporation’s services to
plan, manage, and protect crops
• SPOT Crime : Free public facing crime mapping and
alert website
© Harbinger Systems | www.harbinger-systems.com
Conclusion
• Harness the power of your data to deliver
higher value services and remain competitive
–“Data is the currency of the future” –
Michael Cockrill, CIO State of WA
• Machine learning provides a powerful
framework for extracting insights
© Harbinger Systems | www.harbinger-systems.com
Thank You
© Harbinger Systems | www.harbinger-systems.com

More Related Content

What's hot

What's hot (20)

H2O.ai's Driverless AI
H2O.ai's Driverless AIH2O.ai's Driverless AI
H2O.ai's Driverless AI
 
Machine learning prediction of stock markets
Machine learning prediction of stock marketsMachine learning prediction of stock markets
Machine learning prediction of stock markets
 
18CSS101J PROGRAMMING FOR PROBLEM SOLVING
18CSS101J PROGRAMMING FOR PROBLEM SOLVING18CSS101J PROGRAMMING FOR PROBLEM SOLVING
18CSS101J PROGRAMMING FOR PROBLEM SOLVING
 
C presentation
C presentationC presentation
C presentation
 
C the basic concepts
C the basic conceptsC the basic concepts
C the basic concepts
 
Number System
Number SystemNumber System
Number System
 
MLOps by Sasha Rosenbaum
MLOps by Sasha RosenbaumMLOps by Sasha Rosenbaum
MLOps by Sasha Rosenbaum
 
Semantic analysis
Semantic analysisSemantic analysis
Semantic analysis
 
Er diagrams presentation
Er diagrams presentationEr diagrams presentation
Er diagrams presentation
 
Introduction of data science
Introduction of data scienceIntroduction of data science
Introduction of data science
 
Data representation
Data representationData representation
Data representation
 
Number System (Binary,octal,Decimal,Hexadecimal)
Number System (Binary,octal,Decimal,Hexadecimal)Number System (Binary,octal,Decimal,Hexadecimal)
Number System (Binary,octal,Decimal,Hexadecimal)
 
Data representation
Data representationData representation
Data representation
 
Programming Fundamentals
Programming FundamentalsProgramming Fundamentals
Programming Fundamentals
 
Analysis of the source program
Analysis of the source programAnalysis of the source program
Analysis of the source program
 
Programming languages.pptx
Programming languages.pptxProgramming languages.pptx
Programming languages.pptx
 
About Tokens and Lexemes
About Tokens and LexemesAbout Tokens and Lexemes
About Tokens and Lexemes
 
Compiler lec 8
Compiler lec 8Compiler lec 8
Compiler lec 8
 
Data analytics introduction
Data analytics introductionData analytics introduction
Data analytics introduction
 
Chapter3
Chapter3Chapter3
Chapter3
 

Viewers also liked

Create your Big Data vision and Hadoop-ify your data warehouse
Create your Big Data vision and Hadoop-ify your data warehouseCreate your Big Data vision and Hadoop-ify your data warehouse
Create your Big Data vision and Hadoop-ify your data warehouseJeff Kelly
 
Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...
Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...
Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...Data Con LA
 
Lecture on Data Science in a Data-Driven Culture
Lecture on Data Science in a Data-Driven Culture Lecture on Data Science in a Data-Driven Culture
Lecture on Data Science in a Data-Driven Culture Johan Himberg
 
"Using Data Science to Design Effective Precision Preventative Behavioral Med...
"Using Data Science to Design Effective Precision Preventative Behavioral Med..."Using Data Science to Design Effective Precision Preventative Behavioral Med...
"Using Data Science to Design Effective Precision Preventative Behavioral Med...Hyper Wellbeing
 
How to reach a Data Driven culture
How to reach a Data Driven cultureHow to reach a Data Driven culture
How to reach a Data Driven cultureMark Beekman
 

Viewers also liked (6)

Create your Big Data vision and Hadoop-ify your data warehouse
Create your Big Data vision and Hadoop-ify your data warehouseCreate your Big Data vision and Hadoop-ify your data warehouse
Create your Big Data vision and Hadoop-ify your data warehouse
 
Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...
Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...
Big Data Day LA 2016/ Data Science Track - Backstage to a Data Driven Culture...
 
Lecture on Data Science in a Data-Driven Culture
Lecture on Data Science in a Data-Driven Culture Lecture on Data Science in a Data-Driven Culture
Lecture on Data Science in a Data-Driven Culture
 
"Using Data Science to Design Effective Precision Preventative Behavioral Med...
"Using Data Science to Design Effective Precision Preventative Behavioral Med..."Using Data Science to Design Effective Precision Preventative Behavioral Med...
"Using Data Science to Design Effective Precision Preventative Behavioral Med...
 
How to reach a Data Driven culture
How to reach a Data Driven cultureHow to reach a Data Driven culture
How to reach a Data Driven culture
 
Webinar: UI/UX best practices in cms based web design
Webinar: UI/UX best practices in cms based web designWebinar: UI/UX best practices in cms based web design
Webinar: UI/UX best practices in cms based web design
 

Similar to Application of Data Science in Government Services – IPMA Forum 2016 Speaker Session

Data Analytics and Big Data on IoT
Data Analytics and Big Data on IoTData Analytics and Big Data on IoT
Data Analytics and Big Data on IoTShivam Singh
 
Gse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedGse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedcedrinemadera
 
Machine Learning in Customer Analytics
Machine Learning in Customer AnalyticsMachine Learning in Customer Analytics
Machine Learning in Customer AnalyticsCourse5i
 
In-Depth Data Analytics
In-Depth Data AnalyticsIn-Depth Data Analytics
In-Depth Data AnalyticsYASH GAIKWAD
 
Using Machine Learning to Understand and Predict Marketing ROI
Using Machine Learning to Understand and Predict Marketing ROIUsing Machine Learning to Understand and Predict Marketing ROI
Using Machine Learning to Understand and Predict Marketing ROIDATAVERSITY
 
TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...
TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...
TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...Vasu S
 
DIGITAL TRANSFORMATION AND STRATEGY_final.pptx
DIGITAL TRANSFORMATION AND STRATEGY_final.pptxDIGITAL TRANSFORMATION AND STRATEGY_final.pptx
DIGITAL TRANSFORMATION AND STRATEGY_final.pptxGeorgeDiamandis11
 
What is the Value of SAS Analytics?
What is the Value of SAS Analytics?What is the Value of SAS Analytics?
What is the Value of SAS Analytics?SAS Canada
 
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...mattdenesuk
 
Big Data Tools PowerPoint Presentation Slides
Big Data Tools PowerPoint Presentation SlidesBig Data Tools PowerPoint Presentation Slides
Big Data Tools PowerPoint Presentation SlidesSlideTeam
 
Big Data Analytics Architecture PowerPoint Presentation Slides
Big Data Analytics Architecture PowerPoint Presentation SlidesBig Data Analytics Architecture PowerPoint Presentation Slides
Big Data Analytics Architecture PowerPoint Presentation SlidesSlideTeam
 
Data Science.pptx NEW COURICUUMN IN DATA
Data Science.pptx NEW COURICUUMN IN DATAData Science.pptx NEW COURICUUMN IN DATA
Data Science.pptx NEW COURICUUMN IN DATAjaved75
 
Winning with data
Winning with dataWinning with data
Winning with dataNUS-ISS
 
how to successfully implement a data analytics solution.pdf
how to successfully implement a data analytics solution.pdfhow to successfully implement a data analytics solution.pdf
how to successfully implement a data analytics solution.pdfbasilmph
 
Tips --Break Down the Barriers to Better Data Analytics
Tips --Break Down the Barriers to Better Data AnalyticsTips --Break Down the Barriers to Better Data Analytics
Tips --Break Down the Barriers to Better Data AnalyticsAbhishek Sood
 

Similar to Application of Data Science in Government Services – IPMA Forum 2016 Speaker Session (20)

Discover the Potential of your Data with Machine Learning
Discover the Potential of your Data with Machine LearningDiscover the Potential of your Data with Machine Learning
Discover the Potential of your Data with Machine Learning
 
Machine Data Analytics
Machine Data AnalyticsMachine Data Analytics
Machine Data Analytics
 
AlogoAnalytics Company Presentation
AlogoAnalytics Company PresentationAlogoAnalytics Company Presentation
AlogoAnalytics Company Presentation
 
Data Analytics and Big Data on IoT
Data Analytics and Big Data on IoTData Analytics and Big Data on IoT
Data Analytics and Big Data on IoT
 
Gse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-sharedGse uk-cedrinemadera-2018-shared
Gse uk-cedrinemadera-2018-shared
 
Machine Learning in Customer Analytics
Machine Learning in Customer AnalyticsMachine Learning in Customer Analytics
Machine Learning in Customer Analytics
 
In-Depth Data Analytics
In-Depth Data AnalyticsIn-Depth Data Analytics
In-Depth Data Analytics
 
Sgcp14dunlea
Sgcp14dunleaSgcp14dunlea
Sgcp14dunlea
 
Using Machine Learning to Understand and Predict Marketing ROI
Using Machine Learning to Understand and Predict Marketing ROIUsing Machine Learning to Understand and Predict Marketing ROI
Using Machine Learning to Understand and Predict Marketing ROI
 
TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...
TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...
TDWI Checklist - The Automation and Optimization of Advanced Analytics Based ...
 
DIGITAL TRANSFORMATION AND STRATEGY_final.pptx
DIGITAL TRANSFORMATION AND STRATEGY_final.pptxDIGITAL TRANSFORMATION AND STRATEGY_final.pptx
DIGITAL TRANSFORMATION AND STRATEGY_final.pptx
 
Trends in data analytics
Trends in data analyticsTrends in data analytics
Trends in data analytics
 
What is the Value of SAS Analytics?
What is the Value of SAS Analytics?What is the Value of SAS Analytics?
What is the Value of SAS Analytics?
 
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
 
Big Data Tools PowerPoint Presentation Slides
Big Data Tools PowerPoint Presentation SlidesBig Data Tools PowerPoint Presentation Slides
Big Data Tools PowerPoint Presentation Slides
 
Big Data Analytics Architecture PowerPoint Presentation Slides
Big Data Analytics Architecture PowerPoint Presentation SlidesBig Data Analytics Architecture PowerPoint Presentation Slides
Big Data Analytics Architecture PowerPoint Presentation Slides
 
Data Science.pptx NEW COURICUUMN IN DATA
Data Science.pptx NEW COURICUUMN IN DATAData Science.pptx NEW COURICUUMN IN DATA
Data Science.pptx NEW COURICUUMN IN DATA
 
Winning with data
Winning with dataWinning with data
Winning with data
 
how to successfully implement a data analytics solution.pdf
how to successfully implement a data analytics solution.pdfhow to successfully implement a data analytics solution.pdf
how to successfully implement a data analytics solution.pdf
 
Tips --Break Down the Barriers to Better Data Analytics
Tips --Break Down the Barriers to Better Data AnalyticsTips --Break Down the Barriers to Better Data Analytics
Tips --Break Down the Barriers to Better Data Analytics
 

More from Harbinger Systems - HRTech Builder of Choice

More from Harbinger Systems - HRTech Builder of Choice (20)

Using People Analytics for a Sustainable Remote Workforce
Using People Analytics for a Sustainable Remote WorkforceUsing People Analytics for a Sustainable Remote Workforce
Using People Analytics for a Sustainable Remote Workforce
 
5 Trends That Will Drive the Transformation of EdTech in 2021
5 Trends That Will Drive the Transformation of EdTech in 20215 Trends That Will Drive the Transformation of EdTech in 2021
5 Trends That Will Drive the Transformation of EdTech in 2021
 
Rapidly Transforming Organizational Content into Learning Experiences
Rapidly Transforming Organizational Content into Learning ExperiencesRapidly Transforming Organizational Content into Learning Experiences
Rapidly Transforming Organizational Content into Learning Experiences
 
Scalable HR Integrations for Better Data Analytics: Challenges & Solutions
Scalable HR Integrations for Better Data Analytics: Challenges & SolutionsScalable HR Integrations for Better Data Analytics: Challenges & Solutions
Scalable HR Integrations for Better Data Analytics: Challenges & Solutions
 
5 Key Items HR Should Consider Before Buying HR Technologies
5 Key Items HR Should Consider Before Buying HR Technologies5 Key Items HR Should Consider Before Buying HR Technologies
5 Key Items HR Should Consider Before Buying HR Technologies
 
Best Practices to Build Marketplace-Ready Integrations
Best Practices to Build Marketplace-Ready IntegrationsBest Practices to Build Marketplace-Ready Integrations
Best Practices to Build Marketplace-Ready Integrations
 
HRTech Integration Masterclass Session 4 How to Expand Your Recruitment Datab...
HRTech Integration Masterclass Session 4 How to Expand Your Recruitment Datab...HRTech Integration Masterclass Session 4 How to Expand Your Recruitment Datab...
HRTech Integration Masterclass Session 4 How to Expand Your Recruitment Datab...
 
Recalibrating Product Strategy - Addressing Demand Shifts in Existing Markets
Recalibrating Product Strategy - Addressing Demand Shifts in Existing MarketsRecalibrating Product Strategy - Addressing Demand Shifts in Existing Markets
Recalibrating Product Strategy - Addressing Demand Shifts in Existing Markets
 
How to Gain Key Insights from Data Distributed Across Multiple HR Systems
How to Gain Key Insights from Data Distributed Across Multiple HR SystemsHow to Gain Key Insights from Data Distributed Across Multiple HR Systems
How to Gain Key Insights from Data Distributed Across Multiple HR Systems
 
HRTech Integration Master Class Session 1 -Delivering Seamless Learning Exper...
HRTech Integration Master Class Session 1 -Delivering Seamless Learning Exper...HRTech Integration Master Class Session 1 -Delivering Seamless Learning Exper...
HRTech Integration Master Class Session 1 -Delivering Seamless Learning Exper...
 
Recalibrating Product Strategy - Addressing Demand Shifts in Existing Markets
Recalibrating Product Strategy - Addressing Demand Shifts in Existing MarketsRecalibrating Product Strategy - Addressing Demand Shifts in Existing Markets
Recalibrating Product Strategy - Addressing Demand Shifts in Existing Markets
 
Integrating System of Records and Collaboration Tools
Integrating System of Records and Collaboration ToolsIntegrating System of Records and Collaboration Tools
Integrating System of Records and Collaboration Tools
 
How to Power Your HR Apps With AI And Make It Explainable
How to Power Your HR Apps With AI And Make It ExplainableHow to Power Your HR Apps With AI And Make It Explainable
How to Power Your HR Apps With AI And Make It Explainable
 
Chatbot for Continuous Performance Management
Chatbot for Continuous Performance Management Chatbot for Continuous Performance Management
Chatbot for Continuous Performance Management
 
Leveraging mobile capabilities in your HR application
Leveraging mobile capabilities in your HR applicationLeveraging mobile capabilities in your HR application
Leveraging mobile capabilities in your HR application
 
Automate HR applications using AI and ML
Automate HR applications using AI and MLAutomate HR applications using AI and ML
Automate HR applications using AI and ML
 
Engage for Success: Improve Workforce Engagement with Open Communication and ...
Engage for Success: Improve Workforce Engagement with Open Communication and ...Engage for Success: Improve Workforce Engagement with Open Communication and ...
Engage for Success: Improve Workforce Engagement with Open Communication and ...
 
Building next gen hr solutions with people analytics-final
Building next gen hr solutions with people analytics-finalBuilding next gen hr solutions with people analytics-final
Building next gen hr solutions with people analytics-final
 
A Cloud-based Collaborative Learning and Coaching Platform
A Cloud-based Collaborative Learning and Coaching PlatformA Cloud-based Collaborative Learning and Coaching Platform
A Cloud-based Collaborative Learning and Coaching Platform
 
Extending LRSs and the xAPI for Event-driven Blended and Adaptive Learning
Extending LRSs and the xAPI for Event-driven Blended and Adaptive LearningExtending LRSs and the xAPI for Event-driven Blended and Adaptive Learning
Extending LRSs and the xAPI for Event-driven Blended and Adaptive Learning
 

Recently uploaded

Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Muhammad Tiham Siddiqui
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTxtailishbaloch
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024Brian Pichman
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.IPLOOK Networks
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1DianaGray10
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptxHansamali Gamage
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Alkin Tezuysal
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 

Recently uploaded (20)

Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
SheDev 2024
SheDev 2024SheDev 2024
SheDev 2024
 

Application of Data Science in Government Services – IPMA Forum 2016 Speaker Session

  • 2. Data Science • Extracting knowledge or insights from data – in various forms, structured or unstructured • Utilizes data preparation, statistics, predictive modeling and Machine Learning • Applied to various domains – Discovering new cures, Improving science research – Optimizing supply chains and delivery routes – Reducing traffic congestions, Optimizing energy grids – Forecasting weather, Improving sports performance – Improving security and reducing spam – Targeted marketing, personalization, churn prediction © Harbinger Systems | www.harbinger-systems.com
  • 3. 8-Levels of Analytics (SAS) © Harbinger Systems | www.harbinger-systems.com
  • 4. Information Strategy (Gartner) • Enterprise Information Management – Information is everywhere & growing – Volume, Variety & Velocity – Drive innovation in rapid information processing • Information Strategy – Harness the power of information assets – Drive growth, improve efficiency • Data Analytics – Strategic decision making – Insights from your large and complex datasets – Predict future behaviors, trends and outcomes © Harbinger Systems | www.harbinger-systems.com
  • 5. Machine Learning (ML) A type of Artificial Intelligence that provides computers with ability to learn without being explicitly programmed. – Computer can infer rules inherent in data – Computer adapts when exposed to new data • (Tom Mitchell ) - A computer program is set to learn from an experience E with respect to some task T and some performance measure P if its performance on T as measured by P improves with experience E • Automating Automata © Harbinger Systems | www.harbinger-systems.com
  • 6. What’s a Machine Learning Problem? © Harbinger Systems | www.harbinger-systems.com Emphasis of machine learning is on automatic methods Devise learning algorithms that do the learning automatically without human intervention Program by example: we don't care what the machine does, as long as it does it right Result-oriented rather than process- oriented
  • 7. How can Machine Learning Add Value? © Harbinger Systems | www.harbinger-systems.com ML is a data driven approach • Business knowledge isn’t necessary ML is domain independent • Same algorithms can be used across domains and in different use cases ML creates flexible decision systems • Creates robust systems that can adjust for changing systems without human intervention
  • 8. ML and Big Data ML thrives with big data! – Accuracy of algorithms increases with size of data – Statistical approaches can treat big datasets much better than traditional paradigms – Decision making using ML can adapt to transactional data much better © Harbinger Systems | www.harbinger-systems.com Machine Learning Big Data
  • 9. Fraud Detection: Did the user really do this login/make this purchase? Product Recommendation: Will the user like this product? Stock Trading: Will the stock go up or down? Medical Diagnosis: Given some symptoms, what is the patient suffering from © Harbinger Systems | www.harbinger-systems.com Machine Learning Applications
  • 10. © Harbinger Systems | www.harbinger-systems.com How to Categorize the Problem? Generally, machine learning problems looks to: Identify a Value Assign data points to a category Discover similarities between two data points
  • 11. © Harbinger Systems | www.harbinger-systems.com Flowchart Start Sufficient Data? Sort into category? Predict a value? Define Problem! Labeled Data Clustering Classification Get more! Regression
  • 12. © Harbinger Systems | www.harbinger-systems.com What to look for in algorithms: Flexible across many use cases Able to handle several input types Accurate Resistant to over-fitting/noise/error Machine Learning Algorithms
  • 13. © Harbinger Systems | www.harbinger-systems.com Random Forest Used for classification and regression Works on small subsets of data and combines the result into the best estimate XGBoost Works on classification and regression Starts off with a weak learner that improves over successive iterations K-Means Works on classification and clustering Tries to find boundaries between data points for each individual variable Machine Learning Algorithms
  • 14. © Harbinger Systems | www.harbinger-systems.com Tools and Technologies Emphasis on tools which Can integrate with existing data architecture Have a smooth learning curve Simplify the process of analysis and prediction Have an active community
  • 15. © Harbinger Systems | www.harbinger-systems.com Popular Machine Learning Tools Python Free, open-source, widely popular Consolidates many important libraries in python, C Has an active community Disclaimer: Brand names, logos and trademarks used herein remain the property of their respective owners.
  • 16. © Harbinger Systems | www.harbinger-systems.com Popular Machine Learning Tools R Statistical computing language that simplifies complex statistical operations Large number of libraries available for extending functionality (DB connectors, algorithm, visualization) Disclaimer: Brand names, logos and trademarks used herein remain the property of their respective owners.
  • 17. Open Data and Gov Services © Harbinger Systems | www.harbinger-systems.com • Open data, tools and resources available • ~181K datasets
  • 18. Sample Applications • City-Data provides detailed profiles of all U.S. cities - demographics, crime rates, home values, cost of living, etc. • Farmers can use Climate Corporation’s services to plan, manage, and protect crops • SPOT Crime : Free public facing crime mapping and alert website © Harbinger Systems | www.harbinger-systems.com
  • 19. Conclusion • Harness the power of your data to deliver higher value services and remain competitive –“Data is the currency of the future” – Michael Cockrill, CIO State of WA • Machine learning provides a powerful framework for extracting insights © Harbinger Systems | www.harbinger-systems.com
  • 20. Thank You © Harbinger Systems | www.harbinger-systems.com

Editor's Notes

  1. Applying data science to gain insights, improve efficiency and deliver higher value services. What skillsets, technologies and practices are required to deliver the best value? What you will learn What do you do with the data? What skillsets do you need in order to use the data? How to map data analytics to deliver higher value services and gain efficiencies?
  2. Retrospective analysis Dashboarding - Real-time processing Prediction #8 Optimization: How do we do things better? E.g. price optimization, markdown optimization and size optimization
  3. Big data forces you to wrestle with key strategic and operational challenges Find new ways to leverage information sources to drive growth improve your strategic decision making? You need to know which investments will deliver the most business value and ROI Are there new expectations for information quality and management Known, Known Unknowns and Unknown Unknowns (Insights)
  4. Tom Mitchell – Professor at the Carnegie Mellon University Automating Automata
  5. Adjusts for large amount of data
  6. Product Recommendation
  7. Regressional Analysis - regression analysis helps one understand how the typical value of the dependent variable (or 'criterion variable') changes when any one of the independent variables is varied, while the other independent variables are held fixed
  8. XGBoost is an optimized distributed gradient boosting system designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework http://dmlc.cs.washington.edu/xgboost.html K-Means - k-means clustering aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. This results in a partitioning of the data space into Voronoi cells (Lloyd's algorithm, also known as Voronoi iteration )
  9. https://www.data.gov/impact/ U.S. Postal Service was one of the early pioneers in implementing machine learning at a large scale – Reading postal addresses Fishing services Population Health Management Agriculture Crime mapping Education
  10. `