The document discusses how government agencies can use data analytics to address challenges and improve services. It provides examples of how the State of Indiana is using data analytics to combat the opioid epidemic, how the US Patent and Trademark Office made patent data more searchable, and how the Government Accountability Office tracks Medicare fraud. The document also discusses descriptive, diagnostic, predictive, and prescriptive analytics and how real-time and forensic analyses drive value. Overall, the document promotes how data analytics can help government agencies deliver better outcomes.
1) Anonymizing data is difficult and often destroys the utility of the data. High-dimensional data like images are nearly impossible to anonymize effectively.
2) Synthetic data generated by AI can provide a statistical representation of real private data while preserving over 99% of the value for analysis. It allows unrestricted sharing and use of data.
3) Case studies on public datasets like the US Census and mobility data show synthetic data matches the real data distribution and statistics closely while being fully private. Synthetic data enables new use cases for data sharing, analytics and innovation.
Government Data Exchange and Open Government Data PlatformAnveshi Gutta
Governments worldwide have heaps and heaps of data that is
accumulated every minute across different domains - transport,
traffic, public safety, weather, utilities, urban development et al.
This data is growing at a rapid rate as
Governments launch new services and attempt to drive more
inclusiveness for the existing services.
At the same time, Government agencies have been guilty of
working in silos and having very limited cross-agency visibility
and coordination. This observation magnifies itself in the
Emerging markets. In most cases, it is the citizen who bears
the brunt due to an absolute lack of citizen-centricity.
This presentation was delivered at Open Group Conference as an attempt to provide guidance on how governments can adopt a transformation journey that drives value generation from the data that has always been there.
This document provides an overview of predictive analytics and its growing importance. It discusses how advances in technologies like cloud computing and the internet of things are enabling businesses to gather and analyze vast amounts of data. While descriptive and diagnostic analytics describe what happened in the past, predictive analytics uses statistical techniques to create models that forecast future outcomes. The document outlines several key drivers that are pushing predictive analytics towards mainstream adoption over the next few years, including easier-to-use tools, open source software, innovation from startups, and the availability of cloud-based solutions. It concludes that the combination of big data and predictive analytics will continue to accelerate innovation across industries.
Governing Big Data : Principles and practicesPiyush Malik
This document summarizes key points from a special issue of the IBM Journal that focuses on applications, analytics, software, and hardware technologies for massive-scale analytics and processing big data. It begins with an introduction that defines big data and provides examples of how it is used. The remainder of the document provides summaries of 15 articles in the special issue that cover topics like governing big data, trends in massive-scale analytics stacks, designing systems for big data workloads, platforms for extreme analytics, and using big data for applications like social network analysis and underwater acoustic monitoring.
1. The U.S. Census Bureau faces challenges from the rise of big data sources produced outside of traditional government surveys. These new sources are generated faster and more cheaply than surveys.
2. To remain reliable sources of demographic and economic information, the Census Bureau must integrate these new big data sources with traditional surveys. This requires linking massive datasets and developing new statistical modeling techniques.
3. The Census Bureau is exploring ways to use new big data sources like web search data, social media, and e-commerce transactions to improve surveys and provide more timely, detailed information. However, maintaining privacy and developing new technology is difficult.
The document provides guidance on big data analytics. It discusses why big data analytics is important for companies to drive performance and results. It defines big data analytics as using analytics on large, diverse datasets to discover insights faster. The document then gives examples of how big data analytics has helped companies by increasing revenue, decreasing costs and time to insight, and improving customer acquisition, retention, and security. It addresses common questions around whether to buy or build a big data analytics solution.
1) Anonymizing data is difficult and often destroys the utility of the data. High-dimensional data like images are nearly impossible to anonymize effectively.
2) Synthetic data generated by AI can provide a statistical representation of real private data while preserving over 99% of the value for analysis. It allows unrestricted sharing and use of data.
3) Case studies on public datasets like the US Census and mobility data show synthetic data matches the real data distribution and statistics closely while being fully private. Synthetic data enables new use cases for data sharing, analytics and innovation.
Government Data Exchange and Open Government Data PlatformAnveshi Gutta
Governments worldwide have heaps and heaps of data that is
accumulated every minute across different domains - transport,
traffic, public safety, weather, utilities, urban development et al.
This data is growing at a rapid rate as
Governments launch new services and attempt to drive more
inclusiveness for the existing services.
At the same time, Government agencies have been guilty of
working in silos and having very limited cross-agency visibility
and coordination. This observation magnifies itself in the
Emerging markets. In most cases, it is the citizen who bears
the brunt due to an absolute lack of citizen-centricity.
This presentation was delivered at Open Group Conference as an attempt to provide guidance on how governments can adopt a transformation journey that drives value generation from the data that has always been there.
This document provides an overview of predictive analytics and its growing importance. It discusses how advances in technologies like cloud computing and the internet of things are enabling businesses to gather and analyze vast amounts of data. While descriptive and diagnostic analytics describe what happened in the past, predictive analytics uses statistical techniques to create models that forecast future outcomes. The document outlines several key drivers that are pushing predictive analytics towards mainstream adoption over the next few years, including easier-to-use tools, open source software, innovation from startups, and the availability of cloud-based solutions. It concludes that the combination of big data and predictive analytics will continue to accelerate innovation across industries.
Governing Big Data : Principles and practicesPiyush Malik
This document summarizes key points from a special issue of the IBM Journal that focuses on applications, analytics, software, and hardware technologies for massive-scale analytics and processing big data. It begins with an introduction that defines big data and provides examples of how it is used. The remainder of the document provides summaries of 15 articles in the special issue that cover topics like governing big data, trends in massive-scale analytics stacks, designing systems for big data workloads, platforms for extreme analytics, and using big data for applications like social network analysis and underwater acoustic monitoring.
1. The U.S. Census Bureau faces challenges from the rise of big data sources produced outside of traditional government surveys. These new sources are generated faster and more cheaply than surveys.
2. To remain reliable sources of demographic and economic information, the Census Bureau must integrate these new big data sources with traditional surveys. This requires linking massive datasets and developing new statistical modeling techniques.
3. The Census Bureau is exploring ways to use new big data sources like web search data, social media, and e-commerce transactions to improve surveys and provide more timely, detailed information. However, maintaining privacy and developing new technology is difficult.
The document provides guidance on big data analytics. It discusses why big data analytics is important for companies to drive performance and results. It defines big data analytics as using analytics on large, diverse datasets to discover insights faster. The document then gives examples of how big data analytics has helped companies by increasing revenue, decreasing costs and time to insight, and improving customer acquisition, retention, and security. It addresses common questions around whether to buy or build a big data analytics solution.
This document discusses the need for a new paradigm in big data analytics using algorithms. It begins by describing the limitations of traditional analytics approaches like statistical analysis, data mining, visualization and business intelligence tools when applied to big data. These approaches are query-based and labor intensive. Emerging big data tools like Hadoop and in-memory databases help with storage and queries but do not provide automated insights. The document argues that the new paradigm should focus on algorithms that can automatically surface insights from data in seconds, replacing the need for data analysts to manually query databases. This represents a shift from humans digging for insights to algorithms surfacing insights for humans to evaluate.
Big data refers to large datasets that are difficult to process using traditional tools. It is important for analyzing large amounts of data to gain insights and make better marketing decisions. Big data can help monitor social media sentiment around brands and forecast trends. While over half of people are unfamiliar with big data, most marketing leaders believe customer data is important for decisions. Big data is increasingly being used to measure corporate social responsibility by gathering stakeholder inputs and identifying external trends. Companies are using big data for social good through community outreach and contributory databases.
GGV Capital: Venture Investing and the Cloud (2012)GGV Capital
This document discusses venture investing in cloud computing. It provides an overview of why VCs continue to see opportunities in the cloud sector. The presentation agenda covers trends disrupting the cloud like mobile and big data, as well as opportunities in serving small and medium businesses. The document concludes with advice for cloud startups on effectively approaching VCs for funding, emphasizing differentiation, market size, scalability, financial model, and chemistry over legal terms.
Hadoop is an open-source software framework used for storing and processing large datasets in a distributed manner across commodity hardware. It was created in 2005 by Doug Cutting and Mike Cafarella to address the issue of processing big data at a reasonable cost and time. Hadoop uses HDFS for storage and MapReduce for processing data distributed over a network of nodes in parallel. It allows organizations to gain insights from vast amounts of structured and unstructured data faster and at lower costs than traditional approaches.
The document discusses 25 predictions about the future of big data:
1) Data volumes and ways to analyze data will continue growing exponentially with improvements in machine learning and real-time analytics.
2) More companies will appoint chief data officers and use data as a competitive advantage.
3) Data governance, visualization, and delivery through data fabrics and marketplaces will be key to extracting insights from diverse data sources and empowering partners.
4) Data is becoming a new global currency and companies are monetizing their data through algorithms, services, and by becoming "data businesses."
This document provides an overview and guided tour of big data. It begins with defining big data as any data collection that is too large or complex to be managed with traditional data approaches. It then discusses the key characteristics of big data, including volume, variety, and velocity. The document outlines different types of big data sources and discusses both the opportunities and risks of working with big data, emphasizing the importance of understanding the data quality, representativeness, and ensuring insights provide value. It concludes by discussing the need for big data analysis to be grounded in business questions and combining appropriate statistical and analytic techniques.
This talk is an introduction to Data Science. It explains Data Science from two perspectives - as a profession and as a descipline. While covering the benefits of Data Science for business, It explaints how to get started for embracing data science in business.
Gayatri Patel, eBay, presents at the Big Analytics 2012 Roadshow
The wonders of what data can do for an organization is measured in the productivity and competitiveness of their team's decisions. Some believe more data is the key. Agreed...but good decisions require more than just deriving intelligence from big data. In this dynamic market, the need to socialize and evolve ideas with other teams, quickly correlate information across sources, and test ideas to fail fast early are strong enablers to gain competitive footing. eBay¹s analytic and technology advancements garners insights and approaches that continue to help our employees tell their "data stories" and make better decisions.
Tools and techniques adopted for big data analyticsJOSEPH FRANCIS
This document discusses tools and techniques for big data analytics. It begins by defining big data and explaining why big data analysis is important for businesses. It then outlines the characteristics and history of big data, as well as the challenges and phases of big data analysis. The document proceeds to describe several tools and techniques used for big data analytics, including machine learning, natural language processing, and visualization. It provides examples of how these tools and techniques have been applied through case studies of Indian elections, AirBnB, and Shoppers Stop.
Oracle ACE Director Dan Morgan and PTC Chief Strategy Officer Mark Swanholm, presented this special webinar to discuss Big Data and the choices ahead for organizations. for more details about Performance Tuning Corporation, visit www.peftuning.com .
Organizations are being bombarded with messages telling you that you must make an investment in Big Data, that without it your organization will be rendered obsolete, a mere bystander, on the road to increased growth and profitability.
But do you? How exactly will your organization benefit from Big Data? When do you invest – and does investing in Big Data mean leaving the rest of your data strategy stranded?
Oracle ACE Director Dan Morgan, an internationally recognized expert in database technology and former University of Washington lecturer, and Mark Swanholm, PTC’s Chief Strategy Officer and 22 year IT Veteran, will address the issue of Big Data from the standpoint of what it is, where the value can be found, what is actually required to turn this new technology into something of value.
This Performance Tuning Corporation online event will focus on strategy, management, planning, and budgeting, and will provide you and your management team the information they need to plan make the best possible decision with respect to an investment in Big Data technology.
Now companies are in the middle of a renovation that forces them to be analytics-driven to
continue being competitive. Data analysis provides a complete insight about their business. It
also gives noteworthy advantages over their competitors. Analytics-driven insights compel
businesses to take action on service innovation, enhance client experience, detect irregularities in
process and provide extra time for product or service marketing. To work on analytics driven
activities, companies require to gather, analyse and store information from all possible sources.
Companies should bring appropriate tools and workflows in practice to analyse data rapidly and
unceasingly. They should obtain insight from data analysis result and make changes in their
business process and practice on the basis of gained result. It would help to be more agile than
their previous process and function.
This document discusses Hadoop and big data. It notes that digital data doubles every two years and that 85% of data is unstructured. Hadoop provides a cheaper way to store large amounts of both structured and unstructured data compared to traditional storage options. Hadoop also allows data to be stored first before defining what questions will be asked of the data.
This document provides an overview of how big data solutions from ViON and IBM can help organizations drive decision making, security, and insight. It discusses challenges of leveraging big data, and introduces ViON's DataAdapt platform which removes roadblocks to big data through pre-configured solutions. Specific DataAdapt solutions are presented that can optimize threat detection, eliminate cyber threats, extend capabilities without compromising security, and put powerful experts from ViON on the client's side.
The document discusses big data challenges faced by organizations. It identifies several key challenges: heterogeneity and incompleteness of data, issues of scale as data volumes increase, timeliness in processing large datasets, privacy concerns, and the need for human collaboration in analyzing data. The document describes surveying various organizations in Pakistan, including educational institutions, telecommunications companies, hospitals, and electrical utilities, to understand the big data problems they face. Common challenges included data errors, missing or incomplete data, lack of data management tools, and issues integrating different data sources. The survey found that while some organizations used big data tools, many educational institutions in particular did not, limiting their ability to effectively manage and analyze their large and growing datasets.
Why Data Science is Getting Popular in 2023?kavyagaur3
Data science employs mathematics, statistics, advanced programming techniques, analytics and artificial intelligence (AI) to uncover insights that drive business value for their organisation. Then, this information can be used for strategic planning and decision-making.
Data has flooded in massive amounts as a result of digitization. Businesses are making their utmost efforts to take advantage of every opportunity to increase their businesses. This makes the best opportunity for individuals who want to pursue Data Science. The first step is to get the best data science training.
Unveiling the Power of Data Analytics Transforming Insights into Action.pdfKajal Digital
Data analytics is the process of examining raw data to discover patterns, correlations, trends, and other valuable information. Its significance lies in its ability to transform data into actionable insights, ultimately leading to informed decision-making and improved business outcomes. From optimizing operational processes to enhancing customer experiences, data analytics offers a plethora of benefits across various sectors.
This document discusses the need for a new paradigm in big data analytics using algorithms. It begins by describing the limitations of traditional analytics approaches like statistical analysis, data mining, visualization and business intelligence tools when applied to big data. These approaches are query-based and labor intensive. Emerging big data tools like Hadoop and in-memory databases help with storage and queries but do not provide automated insights. The document argues that the new paradigm should focus on algorithms that can automatically surface insights from data in seconds, replacing the need for data analysts to manually query databases. This represents a shift from humans digging for insights to algorithms surfacing insights for humans to evaluate.
Big data refers to large datasets that are difficult to process using traditional tools. It is important for analyzing large amounts of data to gain insights and make better marketing decisions. Big data can help monitor social media sentiment around brands and forecast trends. While over half of people are unfamiliar with big data, most marketing leaders believe customer data is important for decisions. Big data is increasingly being used to measure corporate social responsibility by gathering stakeholder inputs and identifying external trends. Companies are using big data for social good through community outreach and contributory databases.
GGV Capital: Venture Investing and the Cloud (2012)GGV Capital
This document discusses venture investing in cloud computing. It provides an overview of why VCs continue to see opportunities in the cloud sector. The presentation agenda covers trends disrupting the cloud like mobile and big data, as well as opportunities in serving small and medium businesses. The document concludes with advice for cloud startups on effectively approaching VCs for funding, emphasizing differentiation, market size, scalability, financial model, and chemistry over legal terms.
Hadoop is an open-source software framework used for storing and processing large datasets in a distributed manner across commodity hardware. It was created in 2005 by Doug Cutting and Mike Cafarella to address the issue of processing big data at a reasonable cost and time. Hadoop uses HDFS for storage and MapReduce for processing data distributed over a network of nodes in parallel. It allows organizations to gain insights from vast amounts of structured and unstructured data faster and at lower costs than traditional approaches.
The document discusses 25 predictions about the future of big data:
1) Data volumes and ways to analyze data will continue growing exponentially with improvements in machine learning and real-time analytics.
2) More companies will appoint chief data officers and use data as a competitive advantage.
3) Data governance, visualization, and delivery through data fabrics and marketplaces will be key to extracting insights from diverse data sources and empowering partners.
4) Data is becoming a new global currency and companies are monetizing their data through algorithms, services, and by becoming "data businesses."
This document provides an overview and guided tour of big data. It begins with defining big data as any data collection that is too large or complex to be managed with traditional data approaches. It then discusses the key characteristics of big data, including volume, variety, and velocity. The document outlines different types of big data sources and discusses both the opportunities and risks of working with big data, emphasizing the importance of understanding the data quality, representativeness, and ensuring insights provide value. It concludes by discussing the need for big data analysis to be grounded in business questions and combining appropriate statistical and analytic techniques.
This talk is an introduction to Data Science. It explains Data Science from two perspectives - as a profession and as a descipline. While covering the benefits of Data Science for business, It explaints how to get started for embracing data science in business.
Gayatri Patel, eBay, presents at the Big Analytics 2012 Roadshow
The wonders of what data can do for an organization is measured in the productivity and competitiveness of their team's decisions. Some believe more data is the key. Agreed...but good decisions require more than just deriving intelligence from big data. In this dynamic market, the need to socialize and evolve ideas with other teams, quickly correlate information across sources, and test ideas to fail fast early are strong enablers to gain competitive footing. eBay¹s analytic and technology advancements garners insights and approaches that continue to help our employees tell their "data stories" and make better decisions.
Tools and techniques adopted for big data analyticsJOSEPH FRANCIS
This document discusses tools and techniques for big data analytics. It begins by defining big data and explaining why big data analysis is important for businesses. It then outlines the characteristics and history of big data, as well as the challenges and phases of big data analysis. The document proceeds to describe several tools and techniques used for big data analytics, including machine learning, natural language processing, and visualization. It provides examples of how these tools and techniques have been applied through case studies of Indian elections, AirBnB, and Shoppers Stop.
Oracle ACE Director Dan Morgan and PTC Chief Strategy Officer Mark Swanholm, presented this special webinar to discuss Big Data and the choices ahead for organizations. for more details about Performance Tuning Corporation, visit www.peftuning.com .
Organizations are being bombarded with messages telling you that you must make an investment in Big Data, that without it your organization will be rendered obsolete, a mere bystander, on the road to increased growth and profitability.
But do you? How exactly will your organization benefit from Big Data? When do you invest – and does investing in Big Data mean leaving the rest of your data strategy stranded?
Oracle ACE Director Dan Morgan, an internationally recognized expert in database technology and former University of Washington lecturer, and Mark Swanholm, PTC’s Chief Strategy Officer and 22 year IT Veteran, will address the issue of Big Data from the standpoint of what it is, where the value can be found, what is actually required to turn this new technology into something of value.
This Performance Tuning Corporation online event will focus on strategy, management, planning, and budgeting, and will provide you and your management team the information they need to plan make the best possible decision with respect to an investment in Big Data technology.
Now companies are in the middle of a renovation that forces them to be analytics-driven to
continue being competitive. Data analysis provides a complete insight about their business. It
also gives noteworthy advantages over their competitors. Analytics-driven insights compel
businesses to take action on service innovation, enhance client experience, detect irregularities in
process and provide extra time for product or service marketing. To work on analytics driven
activities, companies require to gather, analyse and store information from all possible sources.
Companies should bring appropriate tools and workflows in practice to analyse data rapidly and
unceasingly. They should obtain insight from data analysis result and make changes in their
business process and practice on the basis of gained result. It would help to be more agile than
their previous process and function.
This document discusses Hadoop and big data. It notes that digital data doubles every two years and that 85% of data is unstructured. Hadoop provides a cheaper way to store large amounts of both structured and unstructured data compared to traditional storage options. Hadoop also allows data to be stored first before defining what questions will be asked of the data.
This document provides an overview of how big data solutions from ViON and IBM can help organizations drive decision making, security, and insight. It discusses challenges of leveraging big data, and introduces ViON's DataAdapt platform which removes roadblocks to big data through pre-configured solutions. Specific DataAdapt solutions are presented that can optimize threat detection, eliminate cyber threats, extend capabilities without compromising security, and put powerful experts from ViON on the client's side.
The document discusses big data challenges faced by organizations. It identifies several key challenges: heterogeneity and incompleteness of data, issues of scale as data volumes increase, timeliness in processing large datasets, privacy concerns, and the need for human collaboration in analyzing data. The document describes surveying various organizations in Pakistan, including educational institutions, telecommunications companies, hospitals, and electrical utilities, to understand the big data problems they face. Common challenges included data errors, missing or incomplete data, lack of data management tools, and issues integrating different data sources. The survey found that while some organizations used big data tools, many educational institutions in particular did not, limiting their ability to effectively manage and analyze their large and growing datasets.
Why Data Science is Getting Popular in 2023?kavyagaur3
Data science employs mathematics, statistics, advanced programming techniques, analytics and artificial intelligence (AI) to uncover insights that drive business value for their organisation. Then, this information can be used for strategic planning and decision-making.
Data has flooded in massive amounts as a result of digitization. Businesses are making their utmost efforts to take advantage of every opportunity to increase their businesses. This makes the best opportunity for individuals who want to pursue Data Science. The first step is to get the best data science training.
Unveiling the Power of Data Analytics Transforming Insights into Action.pdfKajal Digital
Data analytics is the process of examining raw data to discover patterns, correlations, trends, and other valuable information. Its significance lies in its ability to transform data into actionable insights, ultimately leading to informed decision-making and improved business outcomes. From optimizing operational processes to enhancing customer experiences, data analytics offers a plethora of benefits across various sectors.
Practical analytics john enoch white paperJohn Enoch
This document discusses using data analytics to provide value to businesses. It recommends starting with smaller, more manageable data sets and business intelligence (BI) projects that have clear goals and can yield quick wins, like analyzing travel costs. While big data holds promise, the author advises focusing first on consolidating existing data that is stuck in silos and using BI to improve processes and save costs in areas employees already know need improvement. Starting small builds skills for larger initiatives and ensures analytics provides practical benefits.
Big data analytics involves capturing, storing, processing, analyzing, and visualizing huge quantities of information from a variety of sources. This data is characterized by its volume, variety, velocity, veracity, variability, and complexity. Traditional analytics are not suited to handle big data due to its size and constantly changing nature. By analyzing patterns in big data, businesses can gain insights to improve processes and campaigns. However, specialized software is needed to make sense of big data's different types and formats from numerous sources. The right big data solution depends on an organization's specific data, budgets, skills, and future needs.
Big data analytics involves analyzing large datasets to identify patterns. KCC has two categories of big data - individual large datasets like sensor data, and aggregated datasets that combine transactional, customer, and other data. Big data analytics can help streamline transactions, profile customers, analyze locations, investigate social media, and track performance. For KCC to utilize big data analytics, it needs to audit information assets, improve data integration, develop analytics skills, ensure governance and consent policies, and ensure data quality.
The document discusses the concept of "Democratizing Data" which involves making data accessible, structured, and syndicated so it can be used by various applications and devices. This approach integrates transparency with efficiency, streamlined reporting, "smart regulation", and crowdsourcing. Democratizing data provides benefits like more informed policymaking, consensus building, better legislation, transparency, and lower costs. It also allows citizens to become "co-creators" by developing applications using open government data. The document advocates switching to a more data-centric approach across government and private organizations.
1) The document discusses how life sciences organizations are dealing with large amounts of data from various sources (big data) and the challenges this presents for data governance.
2) It recommends that organizations take a "democratized" approach to data governance, involving various business functions rather than just a centralized group.
3) Key aspects of data governance like organization structure, metadata management, and data security need to be realigned to accommodate big data through expanded roles and use of new technologies.
The document discusses how advanced analytics are disrupting marketing by enabling more targeted and personalized strategies. It provides an introduction to an e-book compiling essays from data analytics experts in different fields and industries about how they are applying big data analytics. The essays are grouped into five sections covering topics like how analytics are changing businesses, new technology platforms, industry examples, research applications, and marketing strategies.
Data mining allows companies to analyze large amounts of customer data to discover patterns and trends that can help target new customers and increase profits. It involves extracting, transforming, and storing transaction data, then analyzing it to find useful business insights. Popular data mining algorithms include statistical analysis, neural networks, and nearest neighbor methods. While data mining provides benefits, privacy is a concern as customer information may be shared with third parties without consent.
Machine learning algorithms analyze large amounts of data to identify patterns and make predictions. There are two main types: supervised learning predicts outcomes based on historical labeled data, while unsupervised learning identifies structure in unlabeled data to group it into categories. Effective machine learning requires choosing the right algorithm for the specific data and application, and allows creating intelligent processes that generate useful predictions for a data-driven world.
Big Data & Investment Management: The Potential to Quantify Traditionally Qua...Ken Cutroneo
Big data is a catchphrase for a new way of conducting analysis. Big data principles are being adopted across many industries and in many varieties. However, adoption so far by investment managers has been limited. This may be creating a window of opportunity in the industry.
Big Data & Business Analytics: Understanding the MarketspaceBala Iyer
This document provides an overview of big data and business analytics. It discusses the growth of data and importance of analytics to businesses. The key topics covered include defining big data and data science, analyzing the analytics ecosystem and key players, examining use cases of analytics at companies like Target and Whirlpool, and providing recommendations for building an analytics capability and working with analytics vendors. The presentation emphasizes how data-driven decisions can improve business performance but also notes challenges to overcome like skills shortages and changing organizational culture.
This document discusses big data and its applications in various industries. It begins by defining big data and its key characteristics of volume, velocity, variety and veracity. It then discusses how big data can be used for log analytics, fraud detection, social media analysis, risk modeling and other applications. The document also outlines some of the major challenges faced in the banking and financial services industry, including increasing competition, regulatory pressures, security issues, and adapting to digital shifts. It concludes by noting how big data analytics can help eCommerce businesses make fact-based, quantitative decisions to gain competitive advantages and optimize goals.
Thinking Small: Bringing the Power of Big Data to the MassesFlutterbyBarb
Thinking Small: Bringing the Power of Big Data to the Masses via Adobe with the results of improved access to insights, better user experiences, and greater productivity in the enterprise.
The document discusses big data analytics and creating a big data-enabled organization. It begins with an introduction to big data, defining it and explaining its four V's: volume, variety, velocity, and veracity. It then discusses big data analytics, explaining that it involves more than just data and requires methods like machine learning. The document provides examples of big data analytics in various industries and development contexts. It concludes by outlining three steps to creating a big data-enabled organization: 1) be clear on the specific questions and needs big data can address, 2) build an integrated foundation of data, tools, and skills, and 3) establish a culture of experimentation and learning from failures.
Running title TRENDS IN COMPUTER INFORMATION SYSTEMS1TRENDS I.docxanhlodge
Running title: TRENDS IN COMPUTER INFORMATION SYSTEMS 1
TRENDS IN COMPUTER INFORMATION SYSTEMS 4
Trends in Computer Information Systems, and the Rise to Business Intelligence
Shad Martin
School for Professional Studies
St. Louis University
ENG 2005 Dr. Rebecca Wood
November 23, 2016
Introduction
Our quest to increase our knowledge of Computer Information Systems has produced a number of benefits to humanity. The innovation humans have discovered in Computer Information Systems has led to new sub-areas of study for students and professionals to continue their progression to master all that Computer Information Systems has to offer. Amy Web of the Harvard Business Review reported 8 Tech Trends to Watch in 2016, She noted, “In order to chart the best way forward, you must understand emerging trends: what they are, what they aren’t, and how they operate. Such trends are more than shiny objects; they’re manifestations of sustained changes within an industry sector, society, or human behavior. Trends are a way of seeing and interpreting our current reality, providing a useful framework to organize our thinking, especially when we’re hunting for the unknown. Fads pass. Trends help us forecast the future” (Harvard Business Review, 2015). In short, Amy’s reference to understanding the emerging trends in Computer Information can provide a framework from which, students, professionals, and scientists to conscientiously create a path towards optimizing their efforts. Ensuring we have a fundamental approach to analyze data will enhance our understanding of this subject further.
In this paper I will expound on three of the top trends used to provide insight into the data produced from the advancements in Computer Information Systems. These trends or methods are taking place in my workplace within a financial institution, and in many other industries. It is important to note this paper does not provide an inclusive list of all methodologies that exist. Individuals can now leverage analytics to synthesize insights from data to identify emerging risk, manage operational risks, identify trends, improve compliance, and customer satisfaction. Data in and by itself is not always useful. Regardless of the data source, trained professional must understand the best approach to structure the data to make it more useful. In this paper, I will touch on three popular methodology trends occurring in Computer Information Systems. Students and professionals who work with large data would benefit from having a solid understanding of the fundamental principles of Business Intelligence as data scientific approach and when to use these methodologies.
The rise of Business Intelligence
Computer Information Systems allow many companies to gather and generate large amounts of data on their customers, business activities, potential merger targets, and risks found in their organization. These large sets of data have given rise to vari.
Running title TRENDS IN COMPUTER INFORMATION SYSTEMS1TRENDS I.docx
how you can use data analytics
1. HOW YOU CAN USE
DATA ANALYTICS
TO CHANGE
GOVERNMENT
2. 2 | A GOVLOOP GUIDE
2 | What Is Data
Analytics?
7 | How to Make Data
Analytics User-Friendly
11 | How Data Analytics
Drives Real-Time and
Forensic Analyses
15 | Accelerating
Insights with Multi-
Tiered Flash Storage
19 | The Future of Data
Analytics in
Government IT
23 | In the World of
Data Analytics, Speed
Is Everything
27 | 4 Keys to a
Successful Data
Analytics Program
31 | Data Analytics
as an Asset
34 | 3 Reasons Privacy
Should Be Top of Mind
35|WhatYouCanDo
Now:HowtoCreatea
CultureforDataAnalytics
37 | About Govloop &
Acknowledgments
4 | Fighting the
Opioid Epidemic
16 | Improving Human
Capital Management
28 | Improving Mail
Delivery
8 | Making Patent
Data Searchable
12 | Tracking Medicare
Fraud
24 | Predicting and
Reducing Crime
20 | Empowering
Cyber Analysts
32 | Enhancing IT
Investment Tracking
CONTENTS
3. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 1
Everything about Dr. Farid Fata’s case
seemed too egregious to be true.
When -
ment of Justice back in 2013
unprecedented medical malpractice, govern-
How could a renowned Michigan oncologist
intentionally misdiagnose patients and then
inject them with chemotherapy treatments?
And how could he intentionally underdose
patients who did in fact have cancer?
In total, 553 patients and their loved ones
organ damage, missing teeth, nerve damage
Because of the nature of the harm, govern-
-
ing him to justice took a brave whistleblower,
government collaboration — and reliable
Brzymialkiewicz, Assistant Inspector General
-
tively, nonstop, and pulled enough evidence
together to take quick action, including the
When this case came in, the team checked
sets, including claims data, Brzymialkiewicz
-
story because, in terms of detecting fraud,
Healthcare is one of the key areas in govern-
ment where data analytics is driving value,
government employees are using data ana-
human capital analytics to inform human
there are countless other examples, several
costs, and help government deliver better cit-
important today for you to learn how and why
•
• Identify various use cases at the fed-
• Understand the privacy concerns
• -
But before we dive into the meat of this
-
EXECUTIVE SUMMARY
4. 2 | A GOVLOOP GUIDE
“
WHAT IS DATA ANALYTICS?
Is it big data, data analytics, or just analytics?
Big
:
of the data and turn that insight into action.
-
Analytics is what makes big data come alive.
Without analytics, big datasets could be stored, and they could
be retrieved, wholly or selectively. But what comes out would
be exactly what went in. Analytics, comprising a number of dif-
ferent computational technologies, is what fuels the big-data
revolution. Analytics is what creates the new value in big data-
sets, vastly more than the sum of the values of the parts.”
5. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 3
4 TYPES OF DATA ANALYTICS
ANALYTIC VALUE ESCALATOR
APPLY IT NOW
-
-
1What happened?
2Why did it happen?
3What will happen?
4How can we make
it happen?
DESCRIPTIVE
ANALYTICS
What happened?
DIAGNOSTIC
ANALYTICS
Why did it happen?
PREDICTIVE
ANALYTICS
What will happen?
PRESCRIPTIVE
ANALYTICS
How can we make it happen?
D I F F I C U L T Y
I N F O R M A T I O N
O P T I M I Z A T I O N
VALUE
Using Google Analytics to track
website traffic, such as page
views and number of visitors
Mining data to determine
what caused a spike in Web
traffic over the past month
Using algorithms and data
to predict where burglaries
will most likely occur in a city
Rerouting travel, based on traffic
predictions and pedestrians’
actions (think self-driving cars)
6. 4 | A GOVLOOP GUIDE
The State of Indiana
An interview with Dave Matusoff & Joshua Martin
Director for the Indiana Management & Performance Hub; MPH Chief of Staff
In Indiana, news stories about Hoosiers who are addicted to opioid are
-
according
, “found her son unconscious in a heroin-in-
Although personal stories humanize the drug epidemic, stories alone
lawmakers, and the community need sound data to shape their collec-
a new business intelligence tool to help its agency customers make
-
teresting if the state could use data to better understand the changing
nature of the drug problem?
Historically, decision-makers have had to rely on whatever information
was available to make the tough choices, but what they had access to
-
THE CHALLENGE: Fighting the Opioid Epidemic
7. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 5
1 2 3
It has only been a few months since MPH stood up its drug data dash-
board internally and began enlisting the help of other agencies — the
laboratories to track and visualize the types of illegal substances that
in heroin submissions to the lab but a steady decline in other opioid
2013 state law aimed at tackling the opioid
epidemic as a contributing factor in the decline of prescription opioid
prescribe and issue narcotics, and reduce opioid abuse and ultimately
“When they passed that legislation, it was harder for addicts to get
the number of prescription narcotics submitted to the forensics lab,
begin to look at — in a very real-time way — the unintended conse-
and soliciting data and input from law enforcement and public health
by creating geospatial maps, color-coordinated line graphs with trend
data and even a time-lapse map that shows how the drug epidemic
team is also looking at opioid-related deaths and geospatial data for
-
ported
information, turn that information into knowledge, and hopefully turn
SUCCESS FACTORS
THE SOLUTION
Think through your strategy. Create a
proof of concept, or partner with other
organizations that have already made
investments.
Rethink how you can use your data in
new and meaningful ways.
Solicit the help of administration
officials to break down data-sharing
silos among agencies.
“What we’re trying to do here is take these mountains of data that are in
different places and turn data into information, turn that information into
knowledge, and hopefully turn that knowledge into policy.”
9. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 7
How to Make Data Analytics User-Friendly
An interview with Rob Lindsley
Director of Data Analytics, Oracle Public Sector
INDUSTRY SPOTLIGHT
By now, data analytics has become one of the
-
ant aspects of data analytics is its ability to
integrate with existing systems, in terms of
data sources, applications, infrastructure, as
about ways that organizations can work to
make data analytics as user-friendly and
focused on providing organizations with a va-
riety of cloud options — allowing for self-ser-
vice analytics and data visualization — and
presenting big data and predictive analysis
the strategy is to give customers “access to
the data they need paired with the analytics
can run analytic solutions in their own data-
or agencies can have third parties run their
should be easily available to consumers both
-
ment agencies as running the gamut from
human resources to mission support func-
presence in the central operations of many
government agencies, but we also have a lot
of customers using analytics to accomplish
-
eas, including transportation, a child services
analytics to meet their objectives quickly and
-
on-premise deployment model or in the
change between on-premise deployment
and cloud deployment as projects develop
example, an agency might test a new system
in the cloud and then move it to an on-prem-
data transparency
A variety of data sources are essential to
-
warehouses or other standard data models,
but increasingly government employees need
to integrate their own data into the analysis,
whether through an Excel spreadsheet or a
allow end-users to incorporate a variety of
Agencies are also pushing to instill a culture
data analytics and predictive analysis have
they can be used without a steep learning
information on the services it provides, and it
also has access to information about external
harnessing the power of big data analytics is
-
and the weather, this agency can optimize the
travel experience, make best use of limited
resources and increase the safety of drivers
data to improve current projects and predict
Army, which for more than a decade has
with straightforward metrics to evaluate unit
readiness and then moved toward descriptive
the Army is using predictive analysis to deter-
mine what types of units will be required in
the democratization of data access, the
usability of modern analytics tools, and easy
initiatives can help agencies address a variety
examples remind us what can be achieved
when we can seamlessly incorporate data
10. 8 | A GOVLOOP GUIDE
We can all agree that managing a growing database of more than
online search tool,
-
work, that document the transfer of a patent from a previous owner to
of the previous owner, the new owner, their addresses and the type of
have in our database open for searching, and it really was just a data-
see all the assignments from Motorola to Google using the old system,
alone produces 685 assignments; and each one would have to be
THE CHALLENGE: Making Patent Data Searchable
U.S. Patent & Trademark Office
An interview with Ted Parr
Director of the Public Information Services Group within the Office of the CIO at USPTO
11. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 9
1 2 3
-
software to provide indexed searching rather than the slower, more
transfers and reduce abusive patent litigation and lawsuits stemming
-
cluding execution date, conveyance type, correspondent and assignee
address, as well as allowing wild card and Boolean searches for greater
the new system, users simply enter both names in their respective
from Motorola to Google, rather than having to sift through more than
“In addition, the new tool will have a true data export feature in its next
-
cation programming interface that allows users to build their own tools
can also now view the underlying legal documents related to their
As a major transparency initiative, it drew interest from the White
-
and his team used agile development to add user capabilities in rapid
-
ments and other users to incorporate their feedback; and they were
SUCCESS FACTORS
THE SOLUTION
Get high-level officials
involved early on.
Let the agency’s business units share
in the decision-making.
Explore agile development &
actively participate in daily scrum
& project meetings.
6.7 6k 3k 4M I L L I O N
R E C O R D S
U S E R S
P E R W E E K
S E A R C H E S
P E R D A Y
M O N T H
T I M E L I N E
“The net result is that customers can be very precise with their searches and
can filter and analyze results in many different ways.”
12. 10 | A GOVLOOP GUIDE
Stop Cyber Threats
In Their Tracks
Put Big Data Analytics to Work
in Your Cyber Environment.
ViON’s DataAdapt S2000 Cyber
Secure analyzes large volumes of
sensor data in near-real time to:
• Identify and thwart cyber attacks
• Eliminate insider threat
vulnerabilities
• Provide actionable predictions
for Advanced Persistent Threats
(APTs)
LEARN MORE
www.vion.com/Solutions/
Cyber-Security.aspx
13. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 11
How Data Analytics Drives Real-Time and
Forensic Analyses
An interview with Billy Hill
Customer Solution Architect for Big Data Analytics at ViON
INDUSTRY SPOTLIGHT
-
cheaper as the manufacturing processes
more familiar with the ecosystem of analytics
and data storage technology, said Billy Hill,
-
ability and usability bring analytics tools within
reach for government agencies, businesses
“Although the technology has been around
-
concert, and these technologies are start-
of researchers and mathematicians, who
are helping to drive changes in the types of
time, the algorithms are also dictating how
-
As the technology advances and provides
better insights, the demand for data analytics
and the outcomes they produce continues to
of analytics, such as predictive and prescrip-
tive to better understand what may happen
to conduct — real-time or historical — there
closer look at each of these examples to
better understand what they entail and how
hard to say with certainty what will happen
But agencies can use the data they have to
helps agencies plan accordingly in a variety of
-
er demands for services or internal human
-
“If agencies are able to predict the probability
of an event or action occurring, that infor-
-
you have to be informed, and the better your
and batch refer to the types of analytics
-
gies are usually used to power real-time data
analysis that require immediate data pro-
involve the use of sensors to rapidly collect
Batch analytics, on the other hand, is used in
-
more time to tweak their models for more
doing batch, where as with real-time analysis
there are online algorithms making decisions
-
ing analytics supports real-time analysis, while
Both real-time and forensic analyses provide
great value to agencies, depending on their
analysis is helpful when decisions are based
on quickly evolving data, such as current
more appropriate option for driving forensic
analyses looking at historical data, such as
-
responders would rely on real-time analysis
to determine how far an ambulance is from
a victim, and how far the victim is from the
same data may be used to better understand
which parts of the city receive the most 9-1-1
calls and where ambulances should be sta-
just out of reach, and some people think
agencies have all of these capabilities and
14. 12 | A GOVLOOP GUIDE
Government Accountability Office
An interview with Seto Bagdoyan
Director of Audit Services for GAO’s Forensic Audits & Investigative Service
percent — of that amount was paid improperly, according to estimates
Ineligible providers and suppliers intent on defrauding the gov-
ernment have used all sorts of schemes to enroll in the Medicare
-
A June 2015 report
addresses listed as legitimate practice locations that were later traced
to mailbox rental stores, a construction site, a fast-food franchise and
But how do these blatant cases of fraud slip under the radar? Ac-
suppliers, particularly those with ineligible addresses, requires months
-
going to perform, and then write the appropriate program using tools
results, and verify those results through due diligence before reaching
THE CHALLENGE: Tracking Medicare Fraud
15. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 13
1 2 3
-
-
-
make sure that the emerging results are indeed results, rather than
“By looking at a lot of data and drilling down to the appropriate level, we
-
ily available Internet-based tools and on-the-ground follow-up to verify
-
punch way above our weight because we have the determination to
SUCCESS FACTORS
THE SOLUTION
Ensure data is clean, usable, & reliable
as it’s collected.
Institute multiple levels of review
to double & triple check that the
data is secure.
Adhere to policies & regulations that
govern data collection & publication.
“We’re a very small agency, and my team...we punch way above our weight
because we have the determination to do a good job with the data and the
tools we have.”
$ 5 5 4 , 0 0 0 , 0 0 0 , 0 0 0 paid by Medicare for healthcare
& related services in FY2014
11percent
($60billion!)
was paid
improperly
23,4001teamof GAO employees
1million addresses
addresses that did not
meet federal requirements
sorted
through
and
found
17. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 15
Accelerating Insights with
Multi-Tiered Flash Storage
An interview with Chris Tsilipounidakis & Jason Strawderman
Manager of Product Marketing and Vice President, Federal, at Tegile
INDUSTRY SPOTLIGHT
At its core, big data analytics is about
converting data into knowledge that can be
used to make informed decisions, which
can have a variety of positive implications to
defense to accelerate recognition of pat-
of all types to predict and prevent potential
cyberattacks before they occur, said Jason
-
“Government healthcare organizations are
using big data to manage, analyze, visual-
ize, and extract information that is helping
government are tremendous, the implemen-
tation is proving very challenging as legacy
infrastructure cannot handle the demanding
workload, and disparate systems prevent the
-
ture and transform culture, policies and pro-
to the government; many organizations in
With this transformation comes the need
storage infrastructure silos while improving
performance and increasing capacity, said
-
-
used certain vendors to store mission-
critical data on more expensive, high-pow-
ered storage systems, other vendors to store
less important data on lower-cost storage,
while using more general-purpose systems
for workloads that require an even blend of
“Big data analytics is causing a massive
more requirements to do the same thing
that I was doing before, I need to do it in a
-
lated with a mix of solid-state and hard-disk
-
the workload demands of big data analytics,
-
performance than they are getting from their
legacy storage systems and at a fraction of
-
-
agencies can get better performance and
per gigabyte of traditional hard drives, and
In terms of data management, security is also
federal standards for encrypting data at rest
-
“Most government agencies are migrating to
— or already operating — hybrid cloud archi-
tectures consisting of both private cloud and
existing environments, without them having
When it comes to choosing cloud or tradi-
However, one area where a model like cloud
can be less viable is when agencies want to
process data in real-time and take quick ac-
tions, as opposed to reviewing large amounts
of historical data for less time-sensitive
other workloads, such as virtualization, virtual
desktop infrastructure and archives and
backup, as older legacy systems are retired
infrastructure, and no one has the budget
analytics, server consolidations, desktop
virtualization and mobility projects are a few
18. 16 | A GOVLOOP GUIDE
General Services Administration
An interview with Paul Tsagaroulis
Director of Human Capital Analytics at GSA
-
“A big challenge for us is data management and information manage-
leaves very little time for us to analyze the data and look at trends and
patterns, maybe do some sophisticated analytics, and some predictive
and they all performed the same data analytics services, multiple
-
-
vision to support this for the agency, to alleviate the need for business
THE CHALLENGE: Improving Human Capital Management
19. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 17
1 2 3
new hires and separations by grade, onboarding by major job series,
“We dramatically reduced ad hoc reporting and responding to re-
agency was increasing its costs and diminishing its talent development
people we would expect to have retire or voluntarily separate from
the agency, and we could calculate the cost of the attrition savings
by automating reporting of turnover, onboarding, and performance
employ prescriptive analytics and help managers develop action plans
SUCCESS FACTORS
THE SOLUTION
Tackle data management issues first.
Use visualization techniques to explain
trends & other findings in the data.
Let workforce data dictate new HR
solutions or programs that maximize
employee potential.
“Looking further into turnover data, we can predict with some confidence the
number of people we would expect to have retire or voluntarily separate from
the agency.”
20. 18 | A GOVLOOP GUIDE
Beyond Big Data to Data-Driven Insights
More effective government sits at the intersection of insight
and action.
When you can collect, unify, and analyze all the data that surrounds your
department, government is empowered to uncover key insights that matter
most. It could be an insight that changes the way citizens view your agency,
or how your department sees their role. Learn how big data and analytics
for smart government can help you know more so you can do more in this
informative white paper.
Whitepaper: Analytics in Service of Smart Government
What would you do if you knew? is a trademark, and Teradata and the Teradata logo are registered trademarks of Teradata Corporation and/or its affiliates in the U.S. and worldwide.
What would you do if you knew?™
21. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 19
The Future of Data Analytics in
Government IT
An interview with Alan Ford
Director of Presales Consulting, Teradata Government Systems
INDUSTRY SPOTLIGHT
We may be in for a few surprises, but one
analytics will drive more value in government,
including data transparency, tax fraud, and
these larger datasets are coming at govern-
ment agencies at greater speeds than ever
more data to analyze, government agen-
cies can easily be overwhelmed as they try
to maintain system performance and data
integrity while protecting sensitive informa-
can leverage data analytics, including those in
the cloud, to generate actionable information
and enable better data sharing within and
services like data warehousing to enable
platforms into a comprehensive analytics
solution that enables fast, deep, and powerful
With this architecture and its inherent analytic
capability, agencies can complement and
augment their legacy systems rather than
Healthcare comprises some of the largest
annual expenditures in both the private and
-
ernment funding, which makes it an attractive
with increased data digitization and analytics
“By combining traditional claims data with
other separate but publicly available data –
like bankruptcies, liens, judgments, or even
repeated address changes – into an integrat-
we can run advanced analytics that make
fraud easier to track or even proactively pre-
tool that helps run analytics that could only
we can run older transcribed handwritten
records or typed reports through complex
text analytics capabilities and sentiment
analysis in an automated fashion dramatically
“And with less expensive storage accessible
by sophisticated techniques, all within the
analyze all the data rather than just represen-
data analytics and cloud will make a strong
such technologies, so agencies can do more
-
ment is a huge cost saver, because no longer
do organizations have to set up and main-
way, agencies convert acquisition costs into
-
ever, may seem improbable for government
have spent a tremendous amount of money
acquiring systems over time, they are reluc-
which allows for the harnessing of all of an
the logical data warehouse because it brings
the right type of analytical processing to the
to combine and analyze data from multiple
A logical data warehouse can subsume older
platforms so agencies that choose not to trans-
form and migrate data into an integrated data
accompany legacy systems, complemented
by cloud technologies, government agencies
can better manage the vast amounts of data
of the cloud, government can be ready for
advanced analytical capabilities and harness
22. 20 | A GOVLOOP GUIDE
Defense Information Systems Agency
An interview with Dan Bart & Christopher Paczkowski
Chief of Cyber Situational Awareness Systems; Cyber Situational Awareness &
Analytics Division Chief
that may be nefarious, including malicious insiders and nation states
billions of unsuccessful hack attempts that agencies had to thwart
to see the anomalous behavior and connections between various
alerted an analyst to anomalous activities, he or she could then open
an investigation and look at other sensors to try to draw connections
-
But analysts have to distinguish between anomalous activity that is
THE CHALLENGE: Empowering Cybersecurity Analysts
23. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 21
1 2 3
multiple data sets, such as what time users are logging on to their
computers, how that compares with their normal computer use, and
-
detection can be used to identify a nation state worming its way into
-
at an odd time or from an odd place, the system provides an analytic
connect to the network, and other behavior, then they can combine
-
SUCCESS FACTORS
THE SOLUTION
Let the use case dictate what
sensors are needed and what data
should be collected.
Communicate with other people in the
department, and get a feel for what
systems and capabilities they’re using.
Develop analytics in a common way,
and work toward a common goal across
the department, to create opportunities
to share analytics.
“If you have a number of different log alerts or data sets that are flagged,
then you’re able to correlate those and start answering heuristic questions.”
LOGIN AFTER HOURS
OUTSIDE U.S.
FILES DOWNLOADED
SYSTEM
DETECTS
ANOMALIES
SENDS
PRIORITIZED
ALERTS
TO ANALYSTS
ANALYSTS
VIEW ALERTS
& RESPOND
LOGIN AFTER HOURS
LOGIN AFTER HOURS
OUTSIDE U.S.
3
2
1
25. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 23
In the World of Data Analytics,
Speed Is Everything
An interview with Vaughn Stewart
Vice President, Enterprise Architect, Pure Storage
INDUSTRY SPOTLIGHT
can access, analyze and disseminate infor-
mation impacts more than cost savings and
ability to quickly run queries on individuals
what actions to take based on that informa-
must be able to quickly detect and assess
deteriorating weather conditions and take
necessary precautions, such as closing roads
But how can agencies support the growing
demands for faster services when budgets
-
cies, the answer to powering these types
-
-
ing information faster to they can respond
emergency response, cybersecurity or in the
agencies to run more queries, gain greater
insight, and have a deeper knowledge of their
Much of that data, however, is stored in
Although there are many technologies that
can provide a central repository for data, the
speed at which agencies can process that
data is another factor agencies must consid-
as the game changer for agencies that are
but in many circumstances, it costs less than
-
er than as a last resort to resolve their biggest
them speed their work because the latency
issues often associated with disk storage
poor foundation, you can only build a building
generation of technology or architecture
technology, you can then allow that building
If agencies switch the infrastructure from
storage, it allows them do exponentially more
that can handle 10 times the capacity at
about the same cost of overhead as a spin-
agencies to keep up with the demand without
-
allows agencies to stay modern by upgrading
software and controllers (free of charge)
“Analytics brings to the table an improved
model for what is happening, trending or
changing, and how to immediately implement
performance that is not impacted by mainte-
(and less expensive) alternative to the indus-
try norm, where agencies have to rip and
replace storage regularly due to spectacular
26. 24 | A GOVLOOP GUIDE
Norcross Police Department
An interview with Bill Grogan
Support Services Captain at Norcross Police Department in Georgia
have is, for such a small area, we have quite a number of armed rob-
In 2012, a new police chief took the reins and shared his vision of us-
stumbled upon an article about the use of predictive analytics in law
THE CHALLENGE: Predicting & Reducing Crime
27. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 25
1 2 3
wanted for burglary, and a few weeks later they caught burglars break-
management system, including the type of crime, where it happened,
computers to view the prediction boxes — eight red boxes, each
zoom in on the boxes, which might encompass an apartment building,
-
-
the prediction boxes than in other areas because that increases the
-
It took more than a year to get everyone on board with the software,
-
-
SUCCESS FACTORS
THE SOLUTION
Properly train end users to use any
new analytics software.
Seek their feedback on how to use the
tool and improve processes.
Work with the data analytics provider
to share valuable feedback on how to
improve the tool.
“This is not a crystal ball or ‘Minority Report.’”
29. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 27
4 Keys to a Successful Data
Analytics Program
An interview with Gus Horn
Global Consulting System Engineer, NetApp
INDUSTRY SPOTLIGHT
has the power to transform the way govern-
are countless stories of agencies using ana-
lytics to speed emergency response times,
improve online services and better track
But the path to a successful big data analytics
implementation is one that requires proper
long-term planning, said Gus Horn, Global
involved in acquiring and ingesting data into
also the logic around the analytic compo-
nent, how to write it and gain insight into
block is typically around ingesting the data
with small development and testing environ-
ments and a handful of servers to demon-
small environments are a great starting point
for agencies to demonstrate the power of
big data analytics, the environment must
be adapted to properly plan for future data
-
worldwide, regardless of sector, to start down
-
become trapped in an architecture that is
extremely rigid in its design, and two or three
respond to newer technologies, or implement
on when the system has grown exponen-
tially, servers age and must be replaced,
to a new hardware infrastructure, and the
employees who originally built the system
keeping the system up and running be-
Growth.
for growth when they start talking about a big
by their nature are designed to accommo-
date the velocity, variety and variability of
start out small, they should always be cogni-
use are rapidly evolving and so are the
these two technologies that play integral roles
in supporting big data systems are on two
trying to address, as well as their growth
demands each month, each year and even
attaching the storage to the compute, but
and change out the computational element
without having to migrate or rebalance that
-
replace their servers when needed, without
agencies can keep their data put when updat-
storage important, but so is agility in provid-
types of data to the most economical storage
options, depending on how often the data
not very relevant in your hot bucket, when
you can move that into a cold or glacier-like
storage device and still have access to it
customers is the ability to automate data
migration from higher-priced storage to
agencies to tag the data so they can track its
usage and determine which storage option
-
peating the same mistakes because the task
of moving to a newer technology is perceived
existing technology cluster, migrate the data
to new hardware and decommission older
30. 28 | A GOVLOOP GUIDE
U.S. Postal Service
An interview with Dan Houston Jr.
Manager of Data Science and Exploration, U.S. Postal Service
processed each minute, on average, or nearly 6,000 each second,
agency relies on multiple data sets and tools to analyze operational
he oversees 35 petabytes of data — up from 22 petabytes in 2012 —
“I started two years ago looking at gaps we had from a big data and an-
-
ogies, a lot of traditional relational databases, and those work well to a
plants, “it always turned into an eight- or nine-week data provisioning
work collectively to provide data-driven answers quickly, rather than
THE CHALLENGE: Improving Mail Delivery
31. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 29
1 2 3
platform that could store all types of data, and later format that data
But once the data is together in one place, how do you quickly visu-
search tool, which integrates with the Hadoop environment, comes
“By indexing and ingesting all of that data in one place, we can quickly
you can certainly look at that raw data — but you can also quickly
can get some quick visuals and explore the data quickly before you
-
attacks originated and what parts of the environment attackers were
-
loads, being able to better tell our plant managers and plant workers
equipment those pieces are going to have to be run through to get
through their facility — what pieces are going to go out to local post
Having this level of information will ideally help managers improve
the agency can predict the path a mailpiece or package will take
through a plant and better address defects that may have thrown
the optimal route for mail carriers, which may require adjusting routes
a big load and falls behind on some deliveries; dynamic routing would
allow the agency to shift some of that load to another carrier nearby
SUCCESS FACTORS
THE SOLUTION
To reduce skepticism, find a pain point
that’s relevant to people and show
them how data can make it better.
Make data analytics tools and data
accessible and usable to a wider au-
dience, not just people with advanced
programming experience.
Focus on making data operational, so
people can quickly visualize the infor-
mation and act on it.
“By indexing and ingesting all of that data in one place, we can quickly see
patterns too. Not only do you have the data in one place but you can also
quickly get visualizations, and those are just out of the box.”
USPS IN A DAY
512 $223.7
320,132
money orders issued
113,531
address changes processed
17,029
passport applications accepted
million
mailpieces
processed
& delivered
356,103
a minute
5,935
a second
million
in revenue
32. 30 | A GOVLOOP GUIDE
WebFOCUS iWay Software Omni
informationbuilders.com Get Social
Transforming Data Into Customer Value
Customer-Facing Analytics
Differentiated User Experience
Information as an App
33. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 31
Data Analytics as an Asset
An interview with Dr. Rado Kotorov
Chief Innovation Officer and Vice President, Information Builders
INDUSTRY SPOTLIGHT
But despite its growing popularity, govern-
ment continues to face many challenges with
out there and the cost of management and
analytical processes, many agencies have yet
face a huge challenge when it comes to integra-
sources in a variety of formats, which makes
sources and formats, the more costly it is for
government to run analytics across various
-
lishing and disseminating data to their con-
a public, self-service capacity requires new
When governments fail to provide accessible
data to their constituents, they lose account-
challenges and use data analytics to turn
-
mation Builders, shared how government can
navigate these data analytics challenges and
Information Builders is in the business of
providing full information lifecycle management,
which includes the tools to access, manage,
most is providing platforms to publish this infor-
with the data, rearrange it, and customize it
without having to learn new tools or install
“We try to make it easier for organizations to
distribute analytical information to the end
but what makes data analytics more compli-
-
through the concept of data warehouses
could have many individual data warehouses
and data marts and even many more Excel
seamless communication between an organi-
which is one big data storage pool that
is a logical concept, that is why some people
technology that allows you to quickly access
In this manner, the data lake becomes a logi-
cal structure that maps how you can connect
from each source and combine them togeth-
organizations combine their data warehouses
needs to focus on taking the following steps
get more in-depth and seamless information
Maintaining your
data for data analytics is like maintaining the
monitored, checked for problems, and re-
-
tions need to ensure they have the analytical
-
information to turn it into a product consum-
why Information Builders works to provide
the tools and platforms necessary for govern-
ment to facilitate better analysis, integration,
critical to making data analytics more valuable
to government agencies, more accessible to
citizens, and ultimately, ensuring more trust
continues to keep the consumer in mind
when transforming data analytics from a
34. 32 | A GOVLOOP GUIDE
Pacific Northwest National Laboratory
An interview with Justin Brown
IT Engineer at Pacific Northwest National Laboratory
terrorism and the non-proliferation of weapons of mass destruction,
-
-
customer experience because the more time that we save for our
customers — the researchers — the more it enables them to focus on
monitoring its devices, the desktop team has tools for monitoring logs
and issues with the workstations, and the server administrators have
THE CHALLENGE: Enhancing IT Investment Tracking
35. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 33
1 2 3
monitors the performance and usability of servers, webpages and
Having access to the data makes it easier to perform problem analy-
review its data for any anomalies or issues that occurred around the
Brown and his team can use more targeted communications to inform
-
When Brown and his team began rolling out data analytics capabilities
about two years ago, the focus was on empowering cyberteams to
support side of things and the operational side of things, we just start-
SUCCESS FACTORS
THE SOLUTION
Determine key performance indicators,
or what metrics will be measured. For
each service, determine what matters
most to the organization and begin
monitoring those issues.
Develop a methodology or a systematic
approach to using the data. Without
that, having all the data won’t do a lot
of good.
Show the return on investment. Use the
data to clearly show time savings, cost
savings and any other benefits.
“Once we started seeing the potential of that data for the IT support side of
things and the operational side of things, we just started heavily digging into
it and seeing what we could do.”
accessible IT
performance
data
improved monitor-
ing & automation
capabilities
assigned risk
score to IT
resources
1,800 hours
+ $453,000
SAVED+ =+
36. 34 | A GOVLOOP GUIDE
1 2 3
3 REASONS PRIVACY
SHOULD BE TOP OF MIND
review
-
Big data and analytics
tools can alter the balance
of power between
government and citizen.
Government agencies can reap enormous
government uses of big data also have the
potential to chill the exercise of free speech or
-
alyzed, and stored on both public and private
systems, we must be vigilant in ensuring that
balance is maintained between government
Big data and analytics
tools can reveal intimate
personal details.
merging multiple data sets, drawn from dis-
But this practice, sometimes known as “data
-
mation can be discerned even from ostensi-
more widely used in the private sector
to bring a wellspring of innovations and
consumer privacy protections are in place to
Big data and analytics
tools could lead to
discriminatory outcomes.
As more decisions about our commercial and
personal lives are determined by algorithms
and automated processes, we must pay care-
ful attention that big data does not system-
atically disadvantage certain groups, whether
-
vent new modes of discrimination that some
uses of big data may enable, particularly with
regard to longstanding civil rights protections
37. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 35
WHAT YOU CAN DO NOW:
HOW TO CREATE A CULTURE
FOR DATA ANALYTICS
-
-
Jeff Chen
Chief Data Scientist,
Department of Commerce
Learn the lingo.
When Chen was Director of Analytics for
the New York City Fire Department, he rode
Tell the story about your data.
There are a lot of scientists at Commerce,
Jon Minkoff
Chief Data Officer,
Enforcement Bureau, Federal
Communications Commission
Brand your systems.
know their reports will be accepted only if
Kirk Borne
Data Scientist and former Professor
of Astrophysics and Computational
Science at George Mason University
Understand data as a literacy.
While technical skills are important, they are
Recognize that data is an asset.
is a binary representation of real information,
and real information is what we are trying
create better insights, decisions, services to
38. 36 | A GOVLOOP GUIDE
Caryl Brzymialkiewicz
Assistant Inspector General and Chief
Data Officer at the Department of
Health and Human Services Office of
the Inspector General
Go on a listening tour.
Consider the talent you have on board.
Consider rotation programs.
Joshua Martin
Chief of Staff,
the Indiana Management and
Performance Hub
Make room at the table for a variety of
participants.
agencies are a part of the planning
process and will help shape how the tool
Pull from your past experiences.
Before starting his career as a state govern
late to agencies that have a role in combat
Keep everyone informed.
Linda Powell
Chief Data Officer,
Consumer Financial Protection Bureau
Your data scientists and data manage-
ment professionals have to understand
the jargon of the business.
ing a linear regression, he is going to kick me
Understand what your agency customers
or partners want to accomplish.
39. HOW YOU CAN USE DATA ANALYTICS TO CHANGE GOVERNMENT | 37
ABOUT & ACKNOWLEDGMENTS
CONCLUSION
-
ter outcomes.
-
-
So what’s next for data analytics in government?
to drive cybersecurity operations and predictions about future
systems
something that doesn’t happen than not to predict anything and
-
of government success stories. It’s an invitation to reach out to
G -
-
-
government.
.
| @GovLoop
Th
Author:
Nicole Blake Johnson, Technology Writer
Designers:
Tommy Bowen, Graphic Designer
40. 38 | A GOVLOOP GUIDE
1152 15th St NW, Suite 800
Washington, DC 20005
Phone: (202) 407-7421 | Fax: (202) 407-7501
www.govloop.com
@GovLoop