ERP, SCM, CRM, and transactional Web applications are classic examples of systems processing Transactions. Highly structured data in these systems is typically stored in SQL databases. Web 2.0 are about how people and things interact with each other or with your business. Web Logs, User Click Streams, Social Interactions & Feeds, and User-Generated Content are classic places to find Interaction data.Ambient data tends is coming “Internet of Things”. Mary Meeker has predicted 10B connected devices by 2015. Sensors for heat, motion, pressure and RFID and GPS chips within such things as mobile devices, ATM machines, and even aircraft engines provide just some examples of “things” that output ambient signals…There are multiple types of data personal - > organizational - > public - > private So we should NOT minimize our thinking to just data that flows through an organization. Ex. The mortgage-related data you may have COULD benefit from being blended with external data found in Zillow, for example.Moreover, the government has the Open Data Initiative. Which means that more and more data is being made publicly available.
So – for the purposes of this presentation – let me set the context for what is meant by Big Data. When we refer to Big Data we are referring to data sets that have the following characteristics or attributes: 1. Large Data Volume (Size) >10TB 2. High Data Velocity (Growth) >25% YoY3. Wide Data Variety (Form) >15% Unstructured Data (Text, E-Mail) Weblogs, Video, Images, Sound, As you can see from this graphic – we traditionally look at Data through two lenses – traditional structured data – which accounts for 20% of all data and is what enterprises historically focused on – and unstructured data - which accounts for appx. 80% of all data, but nobody much cared about until recently – it’s often referred to as dark data and sits beneath the surface. Again – historically we did not have the tools to do much with unstructured data – so it was just noise.
Today new types of questions are being asked to drive the business. These questions include:Questions on Social & Web Analytics e.g. What is my brand and product sentiment? How effective is my online campaign? Who am I reaching? How can I optimize or target the correct audience? Questions that require connecting to live data feeds e.g. a large shipping company uses live weather feeds and traffic patterns to fine tune its ship and truck routes leading to improved delivery times and cost savings. Retailers analyze sales, pricing and economic, demographic and live weather data to tailor product selections at particular stores and determine the timing of price markdowns.Questions that require advanced analytics e.g. Financial firms using machine learning to build better fraud detection algorithms that go beyond the simple business rules involving charge frequency and location to also include an individual’s customized buying patterns ultimately leading to a better customer experience. Organizations that are able to take advantage of Big Data to ask and answer these new types of questions will be able to more effectively differentiate and derive new value for the business whether it is in the form of revenue growth, cost savings or creating entirely new business models. Gartner asserts that “By 2015 businesses that build a modern information management system will outperform their peers financially by 20 percent.” McKinsey agrees, confirming that organizations that use data and business analytics to drive decision making are more productive and deliver higher return on equity than those who don’t.
There has never been a more exciting time with respect to the world of data. We are seeing the convergence of significant trends that are fundamentally transforming the industry and a new era of tech innovation in areas like social, mobile, advanced analytics and machine learning. We’re seeing an explosion of data, there is an entirely new scale and scope to the kinds of data we are trying to gain insights from. There’s a lot of talk about this – estimates are that the total amount of digital information in the world is increasing 10X every 5 years, with 85% of this data coming from new data types eg. Sensors, RFIDs, WebLogs etc. This presents a huge opportunity for Businesses that tap into this new data to identify new opportunity and areas for innovation. However, having a platform that supports the data trend is only part of today’s challenge; you need to also make it easier for people to access so that they can infer insight and make better decisions. If you think about the user experience, with everything we are able to do on the Web, our experiences through social media sites, how we’re discovering, sharing, and collaborating in new ways – User Expectations of their business and productivity applications are changing as well.
The world of data is changing in a big way, and customer expectations are changing accordingly. We offer the most complete and scalable portfolio of data storage capabilities for structured, unstructured and streaming data both on premises and in the cloud. Customers can unleash new value through discovering and enriching and connecting to data, services and people outside their organizations and deliver new insights into Big Data for all users through familiar tools such as Office and SharePoint. Specifically, Microsoft’s Big Data solution offers the best end to end platform to manage any data, any size, anywhere with our industry leading database products SQL Server 2012, SQL Server Parallel Data Warehouse appliance, streaming data via SQL Server StreamInsight new capabilities such as our Hadoop-based distribution on Windows Azure and Windows Server for processing petabyte scale unstructured data. New value is created by enriching your data with the world’s data through the industry’s first data marketplace – Azure Marketplace Datamarket. Unlock actionable insights for all users through familiar tools such as PowerPivot for Excel and Power View for SharePoint delivered wherever you are and on any device.
Engage 2013 - Big Data
What is Big Data?
BI Team Lead, Avtex
Data Complexity: Variety and Velocity
Big Data is about building
new analytic applications
new types of data,
in order to
better serve your customers
and drive a better
David McJannet, Hortonworks http://www.informationweek.com/big-data/news/big-data-
How do I optimize my
fleet based on weather
and traffic patterns?
What’s the social
sentiment for my
brand or products
How do I better
GAIN COMPETITIVEADVANTAGE BY MOVING FIRSTAND FAST IN YOUR
Discover data with Data Explorer
Combine with information from
other sources via Azure Marketplace
Refine with advanced analytics
with the World’s Data
Immersive insights for all users
Insights on any data
Embedded insights with simplified
Wherever you are
Windows simplicity and
Manageability for Hadoop
Extend data warehouse with Hadoop
Scale & elasticity of the cloud
Open Big Data Platform
Any Data, Any Size Anywhere
Parallel Data Warehouse
Power Query Excel Add-in