Getting down to business on Big Data analytics
Upcoming SlideShare
Loading in...5
×
 

Getting down to business on Big Data analytics

on

  • 785 views

 

Statistics

Views

Total Views
785
Views on SlideShare
785
Embed Views
0

Actions

Likes
0
Downloads
10
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Getting down to business on Big Data analytics Getting down to business on Big Data analytics Presentation Transcript

  • Handbook 1EDITOR’S NOTE 2WITH BIG DATA SYSTEMS, PLAN AHEAD 3FOCUS SHARPENS ON BIG DATA’S BUSINESS VALUE 4HADOOP LIGHTS PATH TO CONSUMER INSIGHTS VIRTUALIZATION CLOUD APPLICATIONDEVELOPMENT HEALTHIT NETWORKING STORAGEARCHITECTURE DATACENTERMANAGEMENT BI/APPLICATIONS DISASTERRECOVERY/COMPLIANCE SECURITY Getting Down to Business on Big Data Analytics Capturing and storing big data is just the beginning. Reaping real business value and competitive advantages from collections of structured and unstructured data is the end goal.
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 2   GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 1EDITOR’S NOTE It’s All About the Analytics Collecting reams of big data is one thing; doing something useful with all that informa- tion is another. But the former without the latter won’t win business intelligence, analyt- ics and IT teams any plaudits from corporate executives. As Gartner analyst Doug Laney put it at the consulting company’s 2013 Business Intelligence and Analytics Summit, success- ful big data initiatives depend on companies “recognizing that there are opportunities to re- ally innovate with this information”—and then taking the required steps to capitalize on those opportunities. That’s where big data analytics comes in. Finding the business value hidden in hoards of big data can be a tough nut to crack—but there’s a growing body of examples to help or- ganizations get cracking. This three-part guide provides insight and advice on managing big data analytics programs. We start with a look at four key factors to consider in planning projects. Next we catalog guidance on deriving business value from big data. We close with an interview about a Hadoop-based analytics sys- tem that’s being used to examine the shopping habits of Latino consumers. n Craig Stedman Executive Editor SearchBusinessAnalytics.com
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 3  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 2STRATEGY With Big Data Systems, Plan Ahead By providing access to broader sets of information, big data can help maximize the analytical insights generated by data analysts and business users. Successful big data analyt- ics applications uncover trends and patterns that enable better decision making, point to new revenue opportunities and keep companies ahead of their business rivals. But first, organi- zations often need to enhance their existing IT infrastructure and data management processes to support the scale and complexity of big data architecture. Hadoop systems and NoSQL databases have become key tools for managing big data envi- ronments. In many cases, though, businesses are utilizing their existing data warehouse in- frastructure, or a combination of the new and old technologies, to manage the big data flow- ing into their systems. Whatever type of big data technology stack a company deploys, there are some common considerations that must be addressed to en- sure it will provide an effective framework for big data analysis efforts. Before getting started on big data projects, it’s crucial to look at the, er, bigger picture of the new data requirements they entail. Let’s examine four of the consider- ations that need to be taken into account. n Data accuracy. Data quality issues are cer- tainly no stranger to BI and data management professionals. Many BI and analytics teams struggle to ensure the validity of data and convince business users to trust in the accu- racy and reliability of information assets. The widespread use of spreadsheets as personal- ized analytics repositories, or spreadmarts, can contribute to a lack of trust in data: The ability to store and manipulate analytics data in Excel creates an environment that supports self-ser- vice analysis capabilities but might not inspire other users to act confidently on the findings.
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 4   GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 2STRATEGY Data warehouses, coupled with data integration and data quality tools, can help instill that con- fidence by providing standardized processes for managing BI and analytics data. But a big data implementation adds to the degree of difficulty due to increased data volumes and a wider va- riety of data types, particularly when a mix of structured and unstructured data is involved. Assessing data quality measures and upgrad- ing them as needed to handle those larger and more varied data sets is vital to the successful implementation and usage of a big data analyt- ics framework. n Storage fit. One of the core demands of data warehousing is the ability to process and store large data sets. But not all data warehouses are created equally in that regard. Some are op- timized for complex query processing, while others aren’t. And in many big data applica- tions, the addition of unstructured data and the increased velocity at which data is created and collected compared to transactional systems makes augmenting a data warehouse with Ha- doop or NoSQL technologies a necessity. For an organization looking to capture and analyze big data, storage capacity isn’t enough; the im- portant part is where the data is best put so it can be transformed into useful information and made available to data scientists and other users. n Query performance. Big data analytics de- pends on the ability to process and query com- plex data in a timely fashion. A good example is a company that developed a data warehouse to maintain data collected from energy usage me- ters. During product evaluations, one vendor’s system was able to process 7 million records in 15 minutes, while another’s topped out at 300,000 records in the same amount of time. Identifying the right infrastructure to support fast data availability and high-performance querying can make the difference between suc- cess and failure. n Scalability. With growing data volumes and variety in many organizations, a big data plat- form can’t be built without the future in mind. It’s imperative to think ahead and ask whether the big data technologies being evaluated can scale to the levels that will be required going
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 5  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 2STRATEGY forward. That extends beyond storage capacity to include performance as well, particularly in companies that are looking at data from social networks, sensors, system log files and other non-transactional sources as extensions of their business data. Analyzing diverse and complex data sets re- quires a robust and resilient big data architec- ture. By considering these four factors when planning projects, organizations can determine whether what they already have in-house can handle the rigors of big data analytics applica- tions or if additional software, hardware and data management processes are required to achieve their big data goals. —Lyndsay Wise
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 6  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 3PROJECT MANAGEMENT Focus Sharpens on Big Data’s Business Value How best to deliver big data analytics to users was one of big topics of discussion at the TDWI BI Executive Summit in Las Vegas. Presenters and attendees alike looked to chart the way to successful analytics initiatives that connect big data and business value, enabling companies to get their money’s worth from the mountains of structured and unstructured data they’re building up in data warehouses, Hadoop systems and NoSQL databases. The key issue business intelligence (BI) and analytics professionals must address hasn’t changed with the advent of new data types and technologies for managing them, said Bar- bara Wixom, an associate professor of IT at the University of Virginia’s McIntire School of Commerce and a visiting scholar at MIT’s Sloan School of Management. The goal, she said, is still to come up with a good answer to this question: How do we get the data to the users? “There is no value created without use,” said Wixom, who isn’t a fan of the term “big data” but does agree that “data is changing.” Those changes, she added, require data manage- ment professionals to redouble their efforts to develop data architectures that can support the expanding variety of big data captured by companies. “The quantity of sources themselves is be- coming overwhelming. We have more data sources popping up every day,” said Evan Levy, vice president of business consulting at soft- ware vendor SAS Institute Inc. The key prob- lem is figuring out how to move that data around the organization and to deliver it to business users, Levy said in a keynote speech at a TDWI World Conference held jointly with the BI summit.“I have a rat’s nest of code going back and forth,” he said.“What do I know about all the programs moving that data?” Levy said IT and data management teams
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 7  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 3PROJECT MANAGEMENT will be best served by taking incremental ap- proaches to dealing with the influx of big data. He also told attendees to take cues from es- tablished manufacturing supply chain applica- tions, which deliver final, uniform products working from diverse raw materials. SIGHTS SET ON HADOOP The growing supply of Web data challenges conventional data warehouse and analytics de- livery planning, according to Jessica Thorud, director of enterprise travel data warehouse and business intelligence solutions at Sabre Holdings Corp. in Southlake, Texas. She said the travel reservation systems developer is moving to alternative data strategies due in part to the huge volume of shopping data it is gathering. “We know it is coming, and we know how to do it,” said Thorud, who is continually looking for new and better ways to integrate modern Hadoop big data tools with enterprise data warehouses and analytics systems.“We hope the [Hadoop] tools continue to evolve so we can connect them to the BI tools,” she said. While it’s early in the big data era, Thorud predicts that her company will one day de- liver big data analytics to customers, especially those focused on marketing applications.“They want the insight. They have ideas on how to use it in decision support,” she said. To deploy Hadoop, Sabre selected the jointly produced Oracle-Cloudera Big Data Appliance, which is based on the open source distributed process- ing software. Thorud’s comments came as she took part in a summit session on delivering innovation in travel through data and BI products. She also encouraged attendees to focus on usabil- ity and design when designing BI and analytics “We hope the [Hadoop] tools continue to evolve so we can connect them to the BI tools.” —jessica thorud, director of enterprise travel data warehouse and business intelligence solutions, Sabre Holdings Corp.
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 8  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 3PROJECT MANAGEMENT applications, and she reminded them to “know your customers’ needs and capabilities.” MARKETING TAKES THE LEAD Like Thorud, Wixom noted that marketing- focused applications have garnered significant attention among new and innovative analytics initiatives. She told the TDWI audience that she had studied the early efforts of data ware- housing, finding that back then, marketing also led the way in adoption. “There is lots of buzz around big data. Dif- ferent people have different concepts,” said conference attendee Masood Ali, information management architect at the Royal Bank of Canada, who came to the TWDI summit in part to help determine the correct big data strategy for his organization. Despite the hype, he sees greater use on the way. “Big data will soon become normal data,” he said.“The important thing is to build for pur- pose, so big data has value. —Jack Vaughan
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 9  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 4APPLICATIONS Hadoop Lights Path to Consumer Insights Analytics services provider Lumi- nar is counting on big data technology to help it deliver valuable insights about U.S. Latino consumers to marketers and other corporate clients. The Denver-based company ditched a traditional data warehouse setup in favor of a system built around the Hortonworks Data Platform—a Hadoop distribution—in an effort to speed up the analytics process and make it easier to manage the data being collected. The idea behind the company’s service, says Luminar President Franklin Rios, is to provide clients with a far more reliable alternative to focus groups and surveys—and to provide bet- ter information about Latino communities than ever before. For example, Luminar—a unit of Spanish-language media company Entravision Communication Corp.—can tell its customers how Cubans in Miami spend their money on technology, or how much the typical Puerto Ri- can male in New York spends on food. SearchBusinessAnalytics.com spoke with Rios about how his company is using the Ha- doop system to support its big data analytics efforts. Excerpts from the interview follow: What is Luminar and what does it do? In short, Luminar is an analytics and modeling company that specializes in the U.S. Latino con- sumer. What I mean by that is we drive insights through analytics and modeling by diving into [data] that we ingest from multiple sources. We give consumer packaged goods companies (CPGs), retailers or what have you insights into the true behavior of the Latino consumer. Why did you see a need for a Latino- focused market analytics provider? Before Luminar, a lot of marketers or adver- tising agents wanted to start reaching out to the Latino consumer, [but] they relied on highly sampled data from the usual suspects.
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 1 0  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 4APPLICATIONS Companies would use focus groups and self- reported panels and [small surveys]. They would extra-polate [from that] and do statisti- cal numbers to suggest how the rest of the 52 million Latinos in the U.S. behave. There is $1.5 trillion worth of purchasing power in the U.S. Hispanic space. With 52 million Latino men, women and children in the U.S., I don’t know how a sample of 10,000 self-reported Latinos can give any true indication to any retailer or CPG on their behavior. What is Luminar doing that is different from that“legacy”approach? The traditional way is highly sampled. So we’re saying that we are not going to do that. We’re going to take transactional data that we’re go- ing to license from multiple sources. We have about 2,000 sources of data that we ingest and we analyze and we clean up, and then we ap- ply what we call cultural filters in order to truly find out who is a Latino. Some of the data you receive must be ambiguous. How do you tackle the problem of identifying who is who? We have access to data that comes from loyalty systems. So if you belong to a grocery store loyalty system, your name and address and all of that is in there. But how we truly start deriv- ing who is a Latino is by starting to look at the purchases and the transactions that the house- hold has made. We also have access to things like magazine subscriptions, and we know if somebody is getting their utility bill in Span- ish, and all kinds of stuff. We use a scoring mechanism that says if you’re doing 55 or 100 of these behaviors, and if in the grocery store the contents of your basket contain products that are very much Latino products for cooking and such, then the scoring keeps going up and up and up to identify a Latino. Luminar has access to names, addresses and other personal information about individuals. How do you deal with privacy concerns? The privacy issue would come into play if I were sharing that data with my clients at the personal level, and I’m not. I’m aggregating it to create personas, and we identify those personas at a group level and we start telling the behav- ior of the personas to our clients.
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 1 1  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS 4APPLICATIONS Your company launched a Hortonworks- based Hadoop system in 2012. How were you handling this operation before that? We handled it in the traditional way. We had a data warehouse with all of the legacy pluses and minuses that you run into in a data ware- house environment. But we realized that we needed to be significantly more agile in being able to ingest data, clean it up, run our cultural filters into it and analyze the data and start   deriving insights out of it. We are ingesting 2,000 sources of data and the traditional tools for data processing were not cutting it. Are you still running the data warehouse as well as the Hadoop system? Nope. It’s off. Thank goodness we turned it off. Not [only] am I happy but my CFO is happy because of the savings. Currently, we do the [extract, transform and load] process- ing of data using Talend software and the data gets ingested either via Talend or using Sqoop directly into Hortonworks. Once it’s in Hortonworks, we then use a combination of Hive and R to load our analytical models. Then, the results of that are presented via Tableau. What advice do you have for other companies considering a big data analytics initiative? If anybody is starting to look into migrating into this technology, don’t try to do it alone. You’ve got to have the right technology partner or consulting partner to help you through this project. Also, don’t be geographically limited in terms of who you evaluate. We found talent in Latin America that was able to help us with this process. —Mark Brunelli
  • Home Editor’s Note With Big Data Systems, Plan Ahead Focus Sharpens on Big Data’s Business Value Hadoop Lights Path to Consumer Insights 1 2  GETTING DOWN TO BUSINESS ON BIG DATA ANALYTICS LYNDSAY WISE is president and founder of WiseAnalytics, an independent research and analysis firm that focuses primarily on business intelligence deployments at small and midsize businesses. Wise has 10 years of experi- ence in business systems analysis, software selection and implementation of enterprise applications. Email her at lwise@wiseanalytics.com. JACK VAUGHAN is news and site editor of SearchData- Management.com. He covers topics such as big data management, data warehousing, databases and data integration. Vaughan previously was an editor for TechTarget’s SearchSOA.com, SearchVB.com, TheSer- verSide.net and SearchDomino.com websites. Email him at jvaughan@techtarget.com. MARK BRUNELLI is news director of the Business Appli- cations and Architecture Media Group at TechTarget. Email him at mbrunelli@techtarget.com. ABOUT THE AUTHORS Getting Down to Business on Big Data Analytics   is a SearchBusinessAnalytics.com e-publication. Scot Petersen | Editorial Director Jason Sparapani | Managing Editor, E-Publications Joe Hebert | Associate Managing Editor, E-Publications Melanie Luna | Managing Editor Craig Stedman | Executive Editor Linda Koury | Director of Online Design Neva Maniscalco | Graphic Designer Doug Olender | Publisher dolender@techtarget.com Annie Matthews | Director of Sales amatthews@techtarget.com TechTarget 275 Grove Street, Newton, MA 02466  www.techtarget.com © 2013 TechTarget Inc. No part of this publication may be transmitted or re- produced in any form or by any means without written permission from the publisher. TechTarget reprints are available through The YGS Group. About TechTarget: TechTarget publishes media for information technology professionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and pro- cesses crucial to your job. Our live and virtual events give you direct access to independent expert commentary and advice. At IT Knowledge Exchange, our social community,you can get advice and share solutions with peers and experts.