Big Data Challenge: Org, Tech and Process

6/18Business Insights through DataFacing the Big Data Challenge

Gary Angel, President of Semphonic Co-Founder and President of Semphonic, the leading independent web analytics consultancy in the United States. Semphonic provides full- service web analytics consulting and advanced online measurement to digital media, financial services, health&pharma, B2B, technology, and the public sector. Gary blogis at http://semphonic.blogs.com/semangelScott K. Wilder – Partner @ Human1.0 Currently Founder and Digital Strategist at Human 1.0. Before that, Scott was SVP/Social Media Architect at Edelman – Digital. Founded and managed Intuit’s Small Business Online Community and Social Programs. Before Intuit, Scott worked a AOL, Apple, Kbtoys/etoys, Borders, American Express.Scott is also a founding Board member of the Word of Mouth Marketing Association. He received graduate degrees from New York University, The Johns Hopkins University and Georgetown University. Scott’s blog is at http://www.wildervoices.comMarshall Sponder – Founder WebMetricsGuru INC. Marshall Sponder is an Author of the McGraw-Hill book, Social Media Analytics, he is independent Web analytics, data and SEO/SEM specialist working in the field of market research, social media, networking, and Outbound Communications. Marshall is currently working with Principal at WebMetricsGuru INC . Marshall also teaches Social Media Analytics and Art at Rutgers University and UCI Irvine, Extension. Marshall’s blog is http://www.webmetricsguru.com and book site is http://www.smabook.com 2

Agenda• Big-Data: What does it really mean to Your Organization?• The Key Challenges • Approach: Creating the right organization and framework • Gathering: Picking the right technology stack(s) • Analysis: Find Meaning(s) within the Data 3

The Big Data Shift• More marketing dollars moved to digital.• Data growing exponentially with more focus on Big Data• Social and Mobile becoming increasingly important and interconnected (and measureable).• Companies in all sectors have at least 100 terabytes of stored data in the United States; many have more than 1 petabyte and it continues to grow as more people are online in social and mobile.• Better ability to glean customer insights as a result of improvements in semantic technologies.• Desire to expose data externally (gov.org) and share it. 4

Org challenges• Departmental: • Analytic applications are often departmental by nature • Departments deploy their own platforms for big data and analytics • Many organizations today haven’t figured out how to leverage Big Data. • Two thirds of executives believe that there is not enough of a “big data culture” in their organization - this is particularly notable across the manufacturing sector• Technology: • Not all BI/DW technology stacks are designed for advanced analytics • Lack of single digital platform • Difficulty measuring effectiveness – unable to link data to individuals • Complicated buying process/user experience • Not adequately using data they already have • Too much unstructured data to support decision-making• Skills: • Talent shortage • Lack of expertise and experience • Having a just-in-time agile mindset • Ask the right questions 5

Example of the Big Data divide 6

Delivering value across the company 7

Current center of the Big Data Universe Should it Be? 8

The new roles of digital marketersAlmost 60% oforganizations rely on ..marketing to maketechnologyrecommendationsLeading to a mis-matchbetween the goals andtechnology used toexecute. 2011 Digital Marketing 2.0 Study by research effort between DataXu, SNCR and Human 1.0 9

But greater dependency on in-house support and IT organizationsOver 60% of organizationsare relying more oninternal teams thanagenciesAnd only 35% agree that ITis able to provide the toolsthey need to optimize theirdigital marketing 10

No organizational Kumbaya Organizations struggle to make real-time decisions and to pull insights from the large data sets created by digital marketing CMO and CIO teams aren’t always partnering effectively Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree40%35%30%25%20%15%10%5%0% My digital marketing tools provide me with insights into how The CIOs team and the CMOs team in my organization have a My IT groups analysis of digital marketing data on consumer demand for my organizations products and services vary in true partnership in using data to better understand the behavior permits real-time business decisions real-time (depending on time of day, for instance) customer 2011 Digital Marketing 2.0 Study by research effort between DataXu, SNCR and Human 1.0 11

So what’s the hold up?60%agree digital marketing canreduce acquisition costs However, a common issue is not being able to make a case for and prove it to company leadership Digital Marketing 2.0 Study by research effort 2011 between DataXu, SNCR and Human 1.0 12

If you make the change 13

Future Organizational ShiftCEOs will push for more analytics projects --they want to exploit big data for growth CFOs play bigger role on signing off on costs CMO will bring more technical / business CIOs will play a bigger intelligence types into with big data projects their organization 14

Just How Big is Big? 15

CardinalityDistinct Values per Variable Traditional Data Systems relied on the ability to• With lots of distinct values: aggregate most dimensions into small set of – OLAP becomes difficult distinct values to work well. When your dimensions have lots of distinct – Visualization is nearly impossible values (high cardinality), you’re dealing with – In-Memory Systems struggle Big-Data. 16

Complex (and Dynamic) RelationshipsCombining data from different tables:• Joins put lots of stress on the design – Join strategies are complex and hugely impactful – Exposing the data model becomes difficult – Optimizing specific paths limits query flexibilityTraditional Data Systems relied on a small number of static paths to exposereporting data at the aggregate level.When you have to join lots of tables and have unknown or dynamic needs tocombine data (all Analysis applications), then you are dealing with a big-dataproblem. 17

Why Digital is Usually Big DataDigital Measurement is a paradigm case of big-data:• Lot’s of data – Millions (hundreds of?) events per day – Lots of data per event• Lot’s of key High Cardinality variables – Page Name, Product Sets, Referrers, Campaigns, Keywords – and Customers• Lot’s of complex relationships and joins: – Page -> Visit -> Campaign_Touch -> Visitor -> Channel• Traditional variables don’t aggregate meaningfully: – Views, Page Time, Visits, etc. 18

It’s About Getting Your Hands on the DataIn the digital world, there’s little correlation betweensize of enterprise and size of data. For mostorganizations, the real challenges are around accessingand integrating digital data regardless of it’s volume.• Regardless of your data volumes, direct access to the data presents new challenges to digital analytics – The need to model the data meaningfully – New types of analysis and reporting possibilities – More complex technologies that aren’t always SaaS – New types of resource requirements and skills 19

Choice Vectors Handling Very Large Data 100 90 80Ease of Management 70 Richness of and Setup 60 Technology Stack 50 SQL-Server 40 30 Oracle 20 Teradata 10 0 Netezza Availability of Aster Ease of Integration Expertise Hadoop In-Memory Appropriateness to Cost / Size Realtime 20

Conclusions & Final Thoughts

Good Questions Drive ResultsHaving the right organization for Big Data, choosing the righttechnology, and developing a strong foundation for analysis areALL critical to success: Organizational Approach Rich Customer Technology Segmentation Stack Foundation 22

Thank you for your timeGary Angelgangel@semphonic.com@garyangelBlog:http://semphonic.blogs.com/semangel/Scott K. Wilderscott@human1.com@skwilderBlog: www.wildervoices.comMarshall Spondernow.seo@gmail.com@webmetricsguru / @smanalyticsbookBlog: www.webmetricguru.com 23

http://wildervoices.com	27
http://www.csuitetwo.com	20
http://www.wildervoices.com	7
http://feeds.feedburner.com	1
http://www.typepad.com	1

Big Data Challenge: Org, Tech and Process

by Scott K Wilder on Jun 26, 2012

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 56

Statistics

Big Data Challenge: Org, Tech and Process Presentation Transcript