• Like
Big Data
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Published

Tony Lock from Freeform Dynamics

Tony Lock from Freeform Dynamics

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
128
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
8
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • MB to modify to tune to how she likes to define the BD problem
  • Emerging foundation for consistent pundit/vendor viewA bit contrived, but useful, and need to know because the 3Vs and becoming increasingly prominent
  • When you look at some of the information feeds, including structured (logs, M2M, etc) and unstructured (social, web), along with unstructured internal sources, you find a lot of them are very ‘dilute’ when it comes to valueThis these are the areas put into the ‘too difficult’ or ‘not cost effective’ to solve categoryThis is where parallel, distributed scale-out architectures – aka Big Data come inStress that it is not just about unstructured – e.g. one of the most common starting points for IT departments cutting their teeth is systems log file analysis
  • But not just about Hadoop. Players like EMC, HDS, HP, IBM, Oracle and others all have offerings in the Big Data arenaZooming out, here are some of the common categories (there are many ways of categorising, this is more of a stack view)Scale out storage can be used for large traditional warehouses as well as big dataDistributed analytics can run on proprietary grids as well as things like HadoopThe point being that no element of Big data is exclusiveIndeed there is nothing exclusive about big data at all – it will live alongside existing solutions (see next slide_
  • Has advantages that derivative insights are a lot more accessible when extracted from Big Data environment and placed in a traditional environment – skills sets/familiarity, toolset availability, well established integration and propagation mechanismsm,etc

Transcript

  • 1. Big Data, Analytics and the Future of Data Centres Where are we and where are we going? VMUG Meeting, Manchester February 12, 2013 Tony Lock – Programme Director tony@freeformdynamics.com www.freeformdynamics.com www.freeformdynamics.comCopyright 2013 Freeform Dynamics Ltd 1
  • 2. About Freeform Dynamics  Industry analyst firm  Track IT industry developments and offerings  Track the evolution of IT related activity and needs in business  Advise both end user organisations and suppliers  Research approach  IT vendor and service provider briefings  Large scale studies - face to face, telephone and online  Community research programme  Investigate strategy, business case, architecture, best practice  Vendor patronage model allows free distribution  Media partnerships for both input and outputCopyright 2013 Freeform Dynamics Ltd 2
  • 3. Agenda  Big data and Analytics  Where are we today?  The evolution of the data centre  Visions of the future  Will we ever reach Nirvana?  Closing thoughtsCopyright 2013 Freeform Dynamics Ltd 3
  • 4. Big Data  What is it?  Is anyone doing it?  The only game in town?Copyright 2013 Freeform Dynamics Ltd 4
  • 5. Defining Big Data?  Analogies  Panning for gold  Finding the needle in the hay stack  Identifying a weak signal in a very noisy environment  Find valuable patterns, trends, correlations, etc. in noisy, unstructured, often complex, and high volume data sets  Doing analytics better / differently?Copyright 2013 Freeform Dynamics Ltd 5
  • 6. How much do you agree or disagree with the following statements? 0% 20% 40% 60% 80% 100% The emergence of advanced storage, access and analytics solutions means the end of the traditional RDBMS Regardless of substance and reality of emerging technologies and techniques, the term ‘big data’ is currently being over-hyped by IT vendors in an unhelpful way I have a clear understanding of what the term ‘big data’ means 5-Totally agree 4 3 2 1-Totally disagree UnsureCopyright 2013 Freeform Dynamics Ltd 6
  • 7. The three Vs of Big Data High physical volumes with low value density Rule of thumb Volume Generally think of Big Data Different sources and formats or when two of Variety information these three apply Velocity Rapid rate of data movement, generation or acquisitionCopyright 2013 Freeform Dynamics Ltd 7
  • 8. The concept of value density Low value density High value density Traditional BI and data Structured warehousing Structured BIG DATA Traditional search and Unstructured Unstructured document management Low value density High value densityCopyright 2013 Freeform Dynamics Ltd 8
  • 9. What level of growth are you seeing in the following types of data within your organisation? 0% 20% 40% 60% 80% 100% Structured data (e.g. tabular data in RDBMSs) Unstructured data (e.g. documents, messages, multimedia content, etc.) 5 (Extremely high growth) 4 3 2 1 (No growth)Copyright 2013 Freeform Dynamics Ltd 11
  • 10. In what form is your organisation’s most valuable or critical dataheld (i.e. your crown jewels in information terms)? 0% 10% 20% 30% 40% 50% Exclusively structured Mostly structured Equal split Mostly unstructured Exclusively unstructuredCopyright 2013 Freeform Dynamics Ltd 12
  • 11. How is this changing? 0% 20% 40% 60% 80% 100% Steady No Steady shift change shift (21%) (43%) (25%) Rapid shift Rapid shift towards value in towards value in unstructured structured data data (7%) (4%)Copyright 2013 Freeform Dynamics Ltd 13
  • 12. To what degree does your organisation exploit its information assets for analysis and decision making? 0% 20% 40% 60% 80% 100% Structured data (e.g. tabular data in RDBMSs) Unstructured data (e.g. documents, messages, multimedia content, etc) 5 (Fully) 4 3 2 1 (Very poorly)Copyright 2013 Freeform Dynamics Ltd 14
  • 13. Use of traditional and emerging technologies Current level of use Change over next 3 years 0% 20% 40% 60% 80% 100% -60% -40% -20% 0% 20% 40% 60% Legacy databases and file systems General purpose RDBMS servers High performance RDBMS configurationsOLAP multi-dimensional database systems Write once read many (WORM) databases Rule-based stream processing engines In memory databases Scale-out storage architectures Distributed indexing and search Distributed data analytics engines 5 (Extensive use) 4 3 2 1 (Not used at all) Unsure Less use More useCopyright 2013 Freeform Dynamics Ltd 16
  • 14. How much do you agree or disagree with the following statements? 0% 20% 40% 60% 80% 100% Developments in advanced storage, access and analytics are allowing us to tackle problems today that were either too hard or too… Developments in advanced storage, access and analytics are allowing us to take different and better approaches to tackling some key… Vendors and consulting firms are well geared up to providing us with the support and services we need to take advanced storage, access… We have a clear idea of the business benefits available to us through the use of big data technologies and solutions We have a clear idea of the advanced data storage and big data analytic technologies that are becoming available Database vendors are well geared up to support their customers with appropriate licensing and commercial arrangements as data related… 5-Totally agree 4 3 2 1-Totally disagree UnsureCopyright 2013 Freeform Dynamics Ltd 17
  • 15. Elephants in the room  Access to data – what data is there and where is it?  Are there governance / regulatory / legal restrictions in play concerning certain data sets?  Skill shortage?  IT skills  Numerical skills in user base  Just what questions could ‘Big Data’ help with?  How do we exploit any results we generate?  Feedback into ‘mainstream systems’Copyright 2013 Freeform Dynamics Ltd 18
  • 16. Data Centre Evolution  Where are we now?  ‘Perfect Visions’  Will we ever get to Nirvana?Copyright 2013 Freeform Dynamics Ltd 19
  • 17. Fragmentation and disjoints Systems Rigid dedicated stacks Teams, processes & tools Server, storage, networking, applications, security Funding and governance Departmental budgeting, ownership and accountingCopyright 2013 Freeform Dynamics Ltd 20
  • 18. How much have you virtualised the following elements of your IT landscape? 0% 20% 40% 60% 80% 100% Your x86 server estate Storage infrastructure Your corporate network Your desktop environment Totally Extensively Partially A bit Not at all UnsureEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 21
  • 19. How much have you architected your IT infrastructure in the form of shared resource pools such a private clouds? Unsure Totally 6% 4% Extensively 11% Partially 20% Not at all 40% A bit 19%Enterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 22
  • 20. YOUR VISION FOR DATA CENTRE COMPUTING Following charts based on this question: Putting all of the existing constraints and the current state of the industry to one side for a minute, how desirable would the following be as part of your perfect IT vision?Copyright 2013 Freeform Dynamics Ltd 23
  • 21. THE CLOUD HOSTING THING (Perfect IT Vision) 0% 20% 40% 60% 80% 100% All/most of our IT requirements would be fulfilled via hosted cloud services All/most of our own IT infrastructure would be based on private clouds Highly desirable-5 4 3 2 Not at all desirable-1 Unsure or N/AEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 24
  • 22. THE WHOLE CLOUD THING 0% 20% 40% 60% 80% 100% All/most of your IT requirements are fulfilled via hosted cloud services All/most of your in-house (or co-located) IT infrastructure is based on private cloud architecture Already there Within 1 yr Within 3 yrs Within 5 yrs Within 10 yrs Later Never Dont knowEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 25
  • 23. OPERATIONS & MANAGEMENT (Perfect IT Vision) 0% 20% 40% 60% 80% 100% We’d be able to provision workloads and new systems capacity on our private clouds with a few clicks on a management console Automation would mean that shared resource usage was continually optimised as demands fluctuate, with no human intervention We’d be able to migrate applications and workloads back and forth between public and private clouds with ease Hybrid cloud management capability would allow us to take an agnostic view of resources, mixing and matching internal and external capacity freely We would have end-to-end visibility across on-premise and hosted systems for management and troubleshooting purposes We would have a consistent/joined-up way of managing security and access across in-house and hosted systems We would have a consistent/joined-up way of managing and protecting data across in-house and hosted systems We’d be taking a unified approach to operations and management across servers, storage, networking, security, etc Highly desirable-5 4 3 2 Not at all desirable-1 Unsure or N/AEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 26
  • 24. OPERATIONS AND MANAGEMENT 0% 20% 40% 60% 80% 100% New workloads and systems capacity are generally provisioned with just a few clicks on a console Use of key shared resources is continually optimised as demands fluctuate, with no human intervention You can quickly and easily move workloads back and forth between public and private clouds An agnostic view of resourcing means choices between internal and external deployment are made purely on requirements and fitness for purpose You have end-to-end operational visibility across on-premise and hosted systems You have a consistent/joined-up way of managing security across in-house and hosted systems You have a consistent/joined-up way of managing data across in- house and hosted systems Already there Within 1 yr Within 3 yrs Within 5 yrs Within 10 yrs Later Never Dont knowEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 27
  • 25. ORGANISATION & SERVICES (Perfect IT Vision) 0% 20% 40% 60% 80% 100% Server, storage, networking, security and other specialists would be working together seamlessly as part of a fully integrated ops team Other IT teams (developers, testers, support staff, departmental IT, etc) would have self-service provisioning capability to obtain IT resources Non-technical users would have self-service provisioning capability for new applications and services IT activity and investment would revolve around the concept of business services rather than systems We would be able to easily and accurately charge or report IT costs back to the business based on activity or consumption Highly desirable-5 4 3 2 Not at all desirable-1 Unsure or N/AEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 28
  • 26. ORGANISATION AND SERVICES 0% 20% 40% 60% 80% 100% Self-service provisioning is in place for IT teams Self-service provisioning is in place for end users IT activity and investment revolves around the concept of business services rather than systems You can easily and accurately charge or report IT costs back to the business based on consumption Already there Within 1 yr Within 3 yrs Within 5 yrs Within 10 yrs Later Never Dont knowEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 29
  • 27. How much are the following standing in the way of progressing towards the vision? 0% 20% 40% 60% 80% 100% Inability of suppliers to deliver on visions and promises Lack of interest/appreciation from senior management The business not ready to upset the status quo IT not ready to upset the status quo Cultural impediments to investment in shared infrastructure Ingrained IT funding models don’t support new ways of working Historical under-investment means the mountain’s too high to climb Lack of a formally defined vision and strategy We simply don’t have the time, resources or budget to focus on anything other than short term priorities Big impediment Significant challenge Minor challenge Not a problem UnsureEnterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 32
  • 28. When it comes to driving forwards, how is it playing out, or how do you think it’s going to play out in your organisation? Single big transformational initiative to modernise things Build a modern Other across the board environment for new stuff, Things will be left largely 4% 4% and leave older systems as they are for the where they are foreseeable future 11% 18% Build a modern Just let adoption of environment for new stuff, modern architectures, and migrate older tools and techniques systems into it creep along in an ad hoc incrementally manner 43% 20%Enterprise 481 respondents Copyright 2013 Freeform Dynamics Ltd 33
  • 29. Summing Up  Can you have you cake (OK Apple) and eat it?Copyright 2013 Freeform Dynamics Ltd 34
  • 30. Moving forwards Senior business • Understanding is needed • Communicate – often and in language business users can understand awareness raising • Ensure governance processes are ‘big data aware’ Minimisation of ad • Proactive local involvement in planning/prioritisation • Don’t forget about getting ‘big data’ derived results back into business use hoc adoption • Clear policies/discipline around data usage • Monitor usage patterns to spot trends early Identify and deal with • Don’t get hung up on transient fads and fashions real business issues • Embrace, substitute or block more persistent activity • More of an orchestration approach to IT leadership Facilitate flexibility via • Architect systems with hard core and flexible edges core IT • Virtualise the edge to handle diversity and personal useCopyright 2013 Freeform Dynamics Ltd 36
  • 31. Adoption attitudes and tactics Resist new ideas and solutions Recipe for Allow new stuff to creep in passively disappointment Ad hoc opportunistic adoption Draw a line, target new apps only Recommended Establish beachhead, then expand by most early movers Big bang migration of everythingCopyright 2013 Freeform Dynamics Ltd 37
  • 32. Available for download now: Big Data and Analytics Dazzling new solutions or irritating new hype? Available from www.freeformdynamics.com http://www.freeformdynamics.com/fullarticle.asp?aid=1590 A Vision for the Data Centre Are you a Mover, Dreamer or Traditionalist? Available from www.freeformdynamics.com http://www.freeformdynamics.com/fullarticle.asp?aid=1604Copyright 2013 Freeform Dynamics Ltd 38
  • 33. Thank You Questions? Comments?Copyright 2013 Freeform Dynamics Ltd 39