Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data has Big Problems

509 views

Published on

The simplest definition of Big Data is large and complex unstructured data (images posted on Facebook, email, text messages, GPS signals from mobile phones, tweets, and other social media updates…etc.) that cannot be processed by traditional database tools.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Big Data has Big Problems

  1. 1. Big Data has Big Problems The simplest definition of Big Data is large and complex unstructured data (images posted on Facebook, email, text messages, GPS signals from mobile phones, tweets, and other social media updates…etc.) that cannot be processed by traditional database tools. As far back as 2001, industry analyst Doug Laney (currently with Gartner) articulated the now- mainstream definition of big data as the Three Vs: volume, velocity, and variety.  Volume. Unstructured data streaming in from social media. Increasing amounts of sensor and machine-to-machine data being collected.  Velocity. Data is streaming in at unprecedented speed and must be dealt with in a timely manner.  Variety. Data today comes in all types of formats—structured, numeric data in traditional databases. Information created from line-of-business applications. But the road to fame is not an easy one, -- and Big Data is still a hot term in the IT and business worlds--, just read the New York Time’s article: “Eight (No, Nine!) Problems with Big Data by Gary Marcus and Ernest Davis on April 6, 2014” which listed 9 valid problems with big data:  Big Data never tells us which correlations are meaningful.
  2. 2.  Big data can work well as an adjunct to scientific inquiry but rarely succeeds as a wholesale replacement.  Many tools that are based on big data can be easily gamed.  The results of a big data analysis often turn out to be less robust than they initially seem.  The echo-chamber effect, which also stems from the fact that much of big data comes from the web.  The risk of too many correlations.  Big data is prone to giving scientific-sounding solutions to hopelessly imprecise questions.  Big data is at its best when analyzing things that are extremely common, but often falls short when analyzing things that are less common  The hype. That’s not the whole picture or “the big picture “of all the challenges and problems facing big data concept. I can add to the list the following challenges:  The privacy challenges around Big Data are nothing new, but with ‘Dark Data’ now becoming a popular talking point  Legal concerns over Big Data privacy are likely to become a serious issue in 2014, as consumers become more aware of the impact of Big Data on their lives.  With the rise of the Internet of Everything, more mobile devices and perhaps even drones buzzing over our heads, this is only going to generate even bigger data, sensory data and maybe even image data too, claims the IEEE Computer Society.  Discrimination: One of the touchiest subjects that need to be addressed is the way data can be used to discriminate. Last year, Microsoft’s Kate Crawford wrote that Big Data is leading to “more précises forms of discrimination,” with social media and health care data being the biggest worries.  Going against gut instincts: With most organizations pursuing Big Data to drive their decision-making, this could soon lead to conflict over the old way of doing things.  Companies are struggling to find the right talent (Data Scientists) capable of both working with new technologies and of interpreting the data to find meaningful business insights.  Data access and connectivity can be an obstacle. A majority of data points are not yet connected today, and companies often do not have the right platforms to aggregate and manage the data across the enterprise.  The technology landscape in the data world is evolving extremely fast. This means working with a strong and innovative technology partner that can help create the right IT architecture that can adapt to changes in the landscape in an efficient manner.  Leveraging big data often means working across functions like IT, engineering, finance and procurement and the ownership of data is fragmented across the organization. To address these organizational challenges means finding new ways of collaborating across functions and businesses.  Security concerns about data protection are a major obstacle preventing companies from taking full advantage of their data.
  3. 3. The NYT article ends with this statement “Big data is here to stay, as it should be. But let’s be realistic: It’s an important resource for anyone analyzing data, not a silver bullet. “ References http://blogs.wsj.com/experts/2014/03/26/six-challenges-of-big-data/ http://www.nytimes.com/2014/04/07/opinion/eight-no-nine-problems-with-big-data.html?_r=1 http://venturebeat.com/2013/12/04/the-3-big-problems-in-big-data-hint-theyre-all-about-people/ http://siliconangle.com/blog/2014/01/03/3-big-data-problems-facing-businesses-in-2014/

×