This document discusses big data and the challenges of working with large datasets. It notes that every 2 days now as much data is created as was created from the beginning of civilization until 2003. The Hadoop ecosystem, including tools like MapReduce and machine learning, are proposed solutions for analyzing large and diverse datasets, but challenges remain around usability, speed of analysis, and finding new applications beyond web logs.

2. • How big is data now ?
• 5V
• Hadoop ecosystem
• Challenges
• Landscape

3. HOW BIG IS DATA NOW ?

4. EVERY 2
DAYS

5. EVERY 2 DAYS
WE CREATE AS
MUCH
INFORMATION
AS WE DID UP
TO 2003

6. INTHE DEVELOPED ECONOMIES OF EUROPE,
GOVERNMENT ADMINISTRATORS COULD SAVE MORE
THAN €100 BILLION ($149 BILLION) IN
OPERATIONAL EFFICIENCY IMPROVEMENTS ALONE BY
USING BIG DATA

7. 5V

8. VOLUME

9. VELOCITY

10. VARIETY
• Text
• Picture, video
• Location
• Health
• …

11. VERACITY (TRUST)

12. volume
veracity
variety
velocityvalue

13. VALUE EXAMPLES
• 1.9 million IT jobs will be created in the US by
2015 to carry out big data projects
• Identify potential crime areas three times more
accurately
• Healthcare industry could save $1,000 / person a
year

14. HADOOP ECOSYSTEM

15. MAPREDUCE

16. GOOGLE FILE SYSTEM

17. HADOOP ECOSYSTEM

18. MACHINE LEARNING

19. CHALLENGES
• Making tools easier to use. More mature software
• Getting quicker answers across large data sets
• Adoption
• Find other use cases than web log

20. LANDSCAPE

22. WHAT’S NEXT ?