The document discusses the intersection of cloud computing and big data as a critical frontier for innovation, emphasizing their capacity to handle vast amounts of data generated by modern scientific endeavors. It outlines the challenges posed by big data, particularly in storage, management, and analysis, necessitating new technologies and methodologies such as NoSQL systems and enhanced data processing techniques. Additionally, it highlights the necessity for the scientific community to adapt and innovate in order to fully leverage the potential of big data and cloud computing.

1. Cloud Computing y Big Data,
próxima frontera de la innovación
Cloud Computing and Big Data,
the next frontier of innovation
Jordi Torres, UPC-BSC
Madrid, 21 Marzo 2013

2. HOW DID SCIENCE START?

3. Source: Prof. Mateo Valero, BSC-CNS 2010

4. Source: Prof. Mateo Valero, BSC-CNS 2010

5. HOW IS SCIENCE ADVANCING TODAY?

6. Source: Prof. Mateo Valero, BSC-CNS 2010

7. Source: Prof. Mateo Valero, BSC-CNS 2010

8. MATHEMATICAL CALCULATIONS?
WHERE?

10. MN3
Cores/chip 8
Chip/node 2
Compute Cores/node 16
Nodes 3028
Total cores 48448
Freq. 2,6
Gflops/core 20,8
Performance
Gflops/node 332,8
Total Tflops 1000,0
GB/core (GB) 2
Memory GB/node (GB) 32
Total (TB) 96,89
Latency (μs) 0,7
Network
Bandwidth (Gb/s) 40
Storage (TB) 2000
Consumption (KW) 1080

11. FOR SOME SPANISH RESEARCH GROUPS!

12. AND…
FOR THE REST OF THE WORLD?

13. GOOD NEWS!
Source: http://news.cnet.com/8301-13846_3-57349321-62
/amazon-takes-supercomputing-to-the-cloud

14. CLOUD COMPUTING?

15. Source: http://www.wired.com/wiredenterprise/2011/
12/nonexistent-supercomputer/all/1

16. Source: http://www.facebook.com/media/
set/?set=a.190842620965185.47008.140375289345252
40 Mw
28.000 m2

17. Foto: Google

18. HUGE DATA CENTERS
Foto: Google
> football pitch x 4

19. Source: http://www.google.com/about/datacenters/gallery/images

20. Source: http://www.google.com/about/datacenters/gallery/images

21. Source: http://www.google.com/about/datacenters/gallery/images

22. Different IT
production
Foto: J.T.

23. CLOUD COMPUTING:
IT as a service
On-demand self-service Pay per use
Rapid elasticity Ubiquitous access
....
Source: http://www.telegraph.co.uk/technology
/reviews/9241719/Power-Ethernet-Sockets-review.html

24. Example of benefits (IaaS):
1 computer in a rack
for 120 hours 120 computers in three
racks for 1 hour
Idea : Tutorial SC2011 - Robert Grossman

25. AND DATA?

26. Source: http://www.docuciencia.es/2009/05/lhc-el-acelerador-de-particulas/
“… the LHC produces 1PetaByte of data every second, big data and
lack of computing resources were becoming the European Organization
for Nuclear Research’s biggest IT challenges…”
Source: computerweekly.com/news/2240173897/CERN-adopts
-OpenStack-private-cloud-to-solve-big-data-challenges

27. 1 Gigabyte (GB) = 1.000.000.000 byte
1 Terabyte (TB) = 1.000 Gigabyte (GB)
1 Petabyte (PB) = 1.000.000 Gigabyte (GB)
1 Exabyte (EB) = 1.000.000.000 Gigabyte (GB)
1 Zettabyte (ZB) = 1.000.000.000.000 (GB)

28. Deluge of data created daily
Source: Economist , Feb 25th, 2010 http://www.economist.com/node/15579717

29. Big Data?
definition?

30. BIG DATA?
Big Data is data that exceeds the
storing, processing and managing
capacity of conventional systems.

31. BIG DATA?
The reason is that the data is too
big, moves too fast, or doesn’t fit
the structures of our current systems’
architectures.

32. BIG DATA?
Moreover, to gain value from this
data, we must change the way to
analyze it.

33. BIG DATA?
Big Data is data that exceeds the storing,
processing and managing capacity of
conventional systems.
The reason is that the data is too big,
moves too fast, or doesn’t fit the
structures of our current systems’
architectures.
Moreover, to gain value from this data, we
must change the way to analyze it.

34. NEW CHALLENGES
that must be addressed urgently, in order to respond
to the needs of the advancement of science
1. Storing
2. Managing
3. Processing
4. Analyzing

35. Affordable Storage

36. But scanning disks…
assume 100MB/sec

37. But scanning disks…
assume 100MB/sec
more than 5 hours

38. approach: massive parallelism
assume 20.000 disks:
scanning 2 TB takes 1 second
Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg

39. 1 Data processing challenges
Rethinking data processing is required:
MapReduce, Storm, S4,…
Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg

40. 2 Data storage challenges
New Storage technologies are required
HHD 100 cheaper than RAM
But 1000 times slower
RAM vs HHD
Solid- state drive (SSD)
Not volatile
Present solutions:
Storage Class Memory (SCM)
Research:

41. 3 Data management challenges
Relational DB can’t support everything
Example: eventual consistency
Solution: “NoSQL systems”
Research: New management systems
Source: gigaom.com/cloud/big-data-
and-nosql-march-to-the-enterprise/
43

42. 4 Obtaining value from data
The information is non actionable knowledge
-
Data prediction using data mining &
+ machine learning techniques
Value
Volume
Information
Research: The majority of algorithms
function well in thousands of
+ registers, however at the moment they
Knowledge - are impractical for thousands of
milions.

43. Cloud Computing
and Big Data:
the next frontier of
science and
innovation

44. Thank you for your attention
www.JordiTorres.org - @JordiTorresBCN
www.smartcityexpo.com www.bsc.es/eBusiness
Autonomic Systems and e-Business Platforms research line at BSC/UPC

46. Thank you for your attention
www.JordiTorres.org - @JordiTorresBCN
www.smartcityexpo.com www.bsc.es/eBusiness
Autonomic Systems and e-Business Platforms research line at BSC/UPC