Intro to Big Data and Apache Hadoop by Dr. Amr Awadallah at CLOUD WEEKEND '13...
Cw13 build open hybrid cloud by diaa radwan-red hat
1. 1 Diaa Radwan | Building Open Hybrid Cloud
Build Open Hybrid Cloud
Diaa Radwan
Red Hat, Solution Architect
Cairo, May 12th
2013
@DiaaRadwan
diaa@redhat.com
2. 2 Diaa Radwan | Building Open Hybrid Cloud
Status Of
Open Source Software
Adoption
3. 3 Diaa Radwan | Building Open Hybrid Cloud
All This Innovation Has Led To Cloud
4. 4 Diaa Radwan | Building Open Hybrid Cloud
Role Of
Open Source Software
In The Cloud
Paradigm Shift
5. 5 Diaa Radwan | Building Open Hybrid Cloud
What Do These Numbers Represent?
28
24
50
6. 6 Diaa Radwan | Building Open Hybrid Cloud
They Represent …
28
Stock exchanges, across
24
countries, run Red Hat Enterprise Linux to handle
more than
50%
of the world’s trading volume
7. 7 Diaa Radwan | Building Open Hybrid Cloud
● Standard and commodity based approach
● 100x performance improvements over legacy OS
● Enable customer innovation
● Model of open, empowered IT
8. 8 Diaa Radwan | Building Open Hybrid Cloud
Open Source Communities As The Foundation
9. 9 Diaa Radwan | Building Open Hybrid Cloud
Pressure On Traditional IT
From Public Cloud
Services
10. 10 Diaa Radwan | Building Open Hybrid Cloud
What Do These Numbers Represent?
31 - 65
67
80
11. 11 Diaa Radwan | Building Open Hybrid Cloud
They Represent …
31 - 65%
The rising expectations of IT over the past few years,
according to a survey of boards of directors.
67%
of IT budgets are shrinking
80%
of IT budgets = keeping the lights on
12. 12 Diaa Radwan | Building Open Hybrid Cloud
Making IT Relevant To The Business
● What if you could...
Respond to the increasing requirements of the business,
delivering innovative solutions to them that are strategic.
● What if you could...
Contain costs more effectively, and deliver services more
efficiently to your constituents.
● What if you could...
Ensure stability and reliability of the infrastructure and
services you provide, while allowing flexibility to seamlessly
move into the future.
13. 13 Diaa Radwan | Building Open Hybrid Cloud
The Role Of IT Is Changing
From Service Provider To Strategic Partner
14. 14 Diaa Radwan | Building Open Hybrid Cloud
Modern Data Center
of the Future
15. 15 Diaa Radwan | Building Open Hybrid Cloud
● Driving towards more dynamic data centers
● Cloud like scale and elasticity
● Commodity building blocks for compute, data, and
network
● Open standards based technologies to enable open
hybrid architectures
18. 18 Diaa Radwan | Building Open Hybrid Cloud
● Pool of resources (compute, storage, network)
● Nodes can run native or virtualized
● (Re)deployment/growth as needed and required by
business
19. 19 Diaa Radwan | Building Open Hybrid Cloud
Compute
Enterprise Virtualization
20. 20 Diaa Radwan | Building Open Hybrid Cloud
● Scale out and scale up
● Consistent platform with RHEL
● Virtualized mission critical workloads with RHEV
21. 21 Diaa Radwan | Building Open Hybrid Cloud
● Load balancing
● Seamless workload migration
22. 22 Diaa Radwan | Building Open Hybrid Cloud
● Seamless data migration with storage migration
23. 23 Diaa Radwan | Building Open Hybrid Cloud
Data
Storage and Big Data
24. 24 Diaa Radwan | Building Open Hybrid Cloud
● Swimming in a sea of data: 2.5 quintillion bytes of data per day
● Amount of data produced doubles every 3 years
● Equivalent to 12 terabytes of tweets... (per day)
25. 25 Diaa Radwan | Building Open Hybrid Cloud
● Need for elastic storage
● Grows and scales as you need across infrastructures
● Built on commodity infrastructure
26. 26 Diaa Radwan | Building Open Hybrid Cloud
● Geo replication
● Serve content more efficiently
27. 27 Diaa Radwan | Building Open Hybrid Cloud
● Geo replication
● Serve content more efficiently
28. 29 Diaa Radwan | Building Open Hybrid Cloud
● Gluster as platform for big data
● Integration with Hadoop
● Build on demand big data storage clusters
31. 32 Diaa Radwan | Building Open Hybrid Cloud
● Dynamic infrastructure driving the need for dynamic network
● Rapid changes based on IT demands
● Open vswitch now part of Linux kernel
32. 33 Diaa Radwan | Building Open Hybrid Cloud
● Network configurations to bridge infrastructures
● Ability to move workloads
33. 34 Diaa Radwan | Building Open Hybrid Cloud
● Need to connect local and public cloud resources
● Openflow integral part for data center networking
34. 35 Diaa Radwan | Building Open Hybrid Cloud
● Building the foundation for a dynamic node-based data center
● Elastic infrastructure to deploy/allocate nodes on demand
● Transparent availability of resources/nodes
35. 36 Diaa Radwan | Building Open Hybrid Cloud
Building A Scalable
Cloud Infrastructure
36. 37 Diaa Radwan | Building Open Hybrid Cloud
Open Hybrid Clouds Provide That Balance
37. 38 Diaa Radwan | Building Open Hybrid Cloud
● Desire to bridge public and private cloud
● Resource allocation across infrastructures
● Today in silos and disparate tooling
● Open hybrid cloud for common tooling and deployments
38. 39 Diaa Radwan | Building Open Hybrid Cloud
● Openstack provides modular architecture
● Enable the use of common components across infrastructure
● Contributions by many industry leaders
● Component and API standardization
39. 40 Diaa Radwan | Building Open Hybrid Cloud
Infrastructure Agnostic
Management
40. 41 Diaa Radwan | Building Open Hybrid Cloud
● Take advantage of resource wherever they are
● Cater for security, SLA and location requirements
● Create seamless pool of resources
41. 42 Diaa Radwan | Building Open Hybrid Cloud
● Drives the need for management tooling
● Consistent tools independent of deployment and location
42. 43 Diaa Radwan | Building Open Hybrid Cloud
● Cloudforms/ManageIQ provides a common and open framework
● Execute administrative tasks across clouds
● Define “once” , deploy “multiple”
● Describe security, configuration and dependencies once
43. 50 Diaa Radwan | Building Open Hybrid Cloud
Setting The Scene For
The Next Decade
44. 51 Diaa Radwan | Building Open Hybrid Cloud
What Do These Industry Leaders
All Have In Common?
45. 52 Diaa Radwan | Building Open Hybrid Cloud
What Do These Industry Leaders
All Have In Common?
● Creativity is a way of life for
DreamWorks and they were limited by:
● high cost proprietary hardware &
software lacking scalability
● manual rebalance of resources to
address storage supply and
demand
Produce more 2D & 3D films without
compromising creative ambitions with:
A scalable infrastructure
Ability to render fluidly across their hybrid
architecture with workloads directed to
the best location without manual
intervention
All at lower costs
BEFORE AFTER
● Delivering more value than the
customer expects is the goal at Frost.
They were feeling constrained
because:
● Legacy e-commerce application
server platform
● Unable to develop and scale
applications to meet their rate of
growth cost-effectively
● Able to deliver on their company
promise with:
● A modernized application
infrastructure allows for flexibility
and scalability.
● Increased developer and admin
productivity and efficiency means
innovation to the front line more
quickly
● In the travel industry efficiency is a
competitive advantage but with
multiple complex platforms and
vendor lock-in, it was hard to be as
efficient.
● Efficiency could be Emirates middle
name now with one standardized
platform - costs has been greatly
reduced, choice and security gained
with no system downtime.
46. 53 Diaa Radwan | Building Open Hybrid Cloud
IT Transformation
Business Needs Driving IT
47. 54 Diaa Radwan | Building Open Hybrid Cloud
IT Transformation For Business
Applications And Tools Are Available Across Your Infrastructure
48. 55 Diaa Radwan | Building Open Hybrid Cloud
IT Transformation For Operations
Infrastructure Is Easily Managed, Rapidly Deployed,
And Allows For Innovation
49. 56 Diaa Radwan | Building Open Hybrid Cloud
IT Transformation For Developers
Tools And Platforms Are Self-Service
50. 57 Diaa Radwan | Building Open Hybrid Cloud
Open Innovation Is Shaping The Future Of IT
51. 58 Diaa Radwan | Building Open Hybrid Cloud
Red Hat Open Hybrid Cloud
