The document provides an overview of various cloud computing, big data, and web development projects. It summarizes achievements in cloud infrastructure using OpenStack and OpenShift, building Hadoop clusters for big data analytics, and developing web applications. It outlines next steps of integrating OpenShift with OpenStack, implementing real-time data processing using HBase, and automating matching between farmers and food processors for a web application.

2. ‫تقدير‬ ‫و‬ ‫شكر‬

3. TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE

4. FaaS(Food as a service )
ICT(Information and Communication Technology)

5. Cloud computing

6. Project objectives
•1-

7. Project objectives
2.

8. Project objectives
3.

9. Project objectives
4.

10. Project Achievements

11. TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE

12. Introduction to Cloud team
What, which, how and why?

13. • In the simplest terms, cloud computing means storing and accessing data and programs
over the Internet instead of your computer's hard drive.
• When it's smarter to rent than to buy…
• Cloud computing is an emerging computing technology that uses the internet and
central remote servers to maintain data and application.

14. But where does The Cloud come in??

15. Hardware
Operating System
App App App
Traditional Stack
Hardware
OS
App App App
Hypervisor
OS OS
Virtualized Stack
Key Technology: Virtualization

16. A complex system consisting of
complex subsystems

17. ArchitectureCloud Computing
Hardware
CentOS
KVM
OpenStack
Big
Data
Web
Open
Shift

18. Host OS. distro

19. Why centos?
• Familiar of deploy and configure “recently released RDO”.
• Community support.
• Performance benchmarks “relating to updates”.
• Number of bugs.

20. Why GO open?
• Multi-Vendor Virtual.
• Brings the efficiency, agility, and cost benefits of cloud.
• prevents one vendor from controlling your access to
the greatest innovation
• Large supporting community.

21. ArchitectureCloud Computing
Hardware
CentOS
KVM
OpenStack
Big
Data
Web
Open
Shift

22. Virtualization

25. ArchitectureCloud Computing
Hardware
CentOS
KVM
OpenStack
Big
Data
Web
Open
Shift

27. OpenStack Architecture
Horizon
Nova
Neutron
SwiftCinder
Keystone
Glance
Ceilometer
Heat

28. OpenStack Installation
• Preparation for installation:
• Update OS.
• Remove Network manager.
• Add repositories.
Installation
Manual
installation
Scripted
installation

29. Installing services one by one
“Manual installation”
• Service “keystone”
1. Installing the service:
# yum install openstack-keystone python- keystoneclient
2. Configure the service:
# openstack-config--set/etc/keystone/keystone.conf sqlconnectionmysql://keystone:KEYSTONE_DBPASS@controller/keystone
3. Start and enable Service
#serviceopenstack-keystonestart
#chkconfigopenstack-keystone on
• Service “glance”
• Service “nova”
• Enable networking
• Add the dashboard “horizon”

30. OpenStack scripted deployment “RDO”
An automated installation with minimal user intervention.
Answer file is a simple text file contains configurations of OS.
Quick start:
1. Getting software repositories.
# yum update –y # yum install -y http://rdo.fedorapeople.org/rdo-release.rpm
2. Getting Packstack installer.
# yum install -y openstack-packstack
3. Run Packstack to install openstack
# packstack--allinone

31. OpenStack administration | cloud images
 VM Image:
Virtual disk that has a bootable operating system installed on it.
 OpenStack supported image formats:
 RAW
 QCOW2: commonly used with the KVM hypervisor.
• The image size is smaller.
• Support for snapshots.
 VM Image metadata:
 CPU architecture.
 The OS installed on the image (Linux, Windows,… ).
 OS_Distro (Centos, Fedora, Ubuntu,…).

32. OpenStack Linux image requirements to be sure that the
image supports the OpenStack features:
• Resize root partition on boot.
• No hard-coded MAC address information.
• Ensure ssh server runs.
• Access instance by using ssh public key.
• Use cloud-init to fetch the public key.
• Inject ssh key into the instance in the first boot.
• Process user data (host name, ssh key).
 Disable firewall and use the host firewall.

33.  We built two cloud images:
 Centos cloud image:
 LAMP web server cloud Image:
• Linux operating system (Centos).
• Apache HTTP Server is the most popular web server.
• MySQL database.
• PHP server-side scripting language designed for web.
 Tools to modify images:
 Guestfish: modify the files inside image.
 Guestmount: mount the image's file system directly.

34. OpenStack administration | Internal Network Architecture

35.  manage project.
 View and manage quotas.
 manage user.
 Manage flavors.
OpenStack administration | Admin user

36. OpenStack administration | End user
• Overview about menus.
• Creating network.
• Creating instances.
• Setting IP’s.
• Ping instances.

37. ArchitectureCloud Computing
Hardware
CentOS
KVM
OpenStack
Big
Data
Web
Open
Shift

38. What is Platform as a Service (PaaS)?
PaaS helps organizations, specifically developers, realize the benefits of cloud computing by providing
a fast and scalable way to host applications in the cloud.
Technically, a PaaS is an Application Platform comprised of an operating system, middleware and other
software that allows applications to run on the cloud with much of the management.
Platform as a Service is changing the way developers approach developing software.

39. What is OpenShift Origin ?
• OpenShift Origin is Red Hat’s open source Platform as a Service (PaaS)
offering.
• OpenShift Origin is an application platform where application
developers and teams can build, test, deploy, and run their
applications.
Code
Deploy
Enjoy

40. Flavors of OpenShift
origin
Public
Cloud
Service
On-premise
or Private
Cloud
Software
Open
Source
Project

41. Modernizing App Development with PaaS

42. OpenShift Machines

43. Key OpenShift Terms
• Broker – Management host, orchestration of Nodes
• Node – Compute host containing Gears
• Gear – Allocation of fixed memory, compute, and storage resources
for running applications
• Cartridge – A technology/framework (PHP, Perl, Java/JEE, Ruby,
Python, MySQL, etc.) to build applications.

44. Simple Application Creation

45. ArchitectureCloud Computing
Hardware
CentOS
KVM
OpenStack
Big
Data
Web
Open
Shift

46. TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE

47. Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive Sqoop
Cloudera
Platform
Eco
Systems

48. Introduction to hadoop

49. What Makes it Big Data?

50. Introduction to hadoop
Big Data
How to store? How to work?

52. Hadoop environment

53. High level hadoop architecture

54. Introduction to HDFS

55. Introduction to MapReduce

56. The important thing in hadoop is
configuration file

57. Practice work at lab

58. Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive Sqoop Cloudera
Manager
and New
Solutions
Eco
Systems

59. What’s the problem ?
2 TB(compresed) raw data per day today

60. What’s the solution?
HIVE

61. Data Warehousing at Facebook Today
Web Servers
Scribe Servers
Filers
Hive on
Hadoop Cluster
Oracle RAC Federated MySQL

62. HIVE Components

63. HIVE x BIG

64. Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive Sqoop Cloudera
Manager
and New
Solutions
Eco
Systems

65. Sqoop
•Why Sqoop
•What’s Sqoop
•How Sqoop works

66. Import data
1. Run Import
2. Pull metadata
3. Lunch
MapReduce job
4. Pull data from
database
5. Writ to data
sink
sqoop import --connect jdbc:mysql://vm2.faas.com/sqoop --username root --password root --table yelid --
target-dir /input/

67. Export data
1. Run Sqoop Export
2. Launch
MapReduce Job
3. Write to staging table
4. sql> INSERT INTO target
(SELECT*FROM staging)
sqoop export --connect jdbc:mysql://vm2.faas.com/sqoop --username root --password root --table location --export-dir
location

69. Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive Sqoop Cloudera
Manager
and New
Solutions
Eco
Systems

70. What’s Cloudera CDH?
• CDH : Is 100% Open Source Distribution including the core elements of Hadoop plus
several open source projects.

71. What's Inside?
• Apache Yarn : (Yet Another Resource Negotiator)
Is the data operating system of Hadoop that enables
you to process data simultaneously in multiple ways.

72. What's Inside?
• Apache HUE : Hue is a suite of applications that provide web-based access to CDH
components and a platform for building custom applications.

73. What’s Cloudera Manager C.M?
• Cloudera Manager : is a unified management interface that makes it
easy to install, configure, and manage a CDH cluster through a web interface “Admin
Console”

74. Other Solutions
1-Hortonworks :
100% Open Source Enterprise Apache Hadoop.

75. TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE

76. System Overview
Web Design and
Development
Big Data Solution
Cloud Computing
Web Design and
Development

77. TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE

78. Our Next Step!.
• Cloud: Integrate openShift with openStack.
• Big Data: Online data processing (real time) using HBase.
• Web: Automate matching between farmer and food processor.