Mohammad Fairus Khalid
Cloud Computing as
Innovation Hub
Agenda
• Motivation
• Technology
• Paradigm Shift
• Use Cases
The First pay-per-use Cloud Computing
Traditionally Why People Move to Cloud
• Moving away from the hardware silo
• Optimized use of resources
• Pay as you use
• Fast deployment and ease of maintenance
…but World has Changed…
MOBILITY
SOCIAL MEDIA
The Internet of Things – infographic The Connectivist based on Cisco data
INTERCONNECTED
INSIGHTS
The World Changed
• Access from anywhere at anytime with any
devices
• Employing more sensors to collect more
data and employing more controllable
devices
• Collaborative in term of application, data
and people efforts
• Greater Intelligence – better decision
making for – health, safety, business,
sports and etc
How Cloud Get Impacted
• No longer just a platform to provide
computing power
• No longer just about cost saving
• Moving away from application and data
silo
• Platform to collaborate – application, data
and people
• Platform to analyze and that help to
prescribe better decision making
Big Data Landscape
Service Stacks
Infrastructure Fabric
Data Collection Fabric
Data Fabric
Analytics Fabric
Applications Fabric
Layer of Complexity
Compute
Storage
Network
CPU
GPU
Virtual
Machine
Container
Operating
System
Platform
Application
File system
Small
Devices
Agent
Big Data
Database
Load balance
People Crowd
Sourcing
Device Crowd
Sourcing
Physical LayerPhysical Abstraction
Layer
Service Abstraction
Layer
Give me ubuntu. I want to
Install my own app
I already have nodejs app.
I just want run the app
My business is expending.
I need crm system.
I want to store a file
What is the relationship
between weather and
durian
I want database size can
grow from 10GB to 100 GB
I want to scale the web
server
Send me pictures of
sea games event
Tell me the bandwidth
consumption pattern in
Puchong
User Stories
Technology
• IT Resource Abstraction
• Minimal Operating System
• Collaborative Tools
• GPU/Multi Core processing
• Distributed Storage
IT Resource Abstraction
• Hypervisor Virtualization Challenges
– Significant overhead
– Significant repetitive work – OS deployment,
configuration
• Container is the answer to this.
• There are 2 types of container
– Application Container
– System Container
Application Container - Docker
• Docker containers wrap up a piece of
software in a complete filesystem that
contains everything it needs to run: code,
runtime, system tools, system libraries –
anything you can install on a server. This
guarantees that it will always run the
same, regardless of the environment it is
running in.
Application Container - Docker
Server
Host OS
Docker Engine
bins/libs bins/libs
App1
App2
App3
App4
App5
App6
Server
Host OS
Hypervisor
Guest
OS
bins/
libs
App 1
Guest
OS
bins/
libs
App 1
Guest
OS
bins/
libs
App 1
Virtual
Machine
Container
Containers have similar resource isolation and allocation benefits as virtual
machines but a different architectural approach allows them to be much more
portable and efficient.
Application Container – Image Layer
Debian
nodejs
node-telegram-bot-api
nedb
gbsbot
IMAGE CREATED CREATED BY SIZE
52fde796cb83 24 hours ago /bin/sh -c #(nop) ENTRYPOINT &{["node" "./myb 0 B
f82158577794 24 hours ago /bin/sh -c #(nop) COPY multi:3b93737c8a69ee5b 39.11 kB
20bcf94ccdf9 24 hours ago /bin/sh -c #(nop) COPY file:d1ca5732ca7afe9b4 19.49 kB
0c9042ee2f2c 24 hours ago /bin/sh -c npm --unsafe-perm install node-tel 9.798 MB
db1e2a0888ac 24 hours ago /bin/sh -c npm --unsafe-perm install nedb 4.649 MB
46ff394d3bd9 9 days ago /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:/u 0 B
85db46af09e1 9 days ago /bin/sh -c mkdir /nodejs && curl http://nodej 17.51 MB
ed701745dfc9 9 days ago /bin/sh -c apt-get update -y && apt-get insta 184.4 MB
e9e824eeee9d 2 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
cba48ac2c991 2 weeks ago /bin/sh -c #(nop) ADD file:41fbf9f100beb80549 84.89 MB
System Container - LXD
• Full operating system functionality within containers, not
just single processes
• Maximum density of guests per host, providing a cost
benefit when running in a public cloud
• Rapid provisioning, instant guest boot
http://www.ubuntu.com/cloud/lxd
Boot Time
0.39
3.39
9.99
11.11
0.00
2.00
4.00
6.00
8.00
10.00
12.00
Avg Application Ready
BootTime(seconds)
Docker LXD wo Copy LXD KVM
Density
69.53
30.64
2.45
70.65
6.83 6.41
84.59
53.67
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00
1 51 101 151 201 251 301 351 401 451 501 551 601 651 701 751 801 851 901 951 1001
Memroy(%)
Number of Guests
Free Memory (KVM) Free Memory (Docker) Free Memory (LXD)
IT Resource Abstraction Use Cases
KVM LXD Docker
Use Case To run mixture of
different kernel
operating systems of
guest machines on the
same host.
To run full linux
operating system
functionality with the
same kernel version of
the host
To run linux
applications which
don’t have
dependency on the
linux kernel version
Examples Windows 7, Ubuntu
10.04, Ubuntu 14.04
Busybox, ubuntu same
kernel version
nodejs, tomcat,
postgres
Minimal Operating System
• Improved stability and security
• Minimize the overhead of tools
Minimal OS
• lightweight, immutable platform
• boot extremely quickly
• designed for security, consistency, and reliability
Benefit of Min OS
• minimize the need for frequent patching
• minimize vulnerability and attack space
• Use less resources CPU, RAM and
storage
• Increase stability
• Fast to boot 5 – 10 seconds
• Read only ensuring system integrity
• Transactional/atomic update – guaranteed
update/rollback
Minimal OS Comparison
Photon OS Snappy Ubuntu
Core
Atomic Host CoreOS Rancher OS
Initiated by Vmware open
source
Ubuntu open
source
Red Hat open
source
CoreOS open
source
Rancher open
source
Why extend the
security, identity
and resource
management to
containerized
workloads.
to ensure a
carrier-grade
update
experience
To provide secure
and reliable
foundation to
orchestrate
multi-container
based
applications at
scale
To provide
capability as
large web
companies like
Google
build a
minimalist Linux
distribution that
was perfect for
running Docker
containers
Supported
Framework
Docker, rkt,
Pivotal Garden
Open Docker Docker, rkt Docker
Size ~300 MB ~110 MB ~400 MB ~220 MB ~30 MB
Collaborative Tools
• Static
– Marketplace
– Repositories
• Dynamic
– API to services
– Social Media
– Mobile messaging
– Software code management
Paradigm Shift
• Application Programming Interface
– Open, enable collaboration
• Application Architecture - microservices
– Realtime scalability
– Mobility
• Application Design atomic/transactional
– Atomic update
Automation - API
• APIs are sets of requirements that
govern how one application can talk to
one and another.
Application Design- Transactional
• Separation between stateful and
stateless portion of the application.
• The stateless portion will keep as read-
only, which means they cannot be
tampered with and can be updated
perfectly and predictably every time.
• Delta management keeps the size of
downloads to the bare minimum.
• Transactional, image-based delta
updates for the system and applications
that can always be rolled back.
• Signatures and fingerprints ensure we
are running exactly what was published
by the developer.
Debian
nodejs
node-telegram-bot-api
nedb
gbsbot
data config
Application Architecture – Microservices
• developing a single
application as a suite
of small services
• runs in its own
process
• lightweight
communication
mechanisms, often an
HTTP resource API.
http://martinfowler.com/articles/microservices.html
Application Architecture – Microservices
• Strong Module Boundaries: Microservices
reinforce modular structure, which is particularly
important for larger teams.
• Independent Deployment: Simple services are
easier to deploy, and since they are
autonomous, are less likely to cause system
failures when they go wrong.
• Technology Diversity: With microservices you
can mix multiple languages, development
frameworks and data-storage technologies.
http://martinfowler.com/articles/microservice-trade-offs.html
Use Cases
Use Cases
https://www.hitrecord.org/
https://www.tm.com.my/securitysurveillance/Pages/ind
ex.html
https://www.data.gov/
Neighborhood Watch - challenge
1. To monitor real-time video
2. To recall event trigged video or
images
1. Emergency alert, panic
button
1. Incident
alerts
recap
• World changed
• Cloud computing provide opportunity
beyond cost confinement
• Using different technologies to cater for
new business need
• The way applications are architect,
implement and deployed need to change
• We should look cloud as innovation
platform
Cloud Computing as Innovation Hub - Mohammad Fairus Khalid

Cloud Computing as Innovation Hub - Mohammad Fairus Khalid

  • 1.
    Mohammad Fairus Khalid CloudComputing as Innovation Hub
  • 2.
    Agenda • Motivation • Technology •Paradigm Shift • Use Cases
  • 3.
    The First pay-per-useCloud Computing
  • 4.
    Traditionally Why PeopleMove to Cloud • Moving away from the hardware silo • Optimized use of resources • Pay as you use • Fast deployment and ease of maintenance …but World has Changed…
  • 5.
  • 6.
  • 7.
    The Internet ofThings – infographic The Connectivist based on Cisco data INTERCONNECTED
  • 8.
  • 9.
    The World Changed •Access from anywhere at anytime with any devices • Employing more sensors to collect more data and employing more controllable devices • Collaborative in term of application, data and people efforts • Greater Intelligence – better decision making for – health, safety, business, sports and etc
  • 10.
    How Cloud GetImpacted • No longer just a platform to provide computing power • No longer just about cost saving • Moving away from application and data silo • Platform to collaborate – application, data and people • Platform to analyze and that help to prescribe better decision making
  • 12.
  • 13.
    Service Stacks Infrastructure Fabric DataCollection Fabric Data Fabric Analytics Fabric Applications Fabric
  • 14.
    Layer of Complexity Compute Storage Network CPU GPU Virtual Machine Container Operating System Platform Application Filesystem Small Devices Agent Big Data Database Load balance People Crowd Sourcing Device Crowd Sourcing Physical LayerPhysical Abstraction Layer Service Abstraction Layer Give me ubuntu. I want to Install my own app I already have nodejs app. I just want run the app My business is expending. I need crm system. I want to store a file What is the relationship between weather and durian I want database size can grow from 10GB to 100 GB I want to scale the web server Send me pictures of sea games event Tell me the bandwidth consumption pattern in Puchong User Stories
  • 15.
    Technology • IT ResourceAbstraction • Minimal Operating System • Collaborative Tools • GPU/Multi Core processing • Distributed Storage
  • 16.
    IT Resource Abstraction •Hypervisor Virtualization Challenges – Significant overhead – Significant repetitive work – OS deployment, configuration • Container is the answer to this. • There are 2 types of container – Application Container – System Container
  • 17.
    Application Container -Docker • Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.
  • 18.
    Application Container -Docker Server Host OS Docker Engine bins/libs bins/libs App1 App2 App3 App4 App5 App6 Server Host OS Hypervisor Guest OS bins/ libs App 1 Guest OS bins/ libs App 1 Guest OS bins/ libs App 1 Virtual Machine Container Containers have similar resource isolation and allocation benefits as virtual machines but a different architectural approach allows them to be much more portable and efficient.
  • 19.
    Application Container –Image Layer Debian nodejs node-telegram-bot-api nedb gbsbot IMAGE CREATED CREATED BY SIZE 52fde796cb83 24 hours ago /bin/sh -c #(nop) ENTRYPOINT &{["node" "./myb 0 B f82158577794 24 hours ago /bin/sh -c #(nop) COPY multi:3b93737c8a69ee5b 39.11 kB 20bcf94ccdf9 24 hours ago /bin/sh -c #(nop) COPY file:d1ca5732ca7afe9b4 19.49 kB 0c9042ee2f2c 24 hours ago /bin/sh -c npm --unsafe-perm install node-tel 9.798 MB db1e2a0888ac 24 hours ago /bin/sh -c npm --unsafe-perm install nedb 4.649 MB 46ff394d3bd9 9 days ago /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:/u 0 B 85db46af09e1 9 days ago /bin/sh -c mkdir /nodejs && curl http://nodej 17.51 MB ed701745dfc9 9 days ago /bin/sh -c apt-get update -y && apt-get insta 184.4 MB e9e824eeee9d 2 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B cba48ac2c991 2 weeks ago /bin/sh -c #(nop) ADD file:41fbf9f100beb80549 84.89 MB
  • 20.
    System Container -LXD • Full operating system functionality within containers, not just single processes • Maximum density of guests per host, providing a cost benefit when running in a public cloud • Rapid provisioning, instant guest boot http://www.ubuntu.com/cloud/lxd
  • 21.
  • 22.
    Density 69.53 30.64 2.45 70.65 6.83 6.41 84.59 53.67 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 1 51101 151 201 251 301 351 401 451 501 551 601 651 701 751 801 851 901 951 1001 Memroy(%) Number of Guests Free Memory (KVM) Free Memory (Docker) Free Memory (LXD)
  • 23.
    IT Resource AbstractionUse Cases KVM LXD Docker Use Case To run mixture of different kernel operating systems of guest machines on the same host. To run full linux operating system functionality with the same kernel version of the host To run linux applications which don’t have dependency on the linux kernel version Examples Windows 7, Ubuntu 10.04, Ubuntu 14.04 Busybox, ubuntu same kernel version nodejs, tomcat, postgres
  • 24.
    Minimal Operating System •Improved stability and security • Minimize the overhead of tools
  • 25.
    Minimal OS • lightweight,immutable platform • boot extremely quickly • designed for security, consistency, and reliability
  • 26.
    Benefit of MinOS • minimize the need for frequent patching • minimize vulnerability and attack space • Use less resources CPU, RAM and storage • Increase stability • Fast to boot 5 – 10 seconds • Read only ensuring system integrity • Transactional/atomic update – guaranteed update/rollback
  • 27.
    Minimal OS Comparison PhotonOS Snappy Ubuntu Core Atomic Host CoreOS Rancher OS Initiated by Vmware open source Ubuntu open source Red Hat open source CoreOS open source Rancher open source Why extend the security, identity and resource management to containerized workloads. to ensure a carrier-grade update experience To provide secure and reliable foundation to orchestrate multi-container based applications at scale To provide capability as large web companies like Google build a minimalist Linux distribution that was perfect for running Docker containers Supported Framework Docker, rkt, Pivotal Garden Open Docker Docker, rkt Docker Size ~300 MB ~110 MB ~400 MB ~220 MB ~30 MB
  • 28.
    Collaborative Tools • Static –Marketplace – Repositories • Dynamic – API to services – Social Media – Mobile messaging – Software code management
  • 29.
    Paradigm Shift • ApplicationProgramming Interface – Open, enable collaboration • Application Architecture - microservices – Realtime scalability – Mobility • Application Design atomic/transactional – Atomic update
  • 30.
    Automation - API •APIs are sets of requirements that govern how one application can talk to one and another.
  • 31.
    Application Design- Transactional •Separation between stateful and stateless portion of the application. • The stateless portion will keep as read- only, which means they cannot be tampered with and can be updated perfectly and predictably every time. • Delta management keeps the size of downloads to the bare minimum. • Transactional, image-based delta updates for the system and applications that can always be rolled back. • Signatures and fingerprints ensure we are running exactly what was published by the developer. Debian nodejs node-telegram-bot-api nedb gbsbot data config
  • 32.
    Application Architecture –Microservices • developing a single application as a suite of small services • runs in its own process • lightweight communication mechanisms, often an HTTP resource API. http://martinfowler.com/articles/microservices.html
  • 33.
    Application Architecture –Microservices • Strong Module Boundaries: Microservices reinforce modular structure, which is particularly important for larger teams. • Independent Deployment: Simple services are easier to deploy, and since they are autonomous, are less likely to cause system failures when they go wrong. • Technology Diversity: With microservices you can mix multiple languages, development frameworks and data-storage technologies. http://martinfowler.com/articles/microservice-trade-offs.html
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
    Neighborhood Watch -challenge 1. To monitor real-time video 2. To recall event trigged video or images 1. Emergency alert, panic button 1. Incident alerts
  • 39.
    recap • World changed •Cloud computing provide opportunity beyond cost confinement • Using different technologies to cater for new business need • The way applications are architect, implement and deployed need to change • We should look cloud as innovation platform