The document discusses the challenges of building and managing big data and cloud applications, including outages, disaster recovery, complexity of multi-tier applications, and lock-in to specific clouds. It introduces Cloudify as a solution that allows deploying and managing applications across multiple clouds while avoiding vendor lock-in through portability and automation of monitoring and scaling.

1. GigaSpaces Cloudify
Any App, On Any Cloud, Your Way
February 2012

2.  Intro to Big data world & Cloud apps
 Challenges of big data applications
 Outages & disaster recovery
 Complexity of multi tiers application
 Lock-in to specific cloud
 Monitoring & Scaling
 PaaS make our life easier …
® Copyright 2013 GigaSpaces Ltd. All Rights Reserved2
AGENDA

3. WE’RE LIVING IN A REAL TIME WORLD…
Homeland Security
Real Time Search
Social
eCommerce
User Tracking &
Engagement
Financial Services
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved3

4. BIG DATA PREDICTIONS
“Over the next few years we'll see the adoption of scalable
frameworks and platforms for handling
streaming, or near real-time, analysis and processing. In the
same way that Hadoop has been borne out of large-scale web
applications, these platforms will be driven by the needs of large-
scale location-aware mobile, social and sensor use.”
Edd Dumbill, O’REILLY
4
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

5. 5
THE BIG GUYS…
• AWS – around 0.5M servers
• Facebook – less than 0.1M servers
• Google – around 1M servers

6. STORE 135+ BILLION MESSAGES A MONTH
6
® Copyright 2011 Gigaspaces Ltd. All Rights

7. THE OUTAGE PROBLEM
7

8. A TYPICAL BIG DATA SYSTEM…

9. 9
LOCKED IN TO SPECIFIC CLOUD?

10. 10
DISASTER RECOVERY

11. Realization:
What You
Really Care
about Is
App
Portability

12. http://code.zynga.com/2012/02/the-evolution-of-zcloud/

13. 13

14. 14
• No Code Change
• No lock-in
• Full Monitor & Control
Moving Apps At
Massive Scale –
The Right Way

15. CLOUDIFY POSITIONING IN THE CLOUD STACK
15
PaaS
IaaS
DevOps
(Automation)
Productivity
Control
ChefPuppet
CloudFoundry
Heroku
GAE
OpenShift
Rightscale
Public clouds
(AWS, Rackspace,..) Private clouds
(Vmware, OpenStack..)
High productivity with
full control
Enstratus

16. HOW DOES IT WORK?
1 Upload your recipes
2 Cloudify creates VMs & installs agents
3 Agents install and manage your app
4 Cloudify automates the scaling
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved16

17. EXTENSIVE PLATFORM SUPPORT
® Copyright 2012 GigaSpaces. All Rights Reserved17

18. BOOTSTRAPPING
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved18

19. INSTALL APPLICATION
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved19

20. POST DEPLOYMENT
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved20

21. SELF HEALING
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved21

22. GETTING COMPUTE RESOURCES IN A PORTABLE WAY
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved22
compute {
template "SMALL_LINUX"
}
SMALL_LINUX : template
imageId "us-east-1/ami-76f0061f“
remoteDirectory "/home/ec2-user/gs-files“
machineMemoryMB 1600
hardwareId "m1.small"
locationId "us-east-1"
localDirectory "upload"
keyFile "myKeyFile.pem"
options ([
"securityGroups" : ["default"]as
String[],
"keyPair" : "myKeyFile"
])
overrides (["jclouds.ec2.ami-query":"",
"jclouds.ec2.cc-ami-query":""])
privileged true
}
SMALL_LINUX : template{
imageId "1234"
machineMemoryMB 3200
hardwareId "103"
remoteDirectory "/root/gs-files"
localDirectory "upload"
keyFile "gigaPGHP.pem"
options ([
"openstack.securityGroup" : "default",
"openstack.keyPair" : "gigaPGHP"
])
privileged true
}

23. UNDERSTAND APPLICATION STRUCTURE & DEPENDENCIES
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved23
application {
name="petclinic"
service {
name = "mongod"
}
service {
name = "mongoConfig"
}
service {
name = "apacheLB"
}
service {
name = "mongos"
dependsOn = ["mongoConfig", "mongod"]
}
service {
name = "tomcat"
dependsOn = ["mongos","apacheLB"]
}
}

24. DYNAMICALLY CONFIGURE APPLICATION TIERS
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved24
1
23
serviceContext.attributes.thisInstance["port"] = config.port
dbServiceInstances = ServiceContext.attributes[“db”].instances
dbServicePort = dbServiceInstances[1].port
customCommands ([
"addNode" : "apacheLB_addNode.groovy",
"removeNode" : "apacheLB_removeNode.groovy",
"load" : "apacheLB-load.groovy“
])

25. AUTO SCALE YOUR WAY
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved25
scalingRules ([
scalingRule {
serviceStatistics {
metric "Total Requests Count"
statistics Statistics.maximumThroughput
movingTimeRangeInSeconds 20
}
highThreshold {
value 1
instancesIncrease 1
}
lowThreshold {
value 0.2
instancesDecrease 1
}
}
])

26. 5 MINUTES SETUP
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved26
/* Credentials - You must enter your
* cloud provider account credentials
*/
user="ENTER_USER_HERE"
apiKey="ENTER_API_KEY_HERE"
keyFile="ENTER_KEY_FILE_HERE"
keyPair="ENTER_KEY_PAIR_HERE"
// Advanced usage
hardwareId="m1.small"
locationId="us-east-1"
linuxImageId="us-east-1/ami-1624987f"
ubuntuImageId="us-east-1/ami-82fa58eb"

27.  Intro to Big data world & Cloud apps
 Challenges of big data applications
 Outages & disaster recovery
 Complexity of multi tiers application
 Lock-in to specific cloud
 Monitoring & Scaling
 PaaS make our life easier - Cloudify
® Copyright 2013 GigaSpaces Ltd. All Rights Reserved27
SUMMARY

28. THANK YOU!
Ron Zavner
RonZ@gigaspaces.com
® Copyright 2013 Gigaspaces Ltd. All Rights Reserved
Q&A