SERENE 2014 School on Engineering Resilient Cyber Physical Systems Talk: Measurement-Driven Resilience Design of Cloud-Based Cyber-Physical Systems, by Imre Kocsis

1. Departmentof Measurementand InformationSystems
Budapest University of Technologyand Economics, Hungary
Measurement-Driven Resilience Design of Cloud-Based Cyber-Physical Systems
Imre Kocsis
ikocsis@mit.bme.hu
SERENE’14 AutumnSchool
2014.10.14.

2. A Viewof Cyber-PhysicalSystems

3. Cyber-PhysicalSystems (CPSs)
3
Ubiquitousembeddedand networkedsystems thatcan monitor and control the physical world witha high level of intelligenceand dependabilityNetworkedembeddedsystemseverywhereClouds, „infusable” analytics, Big Data

4. FromembeddedtoCPS
4
Directmanualcontrol,
„closedworld” engineering

5. FromembeddedtoCPS
5
Directmanualcontrol,
„closedworld” engineering
Highlyautonomous,
„cyber” backend, environment, swarms, …

6. FromembeddedtoCPS
6
Directmanualcontrol,
„closedworld” engineering
Highlyautonomous,
„cyber” backend, environment, swarms, …

7. Cyber-PhysicalSystems
Differentflavors
oNSF, EU, academia, industry…
Still: itis here
oFromsmartcities& IoTtoself- drivingcars
oScalable, reconfigurablebackendis a must
7
Health CareTransportationEnergy

8. „Classical” caseforcloudcomputing: a brainforaCPS
Video
surveillance
Citizen
devices
Env. sensors
…
Trafficcontrol
Situationalawareness
Deep analyticsNormaldayDisaster
See: Naphadeet. al(IBM), „SmarterCitiesand TheirInnovationChallanges”, Computer, 2011Elastic, reconfigurablecomputing Reconfiguration

9. Converging domains
CPS
Cloud
computing
Big Data
9

10. Detour1: CloudComputing

11. Cloudcomputing: leasedresources
Source: http://cloud.dzone.com/articles/introduction-cloud-computing

12. Definition?
NIST 800-145
Cloudcomputingisamodelforenablingubiquitous, convenient,on-demandnetworkaccesstoashared
poolofconfigurablecomputingresources(e.g.,networks, servers,storage,applications,andservices)that
canberapidlyprovisionedandreleasedwithminimalmanagementeffortorserviceproviderinteraction.

13. Properties
On-demandself-service
Broadnetworkaccess
Resourcepooling
Rapid elasticity
Measuredservice
13

14. Ontheproviderside… ~?

15. Whyis itgoodfortheprovider?
(WithoutCLT)
푋푖independentprob. Varswith휇andσ2
Coefficientof variation: 휎 휇
Exp. valueof sum: sumof exp. values
Varianceof sum: sumof variances
CV푋푠푢푚= 푛휎2 푛휇 = 1 푛 휎 휇 = 1 푛 퐶푉(푋푖)

16. „Statisticalmultiplexing”
Variancew.r.t. meangetssmaller
1 푛 : quick–smallerprivateclouds
Realityis a bit different
Source: http://en.wikipedia.org/wiki/Central_limit_theorem

17. Gartner, 2013
„For larger businesseswith existing internal data centers, well-managed virtualized infrastructure and efficient IT operations teams, IaaSfor steady-state workloads is often no less expensive, and may be more expensive, than an internal private cloud.”
„I needitnow, and needitfast…”?

18. Parallellizableloads
More and more embarrassinglyparallel, „scale- out” applicationcategoriesexist
NYT TimesMachine: publicdomainarchive
oConversiontoweb-friendlyformat: ApacheHadoop, a fewhundredVMs, 36 hours
Inthecloud: coststhesameaswithoneVM
Practically: „speedupforfree”

19. Scalingresources
„Scaleup”
„Scaleout”
oAlgorithmics?
o„webscale”technologies

20. Detour2: Big Data

21. 1.) Big Data atRest
Distributedstorage
„Computationtodata”
„Atrest Big Data”
oNo update
oNo sampling
„Not true, but a very, very good lie!”
(T.Pratchett, Nightwatch)

22. MapReduce(ApacheHadoop)
Distributed File System[ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ , ] [ ,[ , , ]] [ ,[ , , ]] [ ,[ , , ]] [ ,[ , , ]] [ ,[ , , ]] SHUFFLE Map Reduce [ , ][ , ][ , ][ , ][ , ]

23. 2.) „Big Data inMotion”
Streamprocessing
Inherentlyscalablethesameway

24. Streamingdata
Sensordata
oFromsmartgridtoturbinetesting
Images
oSatellites: nTB/day
Web services
Network traffic
Trading
…

25. The streamprocessormodel
Source: Rajaraman, A., & Ullman, J. D. (2011). Mining of Massive Datasets. Cambridge: Cambridge University Press. p130

26. Design & composition
Source: International Technical Support Organization. IBM InfoSphereStreams: Harnessing Data in Motion. September 2010, p76

27. Whenwehavea WCET constraint…
Emphasisin„plain” Big Data: keepingstepwithingress
oButlargelythesamefordirecttimeliness
No (direct) diskaccess
Memory: bounded
Per-tupleprocessing: bounded
Algorithmicpatterns:
oPer-tupleprocessing
oSlidingwindowstorageand processing
oSpecializedsampling
•Getsuglyfast
oVariousheuristics

28. Applicationclasses
Source: International Technical Support Organization. IBM InfoSphereStreams: Harnessing Data in Motion. September 2010, p80

29. Takesoncyber-physicalclouds: Cloud-in-CPS…

30. Converging domains
CPS
Cloud
computing
Big Data
30
standard link
Intelligence Reconfigurability

31. CloudsinCPS –reality, notpromise
31
SENSORSACTUATORS

32. Architecturallandscape
32

33. Takesoncyber-physicalclouds: …CPS-in-cloud

34. ExtendingApacheVCL forCPS
34
Apache VCLVirtualized Data Center... VirtualmachinesInternet/CAN/LANRemote clientReservationEstablishing connectionRemote desktop or terminal access

35. Proofof Concept
35
Time-shareable arrangements Cloud-on-Cloud Apache VCLVCL management networkVCL public networkCloud instanceNetwork-attachedphys. devicesExperiment video stream

36. „CloudonCloud” capability
36
Apache VCLVCL management networkVCL public networkApache VCL/OpenStack/... CoC virtual networks

37. „Cloud on Cloud” capability
37
Apache VCL
VCL management network
VCL public network
Apache VCL/OpenStack/...
CoC virtual networks
Bootstrap &
capture XaaS
Hypervisors

38. „CloudonCloud” (CoC)
38
Withnestedvirtualization
Wehave… ovirtualesxi
oVCL over VCL onthat
Somerestrictionsapply;inVCL, no…
ostoragevirtualization
onetworkvirtualization
odynamicreservations

39. Integratinga fielddevice: RaspberryPi
39
Surprisinglypopular
oInthetargetdemographic
Almost a labPC:rpiVCL module
Linux
ogentlerlearningcurve
oInreservation: SSH access
Usefulsetof interfaces
ASM
C
scripting
Java
Wolfram

40. Integratingfielddevices?
Otherdevicetypes: adapter computer needed
oE.g. a RasberryPi foran Arduino
oScopes/spectrometers/…: alreadythere
oAutonomouscameras/meshGWs/…: alreadyinside
Lab.pm: starting point,needsrework
oFielddevices: „sanitization” is strongerconcept
oHarderwork-Pi: reset+ read-onlySD netboot
40

41. Container/VM
Container/VM
Future: fielddevicesastruecloudhosts
Real-time/embeddedvirtualizationis maturing
oCheckout: Siemens Jailhouse
oXenforARM
o…
Alsosee: carrierclouds
RaspberryPi alreadyhas containers!
41
Container/VM

42. Educationalprototype
42

43. Immediateapplications: cloudengineering
CoC:teachingvirt. & cloud
oE.g. weuseitforan ESXilab;
osupportforlocal VCL develinprogress
Real-life: faults, errors, failures
oCPS: performance!
Virtualizationintheloop
oThereareexistingSWIFI tools…
o… and VCL canbe a harness
43

44. Immediate applications: people & labs
44
Internet/CAN/LAN
Remote
client
We have EE/CE in view;
chemistry, biology,
physics, …?

45. Trustingyourcloudwithdeadlines-is ita goodidea?

46. Cloudsfor demanding applications?
Standard infrastructure
vs
demanding application?

47. Cloudsfor demanding applications?
Virtual Desktop
Infrastructure
Telecommunications
Extra-functional reqs:
throughput, timeliness, availability
„Small problems” have high impact
(soft real time)

48. Test automation
Hypervisor
Interference
Lab
OS and
hypervisor
metrics
LLO
HHII
Experimental setup

49. Short transient faults –long recovery
8 sec platform overload
30 sec service outage
120 sec SLA violation
As if you unplug your desktop for a second...

50. Deterministic (?!) run-time in the public cloud...
Variance tolerable by overcapacity
Performance outage intolerable by overcapacity

51. The noisy neighbour problem
Hypervisor
Tenant Neighbor

52. Tenant-side measurability and observability
Hypervisor
Tenant Neighbor

53. CharacterizingIaaSperformance

54. IaaSperformance
HW notnecessarilyknown
Unknown/ uncontrollabledeployment
Unknown/ uncontrollable
scheduling
„Noisyneighbors”
Also: management actionperformance?

55. IaaSperformance
Deploymentdecisions
oShouldI usethiscloud?
Capacityplanning
oTypeand amountof res.
Perf. prediction
oQoStobe expected
oAnd itsdeviances
Benchmarking!

56. Benchmarking (a pragmatictakeon)
(De-facto) standard applications
withwelldefinedexecutionmetrics
thatmayexercisespecificsubsystems
tocompareIT systemsviasaidmetrics.
Popularbenchmarks: e.g. PhoronixTest Suite
Benchmarking asa Service: cloudharmony.com

57. Whytraditionalbenchmarking is notenough
Stability
Homogeneity
Rareevents
Repeatability?
oProvider/tenant
Micro/componentbenchmarks?
oApplicationsensitivity?
oCloudfunctions(scaleinand out)?

58. TowardsMeasurement-DrivenResilienceDesign forClouds

59. A performance featuremodel
+ exp. behavior, homogeneity, stability
Li, Z., OBrien, L., Cai, R., & Zhang, H. (2012). Towards a Taxonomy of Performance Evaluation of Commercial Cloud Services.
In 2012 IEEE Fifth International Conference on Cloud Computing (pp. 344–351). IEEE. doi:10.1109/CLOUD.2012.74

60. ModelingIaaSperformanceexperiments
Li, Z., OBrien, L., Cai, R., & Zhang, H. (2012). Towards a Taxonomy of Performance Evaluation of Commercial Cloud Services.
In 2012 IEEE Fifth International Conference on Cloud Computing (pp. 344–351). IEEE. doi:10.1109/CLOUD.2012.74

61. „Cloudmetrology” and itsapplication
Full stack instrumentationFull adaptive data acquisitionFine-grained storageExploratory Data AnalysisConfirmatory Data Analysis Mystery shoppers and routine excercisesApplication sensitivity model(Platform) fault modelPerformance/capacity modelStructural defensesDynamic defenses MONITORING BENCHMARKING

62. Example:characterizingVDI „CPU ReadyTime”
„Ready”: VM readytorun, butnotscheduled
oVDI: „stutter”
Rareevents
oSampling
Needsfinegranularity!
+ atleasta fewmonths
Very„wide” data
Result: ~QoEcapacity+ load
Big Data tooling

63. EDA: hypothesesfrom„visualtours” of thedata
Cloudresponsetime~ nwdelay
clientID ~ loc
ClientlocationsDoesnotscaleforBig Data (yet)

64. Workflow? (As of now)
 Classical
tools
Slow EDA
On Big Data
Interactive EDA
On samples
statistics
on samples
Big Data
statistics
 Hadoop,
Storm,
Cassandra,
…

65. The effectof CPS cloudbackendinstability

66. Experimentalenvironment
Host1
Host2
Workstation
Workstation
OS_contr
OS_compute
nimbus
OS_network
CollectD
replay
superv2
superv1
Application

67. Applicationtopology
Redisspout
Gatherer1
Gatherer2
Aggregator
Timerspout
Sweeper
<ts, city, delay>
<city, delay>

68. Workload
Baselineworkload
Start of stress
End of stress

69. CPU utilization

70. Processlatency
Relationshipwithguestresourceusage?

71. Correlation: 0.890

72. Acknowledgements
Specialthanksgo fortheexperimentalenvironmentand datatoourOpenStackMeasurement„taskforce”:
Ágnes Salánki, Dávid Zilahi, Tamás Nádudvari, György Nádudvari, Gábor Kiss (BME) and
Gábor Urbanics(QuanoptLtd, ourspinoff)
72