4. 2013 Trend Micro
25th Anniversary
Architecture Overview
•Includes all the implementation of CloudStack's HTTP Query API
•Includes all supported Cloudstack Java Interfaces that can be
used to bridge against external systems.
Services
•Defines all the processes of CloudStack Management Server.
•Interacts with the Adapter Framework to provide customizations
to this process.
Management
•Contains the implementation that map CloudStack commands
to the various physical elements including network, storage, and
hypervisor resources
Resources
•Contains the implementation to the Database Access layer.
Currently it is MySQL but does not have to be limited to a
database.
Data Access
5. 2013 Trend Micro
25th Anniversary
Service Layer
• Synchronous command
– Scope of mgmt server or to the database
– Short-live
• Asynchronous command
– Require a call to an external system
– Take long time
• Command configuration
– commands.properties
– <command name>=<fully qualified java classname>;<ACL>
6. 2013 Trend Micro
25th Anniversary
Management Layer
• Kernel of CloudStack
• Manager
– Singleton to control a process
• Adapter
– Different ways to implement the same functionality.
– Often used when there can be multiple ways to implement
that step.
• Configuration
– components.xml
7. 2013 Trend Micro
25th Anniversary
Resource Layer
• Map commands to the physical elements
– Hypervisor resource
– Network resoure
– Storage resource
8. 2013 Trend Micro
25th Anniversary
Inside a Management Server
API
Servlet
Async
Job
Queue
Mgr
CS API
Services
API
Cmds
Responses
cmd.execute()
Kernel
Agent
Manager
Resources
Agent API
(Commands)
Hypervisor
Native
APIs
Local
Or
Remote
Network
Device
API
MySQL
9. 2013 Trend Micro
25th Anniversary
Accounts
Security
Manager
Events
Manager
Usage
Manager
Domain
Manager
Account
Manager
Limits
Manager
CloudStack Orchestration
VirtualMachine
Manager
StorageManager
NetworkManager
Template
Manager
Snapshot
Manager
CloudStack WebServices API
OAM&P API
End User
API
AWS API Pluggable Service API Engine
CloudStack Plugins
NetScalerNetwrokServiceProvider
VirtualRouterNetwrokServiceProvider
Business Logic
Resource
Manager
Rules
Manager
Update
Manager
HA
Manager
Capacity
Manager
Framework
Agent Manager Cluster Manager Data Access Layer
Adapters
Network Guru
Network
Element
Deployment
Planner
Hypervisor
Guru
10. 2013 Trend Micro
25th Anniversary
Plugins
• Various ways to add more capability to
CloudStack
• Implements clearly defined interfaces
• All operations must be idempotent
• All calls are at transaction boundaries
• Compiles only against the Plugin API module
• Deployed on management server
12. 2013 Trend Micro
25th Anniversary
Anatomy of a Plugin
• Server Component:
– Can implement multiple
Plugin APIs to affect its
feature
– Can expose its own API
through Pluggable Service
so administrators can
configure the plugin
• ServerResource:
– Deployed co-located with
the physical resource
ServerResource
- Optional. Required if Plugin needs to be co-
located with the resource
- Implements translation layer to talk to resource
- Communicates with server component via JSON
Rest API
PluginAPI
Data Access Layer
Implementation
13. 2013 Trend Micro
25th Anniversary
Components of an plug-in
• PluggableService that defines APIs to configure
the plug-in
• ServerResource extension that is meant to be run
co-located with the resource
• Manager code that runs within the management
server with access to management database
• Implement Adaptor interfaces to implement
functionalities required by CloudStack
• Change components.xml to make the plug-in and
adaptors active
14. 2013 Trend Micro
25th Anniversary
Adding a Plugin to CloudStack
• CloudStack assembles the components
according to the xml at startup time
• CloudStack assembles the list of APIs it
supports according to the PlugableService
specified in the properties file
17. 2013 Trend Micro
25th Anniversary
• CS does an excellent job for building clouds
• Growing needs for solutions to secure the Cloud
• Seek security solutions to work with CS
• Works for both private and public cloud
• Multi-tenant ,self-service,
• Easy to provision
• Resource effective
• Easy to integrate
• ElasterShield to combine the existing security
solution with CloudStack
Third party security solution integration
18. 2013 Trend Micro
25th Anniversary
Brief intro to ElasterShield
• Bridge between DeepSecurity and
CloudStack
• Hypervisor-based, agentless security
protection
• Features
• Firewall (L2, L3, L4)
• IDS/IPS
• Application Control
• Web server protection
• Anti-malware
• XenServer
ElasterShield
CloudStack API
DeepSecurity API
19. 2013 Trend Micro
25th Anniversary
Hypervisor
Hypervisor-based
• Agentless for user VM, need an
‘appliance’ on hypervisor
• Capture network traffics
• Scan VM memory
• Can work with any complex
network environment
User VM
User VM
Appliance
20. 2013 Trend Micro
25th Anniversary
Hypervisor-Based Solution
• Security Manager (DSM)
• Storing rules/profiles
• Provide UI/ web service API
• Send command to DSVA
DSM
Hypervisor
User VM
Hypervisor
User VMDSVA
DSVA• Security Virtual Appliance (DSVA)
• One for each hypervisor
• Hypervisor redirects VM’s traffic to
DSVA
• Includes an engine to do the
detection/prevention
• Bi-directional communicate
between DSM and DSVA
• No impact to user VM if DSVA stops
21. 2013 Trend Micro
25th Anniversary
Integrate as a CS Plug-in
• Security as a service, DeepSecurity as a service provider
– Multi-tenant, defined by admins, subscribed by users,
provisioned to VMs
• No modification on CloudStack kernel, keep it as it is
• Provide new security related APIs
• Treat DSM as a external device and send commands to it
• DSVA monitor and security management
• Tailor User VM lifecycle and Hypervisor management
with existing CloudStack framework
22. 2013 Trend Micro
25th Anniversary
Architecture Overview
CloudStack
Kernel
ElasterShield DSM
Hypervisor
DSVA
User VM
Hypervisor
DSVA
User VM
DS APINew CS API
Managed by CS
23. 2013 Trend Micro
25th Anniversary
Workflow: Add DSM
CloudStack Manager
ElasterShield Plug-in
DSM
2) Connect to DSM1) Call plug-in API
Database 3) Persist into DB
1) Admin user or external UI calls Plug-in API
• Parameters include DSM url/account/password
• One DSM per data center
2) Plug-in calls DSM’s APIs to:
• test connection with specified authentication
• Get DSM info such like version, license, etc.
3) Plug-in persists the DSM info into CS database
24. 2013 Trend Micro
25th Anniversary
Workflow: Enable Security protection
CloudStack
Manager
ES Plug-in DSM
3) Activate DSVA
1 ) Call plug-in API:
1) Admin user or external UI calls plug-in
API
2) ES Plug-in deploys DSVA
• Deploy DSVA on each host of the
cluster
• With specified service offering
and template
• Start DSVA
3) Activate DSVA
• Plug-in calls DSM API to activate
each DSVA
Hypervisor
2) Start DSVA on each
hypervisor of the cluster
Hypervisor
DSVA DSVA
25. 2013 Trend Micro
25th Anniversary
Workflow: Define Security Profiles
CloudStack Manager
ElasterShield Plug-in
DSM
2) Call DSM API
1) Call plug-in API
• For this Stage, this will be not implemented in plug-in, because it’s
product-specific
• Admins use DSM’s UI to define profiles
• Users use Plug-in API to list and apply profiles
Admin defines
Rules/profiles
26. 2013 Trend Micro
25th Anniversary
Workflow: Apply security offering
CloudStack Manager
ElasterShield Plug-in
DSM
3) Call DSM API,1) Call plug-in API
1) User or external UI calls plug-in API
• Specify VM id and security offering id
2) Plug-in persists <VM, offering> mapping into DB
• Address multi-tenant problem
3) Plug-in calls DSM API to assign corresponding profile if VM is running
Database 2) Persist into DB
27. 2013 Trend Micro
25th Anniversary
Workflow: Add a new host
CloudStack Manager
ElasterShield Plug-in
DSM
5) Call DSM API to activate
1) Call CS API
addHost
1) User or UI calls CS API
2) CS add a new hypervisor host as usual
3) Deploy DSVA
4) Plug-in persist DSVA info
Hypervisor
DSVA
2) Add host 3) Deploy DSVA
28. 2013 Trend Micro
25th Anniversary
Workflow: Maintain a host
CloudStack Manager
ElasterShield Plug-in
DSM
3) Call DSM API to de-activate DSVA
1) Call CS API
prepareHostForMaintenance
1) User or UI calls CS API
2) Stop DSVA on it
• update DSVA status into DB
3) Plug-in call DSM API to deactivate DSVA
• unassign security profiles to VMs that cannot be migrated
Hypervisor
DSVA
2) Power off DSVA
29. 2013 Trend Micro
25th Anniversary
Workflow: Start Virtual Machine
CloudStack Manager
ElasterShield Plug-in
DSM
3) Fetch VM info
1) Call CS API
startVirtualMachine
1) User or UI calls CS API
2) CS checks DSVA status and starts user VM as usual
3) DSM calls CS API to fetch VM ‘s information
4) DSM notifies DSVA in the same hypervisor with VM
Hypervisor
DSVA
User VM
2) Start User VM
4) DSM notifies ESVA
30. 2013 Trend Micro
25th Anniversary
Workflow: Stop Virtual Machine
CloudStack Manager
ElasterShield Plug-in
DSM
3) Fetch VM info
1) Call CS API
stopVirtualMachine
1) User or UI calls CS API
2) CS powers off user VM as usual
3) DSM calls CS API to fetch VM’s information
4) DSM notifies DSVA in the same hypervisor with VM
Hypervisor
DSVA
User VM
2) Stop user VM
4) DSM notifies DSVA
31. 2013 Trend Micro
25th Anniversary
Workflow: Destroy Virtual Machine
CloudStack Manager
ElasterShield Plug-in
1) Call CS API
destroyVirtualMachine
1) User or UI calls CS API
• VM is destroyed as usual by CS
2) When VM gets expunged, plug-in removes <VM, profile> mapping from
DB
• VirtualMachineGuru.finalizeExpunge
Database
2) Remove from DB
32. 2013 Trend Micro
25th Anniversary
Workflow: Query Events/Alerts
CloudStack Manager
ElasterShield Plug-in
1) Call plug-in API
1) User or external UI calls plug-in APIx
• With query parameters such as time, VM id, user id, etc
2) Plug-in calls DSM’s API to fetch events/alerts
2) Call DSM API
DSM
33. 2013 Trend Micro
25th Anniversary
Workflow: DSVA upgrading
CloudStack Manager
ElasterShield Plug-in
DSM
2) Call DSM API to upgrade
1) Call plug-in
API
1) Call plug-in API to upgrade DSVA , with a URL parameter
2) Plug-in forward the API to DSM
3) DSM notifies all DSVA to upgrade
4) DSVA downloads upgrading package from the URL specified
5) DSVA upgrade itself, it still functions during the upgrading.
Hypervisor
DSVA
3) Notify DSVA
Web server
4) Download upgrade package
