Enterprise Use Case Webinar - PaaS Metering and Monitoring

Enterprise Use Case – PaaS Metering &
Monitoring
Sinthuja Rajendran
Software Engineer
WSO2 Inc
Gihan Anuruddha
Senior Software Engineer
WSO2 Inc

About WSO2
•  Providing the only complete open source componentized cloud
platform
–  Dedicated to removing all the stumbling blocks to enterprise agility
–  Enabling you to focus on business logic and business value
•  Recognized by leading analyst firms as visionaries and leaders
–  Gartner cites WSO2 as visionaries in all 3 categories of application
infrastructure
–  Forrester places WSO2 in top 2 for API Management
•  Global corporation with offices in USA, UK & Sri Lanka
–  200+ employees and growing
•  Business model of selling comprehensive support &
maintenance for our products

150+ globally positioned support customers

IaaS, SaaS, PaaS
•  Software as a Service(SaaS): applications
that are available for use without having to
buy, deploy and manage.
•  Platform as a Service (PaaS): middleware
that offers a computing platform which
leverages an un-derlying IaaS and is the
basis for building SaaS applications.
•  Infrastructure as a Service (IaaS): low-level
computing resources such as CPU,
memory, storage and networking

What is PaaS?
•  It stands for platform-as-a-service. At a high-level, a PaaS helps
organizations, specifically developers, realize the benefits of cloud
computing by providing a fast and scalable way to host applications in the
cloud.
•  a PaaS is an Application Platform comprised of an operating system,
middleware and other software that allows applications to run on the cloud
with much of the management, security, scaling and other stack related
headaches abstracted away.
•  Along with software as a service (SaaS) and infrastructure as a service
(IaaS), it is a service model of cloud computing.
•  In this model, the user creates the software using tools and/or libraries
from the provider. The user also controls software deployment and
configuration settings. The provider provides the networks, servers,
storage, and other services.

PaaS Deployment Model
In Public
• PaaS available on the Internet
• Use one of the cloud service providers
• Information is stored and managed by provider under SLA
In Private
• Have a cloud, in-house
• IaaS provides by hardware on your data centers
• PaaS running on your IaaS
• SaaS executing on your PaaS
Or use both
• Hybrid Cloud

What is SLive - Stratos Live?
•  Publicly Hosted PaaS.
'WSO2 Stratos is a complete SOA and
developer platform offered as a self-
service, multi-tenant, elastic runtime for
private and public cloud infrastructures'

In PaaS ..
•  Throttling
q  When hosting a cloud, you have to be aware of the multi-tenant aspect
which can cause problems in consuming cloud resources.
q  A large number of users may using the cloud environment in different
ways.
q  At any circumstance If a user is consuming too much of your resources,
it can cause problems to your entire cloud environment.
q  We have to control the user's behaviour or you have to scale up your
resources to cater the situation and then charge the user for consuming
resources.

In PaaS.. (cont..)
•  Security Related Issues
q  Security and data privacy are major concerns of cloud consumers when
hosting their sensitive data, applications and performing critical
operations/transactions in the cloud.
q  On the other hand, it is a major challenge for cloud providers to meet
security, trust and privacy requirements of their clients.
q  Following can be identified as some of the main security challenges
among many others,
a.  Physical security
b.  Availability
c.  Data isolation and protection
d.  Execution isolation, logic isolation
e.  Malicious code
f.  Identity management

In PaaS.. (cont..)
•  QoS
q All users subscribed to the same usage plan has to be served
with same QoS
q Guaranteed QoS
q Throttle the excess usage of resource and maintain the QoS in
the public cloud is important

In PaaS.. (cont..)
•  Billing
q When hosting a public cloud, it is necessary to bill the users
for consuming the cloud resources.
q It is how you get to earn money for offering the cloud. It is
very important to bill the user for the correct usage, not
less, nor more.
q If your values are wrong, users will loose their trust on your
cloud deployment. On the other hand, users should be
billed without any losses to the hosting party. Billing system
has to be flexible too.

Need Metering & Monitoring...
Ø To cater the discussed important factors of
PaaS, we need to have the 'Metering and
Monitoring'.
•  Pay for the resource you have consumed - 'Pay
for just what you use'
•  Identify frequently used applications
•  Measure the application's resource utilization
•  Security
•  Monitor the traffic for the hosted application

What to Meter in PaaS?
•  Bandwidth Usage
o  Services, web application will be getting the requests, serve the
request and send back the response. Each of these operations are
entitled to billing and throttling.
•  Storage usage
o  Since user doesn't have the explicit direct access to the actual
infrastructure/Operating system in PaaS, there should be a way to
monitor the actual usage of the Storage such as RDBMS, Cassandra
used.
•  Registry Usage
o  Registry incoming data actions - put/ importResource/ restore
o  Registry outgoing data actions - dump/ get
•  User count for a Tenant

•  Usage Agent - The Usage Agent is responsible of measuring parameters and
publishing them to WSO2 BAM Where we store data and monitor them. For
each and every tenant, the agent measures some important parameters
related to their usage of bandwidth and resources (Service/webApp
bandwidth, Registry usage, User count).
•  Usage Summary Generator - Summery generator component is responsible
for summarizing captured data. After collecting raw data, we summarize
periodically and store those records in database. Those records are available
per hourly, monthly, daily, quarterly and annually basis.
•  Throttling Manager -Throttling manager periodically checks the usage
records (data captured by the usage agent mentioned above) available,
evaluates throttling rules and defines sets of actions for each and every
tenant. These actions contain what tenants can or cannot do and the error
messages shown to them.
•  Throttling Agent - Throttling agent is acting as a supervising agent which
checks the actions taken by users. Basically, it checks registry entry for a
given tenant (updated by throttling manager).

Usage Agent BAM Deployment architecture
Usage Agent BAM Receiver
Cassandra Cluster
SS Server
Hadoop Cluster
Hive
BAM Analytics Engine
MySQL Server

What to monitor in PaaS?
•  Logs
ü  Since there won't be any console access for the
tenants using the PaaS, there should be log viewer
which will give the indication about the current logs
in their application/system.
ü  All application/system logs need to be viewed.
ü  Isolation of logs within users/applications.

Distributed Logging with WSO2 BAM

Distributed Logging with WSO2 BAM (Cont...)
•  Log4j appender
•  BAM receiver
•  Stored in cassandra cluster
•  Hive job for summarize
•  HDFS to save summarize log

Distributed Logging with WSO2 BAM (Cont...)
• Asynchronous and Non-Blocking Data publishing
• Receives and Stores Log Events Cassandra Cluster which is
a highly scalable and Big Data Repository
• Can be shared with WSO2 CEP for real time Log Event
analysis
• High Performance

What to monitor in PaaS? (Cont..)
•  Service Statistics
q There may be services hosted in the PaaS, and the users
will be interested on the traffic they are getting on each
services they have hosted.
q The following may be some interested statistics figures they
are interested in.
q  Request/Response/Fault Count for a service in a
Month/Day/Hour
q  Average Response Time of the Service Hosted in a
Month/Day/Hour.

Service Statistics Monitoring With BAM in PaaS
MySqlSS server
Hive
Analyzer
Meta info
Job
submit Summarize
data
Hadoop Cluster
Cassandra
Cluster
BAM Receiver
Application Server
ClusterPublish Service statistics
events

Service Statistics Monitoring in SLive
1.  Enable Service Stats Monitoring

Service Statistics Monitoring in SLive (Cont..)
2. Create your Service Statistics Toolbox
•  Download the service statistics toolbox from http://
wso2.com/products/business-activity-monitor/toolboxes/
•  Extract the toolbox and edit the Hive script as replacing
your tenant credentials.

3. Create your Database in SS
•  Create Database

•  Create privilege template if not any created so far.

•  Create a user for Database

•  Attach the user to the created database with created
privileged template you wanted to grant

4. Create Datasource 'WSO2BAM_DATASOURCE' in WSO2
BAM for the database created in SS.

5. Install the modified toolbox

Now you can view your statistics. :)

• Proxy/Sequence/Endpoint Statistics
q There may be Proxy service hosted in the ESB, and the
users will be interested on the traffic they are getting on
each services they have hosted.
q The following may be some interested statistics figures they
are interested in.
q  Request/Response/Fault Count for a Proxy/
Sequence/Endpoint in a Month/Day/Hour
q  Average Response Time of the Proxy/Sequence/
Endpoint in a Month/Day/Hour.

Mediation Statistics Monitoring in PaaS with
BAM
MySqlSS server
Hive
Analyzer
Job
submit Summarize
data
Hadoop Cluster
Cassandra
Cluster
BAM Receiver
ESB Server Cluster
Publish Mediation
statistics events

Mediation Statistics Monitoring in SLive
1.  Enable Service Statistics Monitoring

Mediation Statistics Monitoring in SLive (Cont..)
2. Create your Mediation Statistics Toolbox
•  Download the mediation statistics toolbox from
http://wso2.com/products/business-activity-monitor/toolboxes/
•  Extract the toolbox and edit the Hive script as replacing your
tenant credentials.

3. Create your Database in SS
•  Create Database

3. Create your Database in SS...
•  Create privilege template if not any created so far.

•  Create user and add privilege for that user

•  Attach the user to the created database

4. Create Data Source 'WSO2BAM_DATASOURCE' in WSO2
BAM for the database created in SS.

5. Install the modified toolbox

Now you can view your statistics. :)

•  API Call Statistics/Usage
q In PaaS, tenant can create their own API and those APIs exposed to
the outside world to access.
q In this case, the API exposed by tenants need to be monitored and
the the following are some of the interested metrics.
q  Number of API Calls
q  Number of Subscriptions for API
q  Response time of APIs
q  APIs last access times
q  APIs access by version

•  Storage Usage/Capacity
q Since user doesn't have the explicit direct access
to the actual infrastructure/Operating system in
PaaS, there should be a way to monitor the actual
usage of the Storage such as RDBMS, Cassandra
used.
q With this monitoring the user will be able to control
the usage of the storage according the to usage
plan he/she has subscribed to.

•  Bandwidth Usage
q In PaaS users may have hosted the Services, Web
applications, etc.
q Services, web application will be getting the
requests, serve the request and send back the
response. Each of these operations are entitled to
billing and throttling.
q Therefore users should be able to monitor this and
view their current total, In/Out bandwidth.

Usage analysis with BAM
Usage Agent BAM Receiver
Hive
Hadoop
Analytics Engine
Cassandra Cluster
SS
Server
Usage
Reports/billing

Engage with WSO2
•  Helping you get the most out of your deployments
•  From project evaluation and inception to development
and going into production, WSO2 is your partner in
ensuring 100% project success

Enterprise Use Case Webinar - PaaS Metering and Monitoring

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Enterprise Use Case Webinar - PaaS Metering and Monitoring

Similar to Enterprise Use Case Webinar - PaaS Metering and Monitoring (20)

More from WSO2

More from WSO2 (20)

Recently uploaded

Recently uploaded (20)

Enterprise Use Case Webinar - PaaS Metering and Monitoring