Enterprise application development, testing, staging and ultimately pushing out to production involves number of operational activities. Moreover operational activities will also expand to the runtimes of each environment. Today the strength of a devops engineers is measured by the weight of his tool belt. That showcases his/her ability to manage the operational process and the ability to drill down the root causes and optimization possibilities at the runtime. Operational duties also expands to system maintenance, system health check and guaranteed availability. The ability to know the system behavior is the key to implement a proper operational process. During the talk Nuwan will be elaborating operational intelligence with regard to above processors and planning to discuss about the right level of information at each platform environment and runtime.

1. Operational Intelligence
Nuwan
Bandara
Solu%ons
Architect
-­‐
WSO2
@nuwanbando

2. Data, Information & Knowledge
h9p://spreadgoodprac%ce.blogspot.com/2010/12/model-­‐4-­‐data-­‐informa%on-­‐knowledge.html

3. Producers of data
movement
temp
power
institute
people
storage
servers
applications
POS
router camera
vibration

4. Produces of information

5. Knowledge
o Deriving meaning from information
o Co-relation
o Historical analysis
o Mathematical analysis
o Model driven analysis
o Graphs & charts based comparison

6. Wisdom
o Act upon knowledge
o Business decisions (when to invest / when to withdraw)
o When to compete
o Tactics and strategies
o Automated decision making
o Triggering alerts / email / SMS
o Device control (Temperature control / traffic control etc.)
o Failover decision making
o Elastic scaling decision making

7. So what is intelligence
o Collective correlated knowledge which enables
informed decision (wisdom) making
h9ps://dmaiph.wordpress.com/2015/01/28/%me-­‐consuming-­‐tasks-­‐your-­‐small-­‐business-­‐should-­‐outsource/

8. Looking back at data collection
o Collecting right (?) data is the key
o Collection method is important
o Storing the un-sorted, need a proper thought
process

9. Operational data
o The type of data that is core to the system of
operation
resource usage of the cloud
container useage
app usage
App Cloud Echosystem
raw material product manufature
material
inventory
demand
statistics
product
inventory
Manufacturing Plant

10. Conversion of data to information
o Data correlation
o Data graphs and trees
o Application of dimensions
o Summarization
o Data dictionary reference

11. When should we posses information
o At all stages of the operation
o i.e.: An application pushed through development,
testing, staging and production lifecycle
o At each stage the knowledge we gather is different,
hence information has a different shape at each step

12. An example
o Deploying an elastically scaling middleware
platform for application development &
integration
app serverbusapi gateway
elastic nodes /
containers
data nodes

13. The application
o Class / package dependencies &
associations
o Exception / stack trace logs
o Memory / load statistics
o Database connection statistics
o JMX channel data on garbage collection /
deadlocks / heap usage etc.
Package (p0)
Class (cp0)
Package (p1)
Class (cp1)
Package (p1)
Class (cp1)
data bridge /
event receiver
exception / error
logs
memory / load
stats
JMX
application

14. Application server & server container
o Server logs
o JMX channel data of the server
o Server health check
o Memory / processor / network usage
o Response times
o Container health
o Container load
o Container storage space stats
application (a1)
service (s2)service (s1)
server logs server JMX
container
stats
server response
time
container
app server
data bridge /
event receiver

15. Integration platform
o Mediation statistics
o Workflow statistics
o Data services query history / audit
o MQ statistics
o Container statistics
o Integration platform latency
o Request-response log
containers
c1 c2 c3 c4 cn
bps log
esb log
kafka log
data bridge /
event receiver

16. API platform
o Load balancer logs – container
routing / fail-over stats
o API subscriber statistics
o API usage statistics
o API lifecycle statistics
containers
c1 c2 c3 cn
load balancer
data bridge /
event receiver
api gateway
api store api publisher
API subscriber stats
LB logs
container stats
API usage
stats
API lifecycle
stats

17. Information factory
data bridge /
event receiver
realtime
near realtime /
batch
realtime info feed
summarized /
co-related feed

18. Knowledge factory
data bridge /
event receiver
realtime
near realtime /
batch
realtime
info feed
summarized /
co-related feed
Machine learning
lambda
loop back
loop back
intelegence
ML
feed back
ML
feed back

19. Decision enablers
data bridge /
event receiver
realtime
near realtime /
batch
realtime
info feed
summarized /
co-related feed
Machine learning
lambda
loop back
loop back
intelegence
ML
feed back
ML
feed back
integration
engine
APIs
dashboards
topics

20. Operational intelligence reference model
application (a1)
service (s2)service (s1)
server logs server JMX
container
stats
server response
time
app server
data bridge /
event receiver
containers
c1 c2 c3 c4 cn
bps log
esb log
kafka log
data bridge /
event receiver
containers
c1 c2 c3 cn
load balancer
api gateway
api store api publisher
API subscriber
stats
LB logs
container stats
API usage
stats
API lifecycle
stats
realtime
near realtime /
batch
realtime
info feed
summarized /
co-related feed
Machine learning
lambda
loop back
loop back
intelegence
ML
feed back
ML
feed back
integration
engine
APIs
dashboards
topics

21. Knowledge is power
"Technology is so much fun but we can drown in our technology. The
fog of information can drive out knowledge." -- Daniel J. Boorstin

22. //Questions ?
Thank
You