1. INFORMATICS INSTITUTE OF TECHNOLOGY
in collaboration with
the University of Westminster, UK
Department of Computing
BEng (Hons) Degree Programme in Software Engineering
Report On Industrial Placement
Of
Miss. Gowthamy Vaseekaran
[2015228 | w1608458]
in relation to the work carried out at
WSO2
20, Palm Grove,
Colombo-03.
Year of Submission
2018
2. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 1
Acknowledgement
The internship opportunity I had with WSO2 Lanka (PVT) Ltd was a great chance for
learning and professional development. Therefore, I consider myself as a very lucky
individual as I was provided with an opportunity to be a part of it. I am also grateful for having
a chance to meet wonderful people and professionals who led me though this internship
period.
I would like to take this opportunity to thank a few people, without whose assistance this
placement year would not have been successfully accomplished.
I express my deepest thanks to my industrial mentor, Dr. Miyuyru Dayarathna Senior
Technical Lead for taking part in useful decision and giving necessary advices and guidance
and arranged all facilities to make life easier.
It is my radiant sentiment to place on record my best regards, deepest sense of gratitude to
Analytics Team and Research team of WSO2. I am grateful especially to Mr. Sriskandarajah
Suhothayan [Director at WSO2], Mr. Malith Jayasinghe [Director at WSO2],
Mr. Mohanadarshan Vivekanandalingam [Technical Lead], Mr. Tishan Pubudu Kanishka
Dahanayakage [Associate Technical Lead], Mr. Anoukh Ashley Jayawardena [Software
Engineer] for their careful and precious guidance which were extremely valuable for my
study both theoretically and practically.
I perceive as this opportunity as a big milestone in my career development. I will strive to use
gained skills and knowledge in the best possible way, and I will continue to work on their
improvement, in order to attain desired career objective.
I offer my warmest thank you to my placement supervisor, Mr. Saman Hettiarachchi, whose
guidance and assistance I appreciate very much.
Another big thank you to Mr. Kaneeka Vidanage and Mrs. Aloka Fernando for all the support
they gave as the Placement Coordinators at IIT.
I also express my gratitude to IIT for providing us this one-year placement experience.
Finally, I thank my family for their continuous support in everything.
Thank you all once again for helping my efforts in making such a successful placement year!
3. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 2
Table of Contents
Acknowledgement........................................................................................................................ 1
Table of illustrations and abbreviations.......................................................................................... 3
Table of illustrations ............................................................................................................... 3
Abbreviations.......................................................................................................................... 3
1. Introduction ............................................................................................................................. 4
2. Company Profile ....................................................................................................................... 5
2.1 Details of the company................................................................................................ 5
2.2 Company Mission........................................................................................................6
2.3 History of WSO2..........................................................................................................6
2.4 Products of WSO2 ......................................................................................................7
2.5 Nature and Size of the business and Its markets....................................................... 8
2.6 Organizational and management Policy....................................................................9
2.6.1 Employee Hierarchy.......................................................................................... 9
2.6.2 Communication..................................................................................................9
2.7 Available career paths............................................................................................... 10
3. Work Carried out.................................................................................................................... 11
3.1 Project 1: Analyzing performance of WSO2 Stream Processor............................. 12
3.1.1 Description and purpose of the project ................................................................... 12
3.1.2 Tasks assigned........................................................................................................ 12
3.2 Project 2: Design and implementing Distributed View in the status dashboard . 14
3.2.1 Description and purpose of the project ................................................................... 14
3.2.2 Tasks assigned........................................................................................................ 16
3.3 Project 3: Design and implementing App View in the status dashboard ............. 21
3.3.1 Description and purpose of the project ................................................................... 21
3.3.2 Tasks assigned........................................................................................................ 21
3.4 Project 4: Feature Improvements and UI improvements of status dashboard .... 22
3.4.1 Description and purpose of the project ................................................................... 22
3.4.2 Tasks assigned........................................................................................................ 22
5. Reflections and Comments.................................................................................................. 23
6. Appendices .......................................................................................................................... 24
Status Dashboard Distributed View Screenshots................................................................ 24
Technologies Used .............................................................................................................. 26
Links related to my projects ................................................................................................. 26
4. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 3
Table of illustrations and abbreviations
Table of illustrations
Figure No Description Page No
Figure 1.1 Company Logo 05
Figure 1.2 Employee Hierarchy 09
Figure 1.3 Wso2 Stream Processor 11
Figure 1.4 Stream Query Processing Architecture 12
Figure 1.5 Event Ingestion with Persistence Architecture 13
Figure 1.6 WSO2 Status Dashboard 14
Figure 1.7 Distributed System Architecture 15
Figure 1.8 Dashboard Command Line 16
Figure 1.9 Node view thumbnail 17
Figure 1.10 Parent Siddhi Application Thumbnail 18
Figure 1.11 Parent Siddhi Application – code view 19
Figure 1.12 Distributed siddhi app deployment – Graphical view 20
Figure 1.13 Child App Details Table 20
Figure 1.14 App View 21
Figure 1.15 Graphical View for Siddhi Apps 22
Abbreviations
IIT - (the) Informatics Institute of Technology
BEng - Bachelor of Engineering
IP - Industrial Placement
HR - Human Resources
UK - (the) United Kingdom
CEO - Chief Executive Officer
CTO - Chief Technical Officer
SP - Stream Processor
HA - High Availability
WSO2 - Web Service Oxygen
SOAP - Simple Object Access Protocol
API - Application Program Interface
5. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 4
1. Introduction
Informatics Institute of Technology, in collaboration with the University of
Westminster, UK is the best place for higher education. If anyone is seeking for a
good degree related to IT, without doubt Informatics Institute of Technology (IIT) is
the best place to attend to. The uniqueness of IIT is offering one-year internship
during third year of the four-year degree programme. Industrial Placement is an
important component of the BEng(Hons) degree programme, which is referred to as
a “sandwich degree” once the IP is included.
For most of us this internship year is the first introduction to the world of work.
Internships are designed to expand breadth and depth of our academic learning. It’s
a big chance for us to earn experience in applying theory learned in the lecture hall.
Internship can also help us to motivate us to create new opportunities, expose new
ideas and overall it gives a path for the positive change. A successful placement will
help us to make a decision about the direction of the employment or future studies.
An internship is a chance to not only use and develop industry related skills and
knowledge, but also it helps us to expand our soft skills. Software industry develops
gradually. What we learn today may be getting outdated tomorrow or in a week or in
a month even in a few years later. Working with the real time industry helps us to
update ourselves with new technologies. By doing our work in an intelligent way will
help us to get a firm job opportunity at the end of our degree program. In my opinion
as a student we have to get maximum benefits from this opportunity.
So as I mentioned earlier, according to IIT’s degree programme procedure, in my
third year, I got selected to WSO2 for my one-year placement programme. It was
really a great experience. The assistance received from WSO2 is incredible in both
way professional level and comfort level. From the beginning of my internship, they
treated us as permanent employees rather than interns. So we could able to learn
many technical aspects and how they are working and surviving in this competitive
industry. In WSO2 mostly every interns will be assigned to individual feature or
project. So that whole feature or project belongs to us. We have to conduct project
meetings, design reviews, user-story reviews, code reviews and etc. It helped to
develop and showcase our technical and soft skills. They gave us complete freedom
to express our opinions. If there was any doubt when working, my colleagues helped
me a lot to overcome those issues. During my internship, three major Stream
Processor GA releases took place. They gave us opportunity to participate in the
release works. In this one year, WSO2 gave me a chance to understand how
industrial life would be and shaped me as a good programmer. More details about
the company and the work in which I was engaged during my placement time there
is to follow in the coming chapters of this report.
I started my internship on 3rd of July 2017 and completed it on 30th of June 2018.
6. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 5
2. Company Profile
2.1 Details of the company
Figure 1.1 – Companylogo
Name WSO2
Postal Address
Mountain View
787, Castro Street Mountain View,
CA 94041
New York
5, Pennsylvania Plaza Suite,
NY 10001
UK
1st Floor,
20-26, Brunswick P1, London
N1 6DZ
Brazil
Rua Tenerife Andar- Vila Olimpia
Sao Paulo – SP – Brazil
Srilanka
20, Palm Grove,
Colombo-03
Australia
16/F & 17/F,
9 Castlereagh Street,
Sydney, NSW 2000`
Web Address www://wso2.com/
Email Address contact@wso2.com
Headquarters United States
CEO Tyler Jewell
Founders Sanjiva Weerawarana
Paul Fremantle
Founded 2005
7. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 6
2.2 Company Mission
“Proven Fortune 500 Partner. Leading enterprises rely on WSO2 for their mission-
critical applications. Today, our customers span across healthcare, finance, retail,
manufacturing, telecoms and other industries worldwide.”
2.3 History of WSO2
WSO2 was founded by Sanjiva Weerawarana and Paul Fremantle in August, 2005,
backed Intel Capital, Toba Capital, Pacific Controls and others.
WSO2 products are released under the Apache License Version 2. Like the Apache
project itself, WSO2 follows open development principles and airs architecture and
development discussions public mailing lists. The company is noted by Gartner as
being one of the leading competitors in the application infrastructure market.
WSO2 was founded by Sanjiva Weerawarana, an IBM researcher who was one of
the founders of the Web services platform. Weerawarana led the creation of IBM
SOAP4J, which later became Apache SOAP, and went to architect other notable
projects, including Apache Axis, Apache WSIF, the IBM Web Services Gateway and
IBM BPWS4J. He was joined by Paul Fremantle, one of the authors of IBM's Web
Services Invocation Framework and the Web Services Gateway. Fremantle, an
Apache member since the original Apache SOAP project, oversaw the donation of
WSIF and WSDL4J to Apache and led IBM's involvement in the Axis C/C++ project.
He became WSO2's CTO in 2008. Fremantle was subsequently named one of
Infoworld's Top 25 CTOs in 2008. In 2017, Tyler Jewell took over as CEO.
As of 2018, WSO2 has offices in Mountain View, CA, New York City, London, São
Paulo, Sydney and Sri Lanka (Colombo and Jaffna), with the bulk of its research and
operations being conducted from its main office in Colombo. Plans are in place to
open offices in Mexico City and Europe by 2018. In 2011 it was reported
in Information Week that the auction website eBay revealed a key element of their
transaction processing software used the open source WSO2 ESB. Following this
association with eBay the company rose to prominence.
WSO2's first product was code-named Tungsten, and was meant for the
development of web applications. Tungsten was followed by WSO2 Titanium, which
would later become WSO2 Enterprise Service Bus. In 2006, Intel Capital invested $4
million in WSO2 (Intel would continue to invest over the years). In 2010, Godel
Technologies also invested in WSO2 for an unspecified amount, and in 2012 they
raised a third round of $10 million. Official WSO2 records point to this being from
Toba Capital, Cisco and Intel Capital. In August, 2015, a funding round led by Pacific
Controls and Toba raised another $20 million.
A subsidiary, WSO2Mobile, was launched in 2013, with Harsha Purasinghe of
Microimage as the CEO and co-founder. In March 2015, WSO2.Telco was launched
in partnership with Malaysian telecommunications company Axiata with Axiata
holding a majority stake in the venture. WSO2Mobile has since been re-absorbed
into its parent company.
Historically, WSO2 has had a close connection to the Apache community, with a
significant portion of their products based on or contributing to the Apache product
stack. Likewise, many of WSO2's top leadership have contributed to Apache
projects. In 2013, WSO2 donated its Stratos project to Apache.
8. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 7
2.4 Products of WSO2
WSO2 offers a platform of middleware products for agile integration, API
management, identity and access management and smart analytics.
In June 2015, WSO2 announced software as service versions of its open-source
products.
WSO2 Enterprise Integrator
It is an open source, hybrid integration platform which allows developers to integrate
applications, data or systems
WSO2 API Manager
It is an API management program that allows enterprises to create, publish and
manage APIs. It supports API lifecycle management, application development,
access control, rate limiting and analytics in one system.
WSO2 Identity Server
It provides identity management capabilities.
WSO2 Stream Processor
It is a Streaming SQL Engine that supports Real-time Streaming Analytics.
Ballerina
It is a general purpose, concurrent and strongly typed programming language with
both textual and graphical syntaxes, optimized for integration.
9. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 8
2.5 Nature and Size of the business and Its markets
Today, there is a greater focus on enterprise IT because this facilitates change and
aids with adapting to change. A key challenge faced by most companies today is the
lack of resources to keep pace with the required changes in enterprise development,
and WSO2 through its open source middleware has been able to meet some of these
challenges, adds Dr. Sanjiva Weerawarana.
WSO2's main advantage is that all of its products are built on a common foundation-
WSO2 Carbon, a modular, reconfigurable, elastic, OSGi-based architecture which in
turn ensures a solid base for building large scale enterprise applications, as well as
seamlessly integrating with legacy and existing applications.
WSO2 also hosts a variety of events such as webinars, workshops, conferences and
meet-ups for knowledge sharing and networking around the various open source and
other business communities.
The company's clientele is inclusive of renowned business enterprises such as:
eBay
Cisco Systems US
British Airways
Dialog Axiata
Boeing company
Fidelty Investments
Dubai eGovernment
John Keells Holdings
and etc
10. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 9
2.6 Organizational and management Policy
WSO2 is maintaining a very flat and informal structure. In the company every one
considered as equal no matter what the position, how old. This system creates team
spirit and anyone can directly speak with anyone.
2.6.1 Employee Hierarchy
This employee hierarchy is applying in HR management. But in decision making
even an intern can take part if considerable fact have in hand. This is very helpful in
fast decision making at the agile software development process because employees
will get the opportunity of applying perfect solution in right time.
Figure 1.2 – Employee Hierarchy
2.6.2 Communication
Communication inside WSO2 is so transparent that everything goes in the mailing
list of the company domain and this is also very fast in fixing anything. This
transparency allows maximum productivity as and confusions are so lowered.
Following are few mailing lists among them.
Team: Anything regarding the whole WSO2 team goes here. Eg: Organizing
trips, Seating Plans.
11. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 10
Support-dev: This is focusing on support for the developers.
Training: To discuss things related to training inside WSO2 and outside things
that employees can participate
Marketing: Discuss matters related to marketing strategy etc. Anyone can post
their own ideas here on how to promote WSO2 products.
Operations: Any issue regarding daily operations of the company goes here. Eg:
Cleaning.
Infrastructure: Any matters regarding network.
Vacation: Any kind of leave taken should be informed here.
News: Any news regarding the industry that seems useful for the company are
posted here.
Club: Jokes and other stuff goes here for fun mostly
Also there is no restriction to talk to anyone and even we could easily approach to
Dr. Sanjiva Weerawarna and discuss any issue we had. All the doors are open for
folks to directly communicate and there was no need to go in a hierarchy.
WSO2’s Secret of being a successful company is the company consists of best
people for each field. It has experts in any field.
2.7 Available career paths
In WSO2, there are many career paths available for talented undergraduates. Here I
have listed some of them related to IT sector.
Software Engineer
Software Engineer – Test Automation
Senior Software Engineer
Senior Software Engineer – Test Automation
Solution Engineer
Solution Architect
Associate Technical Lead
Technical Lead
In WSO2 I worked as a software engineering intern in Analytics Team and Research
Team.
12. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 11
3. Work Carried out
During my internship, I worked with WSO2 Stream Processor for my two projects. So
first I’ll give a brief introduction to WSO2 Stream Processor.
WSO2 Stream Processor is a Streaming SQL based, high performant, lightweight,
open source stream processing platform, facilitating the creation of real-time,
intelligent, actionable business insights, and data products for digital businesses. It
allows you to collects events, analyzes them in real-time, identify patterns, map their
impacts, and react within milliseconds.
Figure 1.3 – WSO2 Stream Processor
The section demonstrates how to use some of the important functionalities of WSO2
SP:
Understanding the Development Environment: Provides the developer
relevant knowledge on how to use the Stream Processor Editor and its
functionalities.
Developing Streaming Applications: Provides information on how to write
Stream Processing Applications using Siddhi Streaming SQL.
Deploying Streaming Applications: Provides information on how a user can
deploy a Stream Processing application in a production environment.
Visualizing Data: Explains how to create visualizations using dashboards
and widgets.
13. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 12
Managing Business Rules: Demonstrates how developers can define
templates, and how business users can create rules from those templates and
manage them.
Monitoring Stream Processor: Explains how to configure the Status
Dashboard to monitor Stream Processor nodes.
3.1 Project 1: Analyzing performance of WSO2 Stream Processor.
3.1.1 Description and purpose of the project
WSO2 Stream Processor has fairly good high event processing capacity, it helps us
to understand impacts of events and map, identify patterns and react within
milliseconds.
My first project is to analyze the performance of WSO2 Stream Processor. We have
divided the analysis into two phases. They are:
Stream Query Processing
Event Ingestion with Persistence
3.1.2 Tasks assigned
Stream Query Processing
Figure 1.4 – Stream Query Processing Architecture
Infrastructure Used
The experiments were carried out in two c4.2xlarge (8 vCPU, 16GB RAM, EBS
storage with 1000 Mbps max dedicated bandwidth) Amazon EC2 instances.
Linux kernel 4.44, java version "1.8.0_131", JVM flags -Xmx4g -Xms2g
One node operated as a client.
Another node operated as a Stream Processor node.
Experiments were carried out using TCP as the transport.
14. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 13
Using the above infra structure, I tested for multiple siddhi apps scenarios and
analyzed their performance such as
Simple Pass-through
Filter (Multiple filter scenarios)
Windows (Small, Large)
Patterns (using debs-2013-grand-challenge-soccer-monitoring dataset.
As a results of this experiments reported the good performance numbers and
pointed out performance bottle necks of wso2 stream processor.
EventIngestion with Persistence
Figure 1.5 – Event Ingestion with Persistence Architecture
Infrastructure Used
c4.2xlarge (8 vCPU, 16GB RAM, EBS storage with 1000 Mbps max dedicated
bandwidth) Amazon EC2 instances operated as the SP node and the TCP
client.
Linux kernel 4.44, java version "1.8.0_131", JVM flags: -Xmx4g -Xms2g
db.m4.2xlarge (8 vCPU, 32 GB RAM, EBS-optimized storage with 100 Mbps
max dedicated bandwidth) Amazon RDS instance with Oracle operated as the
database node.
db.m4.2xlarge (8 vCPU, 32 GB RAM, EBS-optimized storage with 100 Mbps
max dedicated bandwidth) Amazon RDS instance with MS SQL Enterprise
Edition 2016 operated as the database node.
db.m4.2xlarge (8 vCPU, 32 GB RAM, EBS-optimized storage with 100 Mbps
max dedicated bandwidth) Amazon RDS instance with MySQL Community
Edition version 5.7 operated as the database node.
Customized TCP client operated as the data publisher.
Using above infrastructure, I tested Event ingestions with different data sources
1. Oracle: Oracle 12g
2. MS SQL: MS SQL Enterprise
3. MYSQL: MySQL Community
As a results of this experiments reported the good performance numbers for some
data base operations such as Insert query, Update Query and pointed out
performance bottle necks of wso2 stream processor.
15. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 14
3.2 Project 2: Design and implementing Distributed View in the status
dashboard
3.2.1 Description and purpose of the project
What is WSO2 Status Dashboard?
The Status Dashboard of WSO2 Stream Processor allows user to monitor the
metrics of a stand-alone WSO2 Stream Processor instance or a WSO2 Stream
Processor cluster. This involves monitoring whether all processes of the setup are
working in a healthy manner, monitoring the current status of a Stream Processor
node, and viewing metrics relating to the history of a node or the cluster. Both JVM
level metrics or Siddhi application level metrics can be viewed from the
monitoring dashboard.
We can understand the performance of the system via
Throughput
Latency
CPU, memory utilizations
We can Monitor in various scales
Node Level
Siddhi App Level
Siddhi Query Level
Figure 1.6 - WSO2 Status Dashboard
16. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 15
What is Distributed Set Up?
Figure 1.7 – Distributed DeploymentArchitecture
The most common deployment pattern for WSO2 Stream Processor is the HA
Deployment that offers high availability and high performance with the minimum
amount of resources. However, there are some real time scenarios where the HA
deployment is not sufficient to handle the scenarios. The Fully Distributed
Deployment pattern is introduced to handle to such scenarios. The main benefits to
this approach are high availability and fault-tolerance.
In simple words, a distributed system is that splits into multiple running instances, all
instances work together in a cluster to appear as a single instance to the end user.
Distributed deployment pattern is suitable in such scenarios where single SP
instance fails to manage high amount of data as it allows complex event processing
in a distributed manner. eg: High volume of data will be distributed among multiple
SP instances instead of having them accumulated at a single instance.
Monitoring Distributed System in WSO2 Status Dashboard
Prerequisites:
Fully distributed system must be available.
Download the latest WSO2 Stream Processor
17. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 16
3.2.2 Tasks assigned
How to start WSO2 Status Dashboard?
1. Extract the downloaded zip and navigate to the <SP_HOME>/bin directory
(<SP_HOME> is the extracted directory).
2. Issue the following commands to start the WSO2 Status Dashboard.
For Windows: dashboard.bat
For Linux: ./dashboard.sh
Once WSO2 dashboard is successfully started, a log similar to the following is
printed in the CLI.
Figure 1.8 - Dashboard Command Line
The server log prints the Status Dashboard URL in the startup logs as shown below.
Access the WSO2 Status Dashboard via the
https://<HOST_NAME>:<STATUS_DASHBOARD_PORT>monitoring status
dashboard URL.
The default URL is: https://192.168.1.2:9643/monitoring
The Stream Processor Studio opens as shown below.
You can login to status dashboard by giving ‘admin’ as a user name and
password
Once you login to the status dashboard, the nodes that are already added to
the Status Dashboard are displayed
If no nodes are displayed, you can add nodes for which you want to view the
status by following the procedure below:
Click ‘Add New Node’ button, this opens the dialog box.
Enter the following information in the dialog box and click ADD NODE to add
a gadget for the required node in the Node Overview page.
1. In the Host parameter, enter the HTTPS host ID of the node you want to add.
2. In the Port parameter, enter the HTTPS port number of the node you want to
add.
18. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 17
If the node you added is reachable, automatically status dashboard determines the
runtime environment and will proceed further or else the following dialog box will be
opened.
Node View
The following basic details are displayed for each node.
Figure 1.9 – Node View Thumbnail
CPU Usage: The CPU resources consumed by the SP node out of the
available CPU resources in the machine in which it is deployed is expressed
as a percentage.
Memory Usage: The memory consumed by the node as a percentage of the
total memory available in the system.
Load Average: The average system load calculated over a given period of
time of 1, 5 and 15 minutes
Siddhi Apps: The total number of Siddhi applications deployed in the node.
* Green color indicates active number of siddhi applications
* Red color indicates inactive number of siddhi applications
This dispalys the change that has taken taken place in the CPU usage, memory
usage and the load average of nodes since the status was last viewed in the status
dashboard.
Positive changes are indicated in green (e.g., a decrease in the CPU usage in the
above example), and Negative changes are indicated in red (an increase in the
memory usage and the load average in the above example).
19. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 18
Viewing status details
Description
The nodes that are connected in the distributed deployment are displayed under the relevant
group ID in the status dashboard (e.g : ‘sp’ in the above example).
Managers
The active manager node in the cluster is indicated by a green dot that is displayed
with the host name and the port of the node. Similarly, a grey dot is displayed for
passive manager nodes in the cluster.
Workers
When user adds the active manager node, automatically it retrieves the worker node
details that are connected with that particular deployment and you can see the
metrics of each worker nodes
Purpose
This allows you to determine the following:
Whether the request load is efficiently allocated between the nodes of a
cluster.
Whether the cluster has sufficient resources to handle the load of requests.
What are the nodes connected with the particular deployment?
Viewing Parent siddhi application details
When you open the WSO2 SP Status Dashboard, the Node View page is displayed
by default. To view information specific to a selected acive manager node, click on
the relevant gadget. Passive manager node gadgets are disabled.
This opens a separate page for the manager node.
Description
Figure 1.10 – Parent Siddhi Application Thumbnail
The Active/Deployed parent siddhi application deployed in the cluster is indicated
by a green dot that is displayed with the name of the parent siddhi application.
20. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 19
Similarly, an orange dot is displayed for Inactive/ Un-deployed siddhi applications
in the cluster.
Worker Nodes : This gives the details of worker nodes that are inside
resource cluster.
Yellow Color indicates total number of worker nodes in the resource cluster
(Eg: 2 in the above example which means for that particular distributed
deployment we have two worker nodes in the resource cluster).
Green Color indicates the number of worker nodes used in the particular
parent siddhi application deployment. (Eg : 1 in the above example which
means even though we have two worker nodes in the resource cluster, only
one work node used for this specific parent siddhi application’s deployment).
Child Apps : This gives the total number of child apps of the particular parent
siddhi application
Green Color indicates number of deployed child apps
Red Color indicates number of un-deployed child apps
Groups: This gives the total number of execution groups of that parent siddhi
application. (Eg : In the above example, Testing Parent siddhi application
contains 1 execution groups)
PurposeThis allows you to determine the following:
Summary details of each parent siddhi applications that are inside the specific
active manager node.
Viewing Specific Parent Siddhi Application
To view information specific to a Parent Siddhi application deployed in the active
manager node, click on the relevant parent Siddhi application gadget.
This opens a page with information specific to the selected Parent Siddhi application
as shown in the example below.
Code View
View
Figure 1.11 – Parent siddhi application – code view
21. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 20
Description
This displays the queries defined in the Parent Siddhi file of the application.
This allows you to check the queries of the Siddhi application if any further
investigations are needed based on the kafka diagrams and performance.
Distributed Siddhi App Deployment
View
Figure 1.12 – Distributed siddhi app deployment – Graphical view
DescriptionThis displays how kafka topics connected with each child siddhi applications.
Red color indicator box indicates the kafka topic name.
Blue color indicator box indicates the child siddhi applications
Child App Details
View
Figure 1.13 – Child App Details Table
Description
This table displays the complete list of Child Siddhi applications in the selected
parent siddhi application. The status is displayed in green for active Siddhi
applications, and in red for inactive Siddhi applications. In addition, the following is
displayed for each Siddhi application:
Group Name: The name of the group that particular child siddhi application
belongs to
Worker Node: HTTPS host and HTTPS port of the worker node where that
particular child siddhi application is deployed. By clicking that host and the
port, you can get the worker specific details. This blog mainly focus on the
manager view.
22. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 21
3.3 Project 3: Design and implementing App View in the status
dashboard
3.3.1 Description and purpose of the project
In the previous version of status dashboard, user can only see the summary details
of registered node level information. So as an improvement I was assigned to
implement app view to indicate the summary details of deployed siddhi applications
according to their deployment mode such as
Single node deployment siddhi applications
HA deployment siddhi applications
Distributed siddhi applications
3.3.2 Tasks assigned
When you open the WSO2 SP Status Dashboard, the Node Overview page is displayed by
default. If you want to view all the Siddhi applications deployed in your WSO2 SP setup, click
on the App View tab (marked in the image below). The App Overview tab opens and all the
Siddhi applications that are currently deployed are displayed as shown in the image below.
Figure 1.14 – App View
The status is displayed in green for active Siddhi applications, and in red for inactive
Siddhi applications.
If no Siddhi applications are deployed in your WSO2 SP setup, the following
message is displayed.
23. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 22
The Siddhi applications are listed under the deployment mode in which they are
deployed (i.e., Single Node Deployment, HA Deployment, and Distributed
Deployment).
3.4 Project 4: Feature Improvements and UI improvements of status
dashboard
3.4.1 Description and purpose of the project
As a final task I was assigned to do some small feature improvements and UI level
improvements in the status dashboard. Main feature improvement is Displaying
diagrammatical view for the deployed siddhi applications.
3.4.2 Tasks assigned
Figure 1.15 – Graphical view for siddhi apps
This allows you to check the flow of the queries of the Siddhi application in the
graphical way. I used dagre-d3 charting library in order to draw a graphical view.
Along with this improvement I have done some UI level improvements to increase
the user friendliness.
24. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 23
5. Reflections and Comments
The IP year spent at WSO2 helped me gain an extensive amount of experience and
knowledge through the different challenging tasks which were assigned to me.
Having completed the first two years of study at IIT backed me up at IP, which was
my first industrial exposure. The experience I had got through modules, practical
sections and the course-works covered at IIT were a huge support to me to settle
down comfortably in the work environment. The personal skills such as team
working, presentation skills, communication skills, time management and leadership
skills which were always encouraged at IIT also became helpful at the workplace and
they improved more during the work period.
I highly recommend WSO2 as a great training establishment that gives a complete
exposure to the global software industry in a modern company background. It has
various technologies involved and we could really see the concepts we learnt, at
practice. As WSO2 encourages employee’s personal improvement with training
sessions there was lot of technical knowledge that has enormous value which flows
into our brains.
There were no restrictions on interns using the resources and questioning on
anything. We had the opportunity to learn things from the world-class industry
experts. The company was so enthusiastic on training us that lot of opportunities
were open for us to learn. My industrial mentor Dr.Miyuru Dayarathna never hesitate
to guide and help me in any issue I had and clearly explained anything I asked.
Whole Analytics team and Research team members did not hesitate to help me
whenever I need helps. With the opportunities they gave us to demonstrate our work
we done during the IP, they gave us recognition for what we did and gave good
feedback encouraging us and also guiding us to be better.
Apart from the projects, we had lots of fun there such as Annual Trip, Team trips,
Team outings, New year celebration, Year-end party, carom table, play station, Fuse
ball, and etc.
Altogether, the IP helped me to realize the importance of achieving work experience
while following a degree. It covers the essential practical segment which we tend to
ignore when receiving education most of the time. Therefore, I recommend an IP
year to all the educational students at some stage within their higher studies.
25. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 24
6. Appendices
Status Dashboard Distributed View Screenshots
Status Dashboard Login Page
Landing Page (If there is no node registered in the status dashboard)
Add New Node Page
26. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 25
Distributed Cluster View
Summary Details of deployed parent siddhi applications
Details of each parent siddhi applications
27. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 26
Technologies Used
Java
Distributed Deployment
REST Api
Micro Services
WebSocket
Matrix
React JS
Material UI
JMX
Swagger
MSF4J
JavaScript
VIZ Grammar
CSS
Dagre-D3
Findbugs
Checkstyle
H2 Database
Oracle 11g Database
MySQL
MSSQL
Links related to my projects
This section contains source code of my implementation, official documentation of
my projects and medium articles written by me related to my projects.
Analyzing the performance of WSO2 Stream Processor
https://docs.wso2.com/display/SP420/Performance+Analysis+Results
https://github.com/wso2/product-sp/tree/master/modules/performance
https://medium.com/@gowthamy/wso2-stream-processor-performance-
analysis-internship-project-01-c29a93d58865
https://medium.com/@gowthamy/big-data-battle-batch-processing-vs-stream-
processing-5d94600d8103
28. Industrial Placement Report
GowthamyVaseekaran | 2015228 | w1608458 Page | 27
Design and Implementing Distributed View in the status dashboard.
https://github.com/wso2/carbon-
analytics/tree/master/components/org.wso2.carbon.sp.jobmanager.core
https://github.com/wso2/carbon-
analytics/tree/master/components/org.wso2.carbon.status.dashboard.core
https://github.com/wso2/carbon-
analytics/tree/master/components/org.wso2.carbon.status.dashboard.web
https://github.com/wso2/carbon-
analytics/tree/master/components/org.wso2.carbon.stream.processor.core
https://docs.wso2.com/display/SP420/Viewing+Statistics
https://medium.com/@gowthamy/distributed-view-in-wso2-status-dashboard-
ef2df1472009
Design and implementing App View in the status dashboard
https://docs.wso2.com/display/SP420/App+Overview
https://github.com/wso2/carbon-
analytics/tree/master/components/org.wso2.carbon.status.dashboard.web
https://github.com/wso2/carbon-
analytics/tree/master/components/org.wso2.carbon.status.dashboard.core