The customer was designing their services in creating digital
strategies, laying out the road map and helping organizations by
working toward their digital transformation. As an essential
component of the plan, they wanted a thorough DevOps
implementation with complete automation of their processes and a rigorous Continuous Integration and Continuous Delivery (CI/CD) system in place.
2. THE CUSTOMER
THE CHALLENGE
The customer offers assessment tools and other educational
content in all available media to cater to their diverse audience.
Earlier, they were adopting the traditional method of product
development that often resulted in process delays and
performance issues during the deployment part.
The customer is one of the world’s leading education companies
that develop academic and research content over a wide spectrum
of subjects including business, technology, science, law and
humanities. They are committed to provide carefully-crafted
learning tools and they are also one of the foremost publishers in
the school segment (from primary to senior secondary) with book
lists in English Language Teaching (ELT), sciences, mathematics etc.
3. The existing testing method allows FitNesse execution (an
automation testing application that allows any user to enter
formatted input and creates tests for the system to execute)
to follow a sequential manner of execution. This is an
inefficient process that causes time delays when executing all
the triggered jobs.
Test reports couldn’t be generated by command line
execution which forced the test analysis to be done in a time-
consuming manner in the main FitNesse server.
As the customer was maintaining a lot of servers across the
globe, they wanted an active log monitoring routine on
system-related activities.
In short, the customer required a fully matured DevOps
implementation that would enhance their overall application
delivery.
Aspire’s team
proposed setting up
a build flow plugin
to take care of the
order of jobs
execution.
THE SOLUTION
In order to optimize the test execution time of multiple tests
in FitNesse server, Aspire’s team proposed setting up a build
flow plugin to take care of the order of jobs execution.
This order in the plugin is done as per the requirement in a
specific order by simply ignoring the results of the executed
jobs.
The plugin is also designed to generate test reports
simultaneously as the jobs get executed and the test report
page was also made available in Jenkins fitnesse results page.
FitNesse Jobs Execution
Aspire’s team offered to implement DevOps into the customer’s
project delivery system by executing three major implementations
in their environment.
TECHNICAL
CHALLENGES
4. This method replaces the need for 5 FitNesse servers with
just one, thereby increasing the time and cost efficiency.
Reusable test scripts are used and a new FitNesse server can
be setup instantly by launching multiple instances on the
same server.
Key Features
For continuous code inspection, Aspire’s team utilized
SonarQube managed inside containers by Docker and AWS
ECS (EC2 Container Service) and ECR (EC2 Container
Registry) and the rest of the activities were automated by
Ansible.
The configuration of SonarQube, with MySQL as the
database, is done by Ansible by triggering the installation of
ECS and fetching the respective images from ECR. This means
that the customer can have multiple instances of SonarQube
instantaneously with several versions in it.
Sonar dashboards would allow viewing of codes of the
project that were configured with Sonar.
Sonar and ECS Management
Ansible
EC2 Container Service EC2 Container RegistryDocker Images
Virtual Machines
5. In order to manage multiple servers across geographies,
Aspire’s team proposed the implementation of an ELK stack
(consisting of Elasticsearch, Logstash and Kibana), with an
additional front end interface of Grafana, to acquire real-time
instant reports with different types of graphs and time-
stamps.
The ELK stack provides a SAR log monitoring system, with
feature to search all the logs at one place, which will come in
handy to identify problems and correlating logs with the
servers or applications.
Kibana will be useful for administration-related activities
while Elasticsearch will restore the logs from Logstash and
directs it to processing in Grafana.
SAR Log Monitoring
Scalability can be achieved at ease with the help of
Elasticsearch as it is built to scale horizontally out-of-the-box.
When the customer requires more capacity, all they have to
do is add another node and let the cluster reorganize itself.
Elasticsearch utilizes Lucene, a high-performance text search
engine library, to enable search with multi-language support,
an extensive query language, geolocation support, context-
sensitive suggestions and auto completion.
Elasticsearch also allows the customer to store complex, real-
world entities as structured JSON documents with a default
index for all fields that enable the user to achieve a precise
result within seconds.
Key Features
Fliebeat Server 1
Server 2Fliebeat
Kibana
Dashboard
ELK STACK / LOGSTASH SERVER
Grafana
Dashboard
Reverse Proxy Store Logs Process and
Index logs from S3
Niginx Elastic Search Logstash
Mounting respective
S3 bucket
Amazon S3
6. Cloud Management
AWS and Rightscale
Continuous Integration
Jenkins
Build Tools
Ant
Scripting Languages
bash, python
Log Management
Elasticsearch, Logstash, Filebeat, Kibana, Grafana
and Nginx
ECS Management
Ansible, SonarQube, MySQL, Docker, ECS and ECR
THE BENEFITS
Aspire’s efforts in creating a complete DevOps implementation
plan by using the latest tools and methods allowed the
customer to improve their product efficiency by 60%.
The ELK stack gives the customer the power of real-time data
insights and analytics along with the ability to perform super-
fast data extraction from virtually all structured and un-
structured data sources.
FUTURE IMPACT
Aspire’s systematic, modern approach to containerization helped
the customer to transform the way their project adopted DevOps
practices and will enable them to scale with exponential
efficiency.