.

1. XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Improvement from Proof Of Concept into the
production environment : cater for high-
performance capability
Nor Izyani Daud
Information System Security Lab
MIMOS Berhad
Kuala Lumpur, Malaysia
izyani.daud@mimos.my
Dahlia Din
Information System Security Lab
MIMOS Berhad
Kuala Lumpur, Malaysia
dahlia.din@mimos.my
Galoh Rashidah Haron
Information System Security Lab
MIMOS Berhad
Kuala Lumpur, Malaysia
rashidah@mimos.my
Moesfa Soeheila Mohamad
Information System Security Lab
MIMOS Berhad
Kuala Lumpur
soeheila.mohamad@mimos.my
Abstract— We developed the Trust Engine as a component
of an adaptive multi-factor authentication system. The Proof-of-
concept was tested and found to meet all functional
requirements, but it does not meet the scalability requirement.
It is crucial to make the Trust Engine scalable because the
authentication platform serves applications that have up to
millions of users. We discover that the architecture and
database design of the Trust Engine component is the root cause.
Thus, we redesigned the database and modified the architecture
while maintaining correctness of the functions and compatibility
with the authentication platform. Here we present the
techniques we use and show the success with performance test
results.
Keywords—improve performance, high-performance system
architecture, adaptive authentication, scalable system
I. INTRODUCTION
Good software quality is not only mean the software is free
of bugs. Many other factors need to be considered before we
can claim that the software has a good quality. Paper [9] by
Padmalata Nistala, Kesav Vithal Nori and Raghu Reddy
defined software quality is a degree to which a software
product satisfies stated and implies needs. Good software also
needs to be evaluated by the other factor, for example, the
efficiency, accuracy and the speed of processing. This
situation led you to define a good quality of system also needs
to be capable of the high-performance.
Most of the organization nowadays are moving forward
for the scalable system where you needs to have a high-
performance system. IDC reports [8] predicts that the global
data volume will grow from 130 exabytes to 40000 exabytes;
for the year 2005 to 2020. Social media, for example, Twitter,
Google+ and Facebook, has a large user base as well as an API
for extracting data [10]. To achieve the scalable system that
can support the high-performance, we also can perform
changes in the database. Article by Micheal Rys [11] indicates
that the MySQL database can be scalable by using a NoSQL
database.
In line with this, we adapt the technology for our Trust
Engine component, which is deployed in an authentication
platform. In this paper, we discuss techniques or methods
implemented by others to improve the performance of the
system. Then we introduce our Trust Engine and how it is used
in our Authentication platform, followed by the scalability
issue and the causes identified during the transition to
deployment. The discussion includes the experiment that we
conducted for the Trust Engine system. We elaborate on the
changes made on the POC system architecture and database
design. The performance improvement is shown by
experiment results on the new Trust Engine system
architecture and design.
II. RELATED WORK
There are many approaches conducted by others to
improve system performance.
Paper written by Kun-Che Hsu and Jenq-Shiou Leu in [4],
suggested using JSON instead of XML in their exchange
presence information. The authors experimented by replacing
the XML with JSON for exchange presence information.
Based on the experiment, it shows that data size encoded by
JSON is 40% smaller than XML. By using JSON also, the
transmitting time is shorter, and it consumes less system
resource. The author concludes that JSON data format could
significantly decrease the time of transmitting the presence
messages. It is because JSON is a lightweight architecture
where it gives a high impact on performance.
Analysing scientific data involved huge data sets which
requires long computation time. To speed up those processes
[7] proposed Fast Analysis with Statistical Metadata (FASM).
Their method requires some metadata to be added to the data
set . The metadata storage overhead is small compared to the
resulting dramatic performance improvement. Performance in
the areal application shows that FASM achieves 3.5 time
speed up.
Other works suggest performance improvements to web
applications utilizing eXtensible Markup Language (XML).
XML is a standard format to pass data from one machine to
another. Performance of the application is affected by XML
for the time required to parse and the size of bandwidth or
storage. In [6] the XML technology is used with SQLCLR to
improve the application performance. The SQLCLR method
enhances the native Transact-SQL programming model by
enabling the creation of database objects including store
procedures. Such an approach enables complex computational
tasks in the database server processes. On the other hand, [4]
achieves performance improvement by utilizing JavaScript
Object Notation (JSON) instead of XML in their presence

2. information exchange. The change to JSON format decreases
the data size by 40%, shortened transmission time and reduces
system resource consumption. The authors conclude that
JSON is more efficient to use than XML because JSON is a
lightweight architecture.
Paper entitled ‘Comparison of JSON and XML Data
Interchange Formats: A Case Study’ [5], study the
comparison between the XML and JSON. Based from the
experiment conducted in the paper, they concluded that JSON
is faster and uses a few resources compare to XML.
III. CASE STUDY
A. Background
MIMOS had developed an authentication platform;
Unified Authentication Platform (Mi-UAP) [1]. Mi-UAP is
designed to manage front-end application authentication using
an established protocol; Secure Assertion Markup Language
(SAML). It is also capable of Single-Sign-On where the user
only needs to login once and access multiple application that
integrated with Mi-UAP.
Mi-UAP consists of 4 major components; UAP Gateway,
UAP Server, Web Application Server and Trust Engine [12].
Fig. 1 below illustrates Mi-UAP system architecture.
Fig. 1. Mi-UAP System architecture
In Mi-UAP, when user wants to login, it allows the user to
choose any authentication method that the user preferred. The
example of the authentication method that user can choose is
password, Mi-TCK, Mi-2DBC and MyDigital ID. Mi-UAP
then checks the user credential before sending the information
to the Trust Engine component. The Trust Engine component
then checks is the credential that the user provides is enough
for the required trust level for the application or not. Once it is
enough, the user will be directed to the application.
Trust Engine act as a backend component to support
adaptive authentication in Mi-UAP. In this component, it
analyzes the user login information, for example, time,
location, fingerprint and browser that user use when they login
to Mi-UAP. It then performs the calculation base from the user
login information. In the end, it decides whether the user is
allowed to access the application or require to provide another
authentication method before the user can access the
application.
We deployed Mi-UAP, including Trust Engine, to the
internal organization during the Proof of Concept (POC). The
purpose of having the POC deployment is to have a full load
of performance evaluation from the entire MIMOS
organization. In our current practice, once the POC
deployment success, we deploy the system to the production
environment. We are expecting the number of uses for the
system increase; therefore, POC deployment success is crucial
for us. We are focussing on the performance evaluation of the
system during the POC period.
B. Finding from POC system deployment
During the POC period, we monitored the smoothness of
the system, efficiency and system performance. Based on our
investigation, we find out that there is an issue on the system
architecture and design, including the database design and
system performance.
In Mi-UAP, each time when the user wants to login to the
system, it called Trust Engine component to evaluate the user
login information. Trust Engine checks the time, browser
operating system, location, browser fingerprint and
application that user is accessing during the login.
In the current system design, Trust Engine analysed last 30
records for the particular user login information and compared
with user current login information. All the information also
needs to be compared against the current configuration that is
stored in five difference tables.
After performs calculation based on the information given,
the component decides to allow the user to login to the system
or provides another authentication. The system then inserts the
new login information to the database. The system also needs
to updates the last 30 records of user login information.
Based on our finding, the POC system design requires the
component to have multiple connections to the database. It
requires the component to access to six tables to get the
configuration and three tables to get the user data. With the
number of tables uses in the component, during the POC
period, the component made 60% of the connection to the
database compare the other components in Mi-UAP.
The database design also requires the system to connect
and get data from three different tables for one particular user
at each time when the system needs to perform the calculation
process. Once it complete, the component also requires the
user to update to one of the user tables to include the update
current user login information in three different tables. With
the connection to many tables for the system, it took time for
the component process to be complete.
The figure below shows the Trust Engine process and the
connection to the database for Trust Engine component.
Receive user current login
information
Get last 30 records and
configuration use
Analysis user data
Decision on user login
Update user login information
End
User and
configuration record
Configuration record
User record
User record
Start
Fig. 2. Trust Engine process flow

3. We conducted a performance evaluation of the current
database during POC period. Currently, there are 1674349
records for the entire user. In this activity, we select a random
user to evaluate Trust Engine process. There are two processes
involved in Trust Engine.
Process A is to select the latest 30 logins information from
the database. It then inserts the current user login information
to the database. Besides that, process A also selects the
configuration and IP address information in the analysis
process.
Process B is to update the latest user login behaviour. This
process involves SQL operation such as insert, update or
delete operation to the database for the last 30 records of user
login information.
When we conduct the performance evaluation, we select
ten users for the sample data. Each of the users has a different
total number of records in the database. We execute the
process ten times for each of the users and calculate the
average time taken for each of the processes. Table I illustrates
the performance evaluation information.
TABLE I. TRUST ENGINE PROCESS TIME
User
No. of
records
Process A
(second)
Process B
(second)
Total
(second)
User A 3 47.886 47.666 95.552
User B 17 35.482 35.336 70.818
User C 28 44.459 44.215 88.674
User D 63 29.602 29.466 59.068
User E 154 29.460 29.403 58.863
User F 436 28.908 28.775 57.683
User G 533 36.961 33.344 70.305
User H 732 32.284 32.226 64.510
User I 1058 30.430 33.507 63.937
User J 2642 34.611 40.322 74.933
Average 70.4343
Table I show the result for the Trust Engine process time
during the POC period. The average time for all the users is
70.4343 seconds. Based on the result obtained, the
performance of the Trust Engine are not encouraging.
We have decided to change the system architecture and
design for the Trust Engine process flow, including the
database design to address the problem.
IV. IMPROVEMENT
We had made changes to the Trust Engine component to
improve the performance. The list of changes are:
A. Remove all configuration tables to the configuration file.
In the new system architecture and design, we put all the
configuration data into the configuration file and implement a
Singleton concept. Singleton[2] is a method where it only has
one instance, and it provides a global point accessing into it.
It means that when the first time the program is executed, it
gets all the configuration and put it in the public class. When
the user calls the program again, it uses the same object
without having to initialize it again.
B. Combine user information into one user table and limit
the number of records.
In the new Trust Engine system architecture and design,
we combine all the information in one table. Besides that, we
also limit the number of login information to be stored in the
database to 30 records. Since the system only analyses the last
30 logins information, we decide to keep the last 30 records
for the user login information.
C. Change the process flow and system architecture and
design for Trust Engine component
In the new process, the system select user record from the
database and the variable use for the program from the
configuration file. It then constructs all the login information
that the system received in JSON format and evaluates the
data. The system then decides either user is allowed to login
or needs to provide another authentication method. Once the
process complete, system reconstruct back the JSON data to
include the latest user login information and insert the new
record to the database.
The analysis and evaluation process does not involve any
connection to the database. All the process is being executed
in the memory of Trust Engine server. The new process is
highlighted in the new step in Fig. 3.
Start
Receive user current login
information
Get configuration use
Get user data
Analyze, evaluate and
construct user record
Delete and insert user data
End
User record
Configuration file
Fig. 3. New process flow for Trust Engine
D. Change the data type for the information stored in the
database.
We also change the data type to JSON. JSON [3] or also
known as JavaScript Object Notation is a text-based,
language-independent data interchange format for the
serialization of data. In this approach, each of the users only
has one record in the database. In one record, we have the
latest last 30 login information. The example in Fig. 3 shows
how we store the information for the particular user in JSON
format.

4. Fig. 4. Example of data format in JSON
Based on the changes that we implement, we conducted a
performance evaluation for the new system architecture and
design. We took ten random users from and ran the Trust
Engine process for ten times. Then we calculate the average
time taken for the Trust Engine process for the user. Table II
below shows the result based on the performance evaluation.
TABLE II. NEW TRUST ENGINE PROCESS TIME
User No. of record Total time complete (second)
User A 1 0.19419354
User B 1 0.09636536
User C 3 0.42871727
User D 6 0.29577776
User E 6 0.15877049
User F 6 0.41566072
User G 8 0.16576271
User H 28 0.25104779
User I 30 0.14864599
User J 30 0.29226241
Average 0.244720404
Based on Table II, it shows that for all the users, it only
took less than 0.24 second in average to complete the Trust
Engine process. The time was 99% faster compared to the
POC system design and architecture. With this result, it shows
that the improvement we made is efficient, and it can cater to
the high-performance usage for future deployment.
V. CONCLUSION
The weak system design and architecture design of the
Trust Engine in POC deployment is due to the changes and
enhancement during our research. During the research, we add
new features without properly study the impact on system
performance.
The solution that we give here might not be the good one
for others. However, for us, with the new system design and
architecture, we believe that the system is ready for the high-
performance usage during the deployment to the production
site.
Even though we have solved the performance issue in the
new system architecture and design, we still need to monitor
the performance of the system. We also need to explore the
other technology that we can embed to our system for future
enhancement.
ACKNOWLEDGMENT
We want to thank the project leader for the support and
encouragement during the performance evaluation period
during the POC period. The commitment from the team
members also helps us to improve the current design for Trust
Engine so that it is capable of the high-performance load
during the deployment to the production environment.
REFERENCES
[1] N. I. Daud, G. R. Haron and D. Din, "Adaptive Authentication to
determine login attempt penalty from multiple input sources," 2019
IEEE Conference on Application, Information and Network Security
(AINS), Pulau Pinang, Malaysia, 2019, pp. 1-5.
[2] Sarcar, Vaskaran. "Chapter 3 - Singleton Patterns". Java Design
Patterns: A Tour of 23 Gang of Four Design Patterns in
Java. Apress. © 2016. Books24x7.
<http://library.books24x7.com/toc.aspx?bookid=112048>
(accessed February 21, 2020).
[3] Stokes, David. "Chapter 1 - Introduction". MySQL and JSON: A
Practical Programming Guide. Oracle Press. © 2018. Books24x7.
<http://library.books24x7.com/toc.aspx?bookid=142596>
(accessed February 21, 2020).
[4] Kun-Che Hsu and Jenq-Shiou Leu, "Improving the efficiency of
presence service in IMS by JSON," 2015 Seventh International
Conference on Ubiquitous and Future Networks, Sapporo, 2015, pp.
547-550.
[5] NURSEITOV, Nurzhan, et al. Comparison of JSON and XML Data
Interchange Formats: A Case Study. Caine, 2009, 9: 157-162.
[6] Y. Zheng, L. Wang and J. Xue, "A High Performance Solution for
Automated Computer Examination Systems," 2007 First IEEE
International Symposium on Information Technologies and Applications in
Education, Kunming, 2007, pp. 369-373.
[7] J. Liu and Y. Chen, "Improving Data Analysis Performance for High-
Performance Computing with Integrating Statistical Metadata in Scientific
Datasets," 2012 SC Companion: High Performance Computing,
Networking Storage and Analysis, Salt Lake City, UT, 2012, pp. 1292-1295.
[8] J. Gantz and D. Reinsel, ``The digital universe in 2020: Big data, bigger
digital shadows, and biggest growth in the far east,'' in Proc. IDC iView,
IDC Anal. Future, 2012.
[9] P. Nistala, K. V. Nori and R. Reddy, "Software Quality Models: A
Systematic Mapping Study," 2019 IEEE/ACM International Conference on
Software and System Processes (ICSSP), Montreal, QC, Canada, 2019, pp.
125-134.
[10] Yin Huang, Han Dong, Yelena Yesha, and Shujia Zhou. 2014. A scalable
system for community discovery in Twitter during Hurricane Sandy. In
Proceedings of the 14th IEEE/ACM International Symposium on Cluster,
Cloud, and Grid Computing (CCGRID ’14). IEEE Press, 893–899.
[11] Michael Rys. 2011. Scalable SQL. Commun. ACM 54, 6 (June 2011), 48–
53.
[12] K. A. A. Bakar and G. R. Haron, "Adaptive authentication based on
analysis of user behavior," 2014 Science and Information Conference,
London, 2014, pp. 601-606.