AUTOMATED INFRASTRUCTURE CREATION THROUGH JIRA BUG
TRACKING SYSTEM
_______________
A Thesis
Presented to the
Faculty of
San Diego State University
_______________
In Partial Fulfillment
of the Requirements for the Degree
Master of Science
in
Computer Science
_______________
by
Shilpa Sood
Summer 2011
iii
Copyright © 2011
by
Shilpa Sood
All Rights Reserved
iv
DEDICATION
This work is dedicated to my family, my mentor, who guides me to accomplish my
goals and always motivated me for my hard work.
v
ABSTRACT OF THE THESIS
Automated Infrastructure Creation through JIRA Bug Tracking
System
by
Shilpa Sood
Master of Science in Computer Science
San Diego State University, 2011
It takes tremendous amount of time for developers to setup the environment to fix a
bug in an application. Depending upon the complexity of application, it may take a few days
to several weeks for developer to replicate a production environment. Very few solutions
exist today that help reduce that time. With the help of Amazon EC2 you can in theory set up
instantaneous infrastructure through a bug tracking system (JIRA).For each application, the
developers usually need a machine for the old platform on which the bug exists, and a
separate machine to install the new environment, which has the bug fix. In some situations,
the QA environment could need a separate machine for all the testing. If the total number of
applications increases to say 40, then this leads to a maintenance nightmare, and the whole
solution would be really expensive. The solution will consist of extending an issue tracking
system to capture the configuration of a specific environment through a plugin and automatic
installation and configuration of a system through templating of a system.
vi
TABLE OF CONTENTS
PAGE
ABSTRACT...............................................................................................................................v
LIST OF TABLES................................................................................................................. viii
LIST OF FIGURES ................................................................................................................. ix
CHAPTER
1 INTRODUCTION .........................................................................................................1
2 BACKGROUND AND LITERATURE........................................................................6
WebWork.................................................................................................................9
Seraph ......................................................................................................................9
PropertySet.............................................................................................................10
Cloud Computing...................................................................................................11
3 METHODOLOGY AND DESIGN.............................................................................19
Introduction............................................................................................................19
JIRA Plugin Overview and Software Requirements .............................................19
PicoContainer and Dependency Injection..............................................................21
JIRA Interaction with Database.............................................................................22
Presentation Layer ...........................................................................................24
Business Layer.................................................................................................25
Data Layer........................................................................................................25
Development Phase................................................................................................25
Velocity Overview.................................................................................................27
Mockup Description...............................................................................................29
Functionality of the System ...................................................................................30
vii
4 RESULTS ....................................................................................................................36
5 CONCLUSION AND FUTURE WORK ....................................................................43
REFERENCES ........................................................................................................................48
APPENDIX
ACRONYMS.....................................................................................................................50
viii
LIST OF TABLES
PAGE
Table 1. Types of Dependency Injection .................................................................................22
ix
LIST OF FIGURES
PAGE
Figure 1. JIRA architecture........................................................................................................8
Figure 2. OfBiz framework......................................................................................................23
Figure 3. Example of pom.xml. ...............................................................................................26
Figure 4. Mock up diagram......................................................................................................29
Figure 5. Screen shot of admin screen.....................................................................................30
Figure 6. Screen shot of issue screen with launch button........................................................31
Figure 7. Screen shot of add admin screen. .............................................................................37
Figure 8. Screen shot of role validation...................................................................................38
Figure 9. Screen shot of APT repo validation.........................................................................38
Figure 10. Screen shot of cookbook repo validations..............................................................38
Figure 11. Screen shot of edit admin screen............................................................................39
Figure 12. Screen shot of delete admin screen. .......................................................................39
1
CHAPTER 1
INTRODUCTION
Today, almost every company, large or small, uses a bug tracking system. It is one of
the most basic components of a development process. No system is made perfect in the first
iteration. Bugs always exist in software’s, and a bug tracking system gives an organization an
estimate of the amount of work that needs to be done to approach the goal of perfection.
Server machines are the other component of the development process. Sizable software needs
big machines with a large amount of processing power and memory so that thousands of
users can be connected to it at the same time. Our motivation was to combine the above two
ideas and make them into one synchronized system. We have integrated an open source bug
tracking system (JIRA) with one of the best virtualized server providers (Amazon) so that the
developers can call from one system to another, and can have a very powerful server machine
with the single click of a button.
There are various companies offering bug tracking systems, but JIRA is one of the
best open source products available in the market today. Its open source nature makes it
possible for thousands of developers to use and work on it on a daily basis, thereby enabling
much more rapid updating than its closed code competitors. The open source model also
makes sure that all the security vulnerabilities of the bug tracking system are quickly
identified, and are fixed with the next possible release. As the source code is available to the
public, companies can use the Java based source code, modify the front end or backend
according to their requirements, and create a completely customized product for their
company. They can even release that product back into the development branch of JIRA and
contribute to the continuous development of the project. JIRA is used by many big and small
companies for their day-to-day operations; therefore, one can imagine the scalability of the
software, which can shrink and grow quickly depending upon the daily requirements. There
are hundreds of plugins of JIRA available on the market, some of them cost and some are
free. Different companies can use these add-ons to connect to different kinds of systems.
2
Cloud computing is one of the buzz words in the computing industry today. The kind
of flexibility offered by cloud computing makes it an ideal candidate for use in all kinds of
companies, from dynamic start-ups that require fast infrastructure to big companies which
are trying to consolidate their huge infrastructure by going towards the virtual route. In this
way they can save money by creating virtual machines that require less space and money
when compared to a situation in which they require a physical hardware machine for every
situation. If we have to assign a physical machine for every role in the project, that would
involve considerable investment in power for the machine, cooling equipment and the
personnel to configure and maintain all those machines. As the inventory of physical
machines grow over time, all the above mentioned resources would keep on increasing. With
a virtual infrastructure, all the above mentioned investments would be considerably lower,
and that would save a company a lot of money which could be shown as profit at the end of
the year. If a company uses a third party virtualization provider like Amazon, the company
does not need to worry about all the above mentioned resources, and their engineering
activities could take preference.
There are a lot of virtualization providers in the market today, each of which has their
own unique selling point. Every company is trying to move their services to the cloud to
harness the power of mobile computing, so that all the users of the company can use this
service from any place. As long as the user has a desktop computer and a browser with an
active internet connection, the user is given power to connect to really powerful machines
and either use the machine directly for their own use or connect to those machines and use
some of the services hosted by the cloud provider. Let’s take an example of a big company
like Google, which offers one of the most used mail services, called Gmail, by harnessing the
power of the cloud. All the servers that have the Gmail service hosted are on the cloud and
have some sort of clustering scheme thus ensuring that the service is not affected by one of
the server machines going down because the data on that server is quickly replicated to one
of the other servers and the other machine can take the place of the machine which went
down initially. All the above mentioned steps should be transparent to the user so that he has
absolute confidence in the availability of the service. Similarly, there are other web services
provided by companies like Yahoo, Microsoft, and Salesforce, etc., which provide a unique
3
functionality to the user. All these vendors are said to be using the “Software as a Service”
(SaaS) functionality of cloud computing.
There are other cloud computing providers that provide the “Infrastructure as a
Service” (IaaS) of cloud computing. One of the most popular companies providing that
service is Amazon which offers its suite called “Amazon Web Services.” This suite consists
of different services such as Amazon Simple Storage Service (S3) which provides a huge
amount of storage to the users so that they can upload different things like their personal
information, as well as other things that are to be used by other services in the same suite.
The users of this service first have to make an entity called “bucket” which is the
encapsulating identity for the information. The users can then use tools provided on the
Amazon Management web interface to upload the data, or can even use Amazon Command
line AMI and API tools to upload the different kinds of data in the bucket. This data can then
be retrieved either by using the same suite of command line tools, or using the Web interface
to download the items on some other computer at a different location. Amazon Elastic Cloud
Computing (EC2) is another service that the users can use to get (rent) powerful machines,
which can be used to host different kinds of services required by the user. A user is given an
option of choosing a small, medium or large instance, each of which is different in terms of
the processing power and the amount of memory available on the machine. A small instance
usually has 4 GB of RAM with a dual core processor (this configuration can be slightly
different depending upon the user), which is a good enough machine and powerful enough to
run a small service like a web server which can be used to service hundreds of users with a
simultaneous connection. The machines can have different kinds of Operating systems such
as Windows or Linux, each of which can have different features and different architecture,
and the user has the option of choosing an operating system which best suits his
requirements. All of these different kinds of machines can be hosted in different zones. The
United States has different zones available; the same applies to other different geographic
parts of the world like Europe and Asia. Choosing machines in different zones and then
replicating information between them ensures that the machines have a high resistance to
different natural disasters. And if the machines in a particular zone are affected due to some
uncontrollable circumstances, there are other machines always available in other parts of
world which ensures that the service provided by the user using Amazon’s infrastructure has
4
a readily available service resistant to different kinds of disasters. There are a lot of other
services provided by Amazon Web Services suites, such as Amazon Elastic Map Reduce,
CloudFront, and Elastic Benstalk. All of these services are tightly integrated and work
closely with each other to give complete control over every aspect of the cloud computing
experience.
In this project, we have used different technologies like Velocity, Ofbiz and JDBC on
the JIRA side to create a different screen that will be used by the administrator of the system
to program different “Nimbo” related attributes in the standard JIRA bug tracking system.
We have created a web page called “Admin Nimbo Screen” that will be used to associate a
project with the Nimbo functionality. We have to first log in using the administrator user
account that has the privilege to access all the projects of the company mentioned in the JIRA
bug tracking system. Once we do this, and we have logged in using the administrator user
account, we get a list of all the projects in the company. We can then select a single project
and fill in the values for all the Nimbo related attributes like Role, ImageId, Apt Repo and
Cookbook Repo and then click the submit button to store all of these values. Once the submit
button is clicked, we validate all these attributes for their correct values so that we do not
have any problems while the image is being generated on Amazon EC2 for these projects.
Different checks like the NULL value check are done to make sure that we don’t insert any
blank values into these attributes. We also check to make sure that the value for cookbook
repo and apt repo has a web address inside them to ensure that they have an http or an https
value in those attributes. Once all the checking is done, and the values entered are valid, we
then store these values using a SQL query (using JDBC connection) in a separate table called
PROJECTNIMBO that has the “Project Name” as the primary key of the table.
Once the Nimbo functionality is enabled on a particular project, and a role is
associated with the project, all the issues in that project get a “Launch Nimbo” button on the
top left-hand side of the issue screen, which can be clicked by the developer working on that
issue to generate an image for that particular role by just clicking that button. Once that is
done, the user is taken to another screen which shows that the image is being created on
Amazon EC2, and then all the software’s associated with that role will be automatically
installed on that machine. At the end, he will be sent mail to the email address associated
with his user account on the JIRA bug tracking system with the instructions to connect to the
5
machine using a SSH client. The administrator of the JIRA system has to make sure that the
key that is used to connect to those machines is already given to the developers, and that they
know all the steps to follow in order to connect to that machine effectively.
When the user clicks the button to launch the image, a server side Java wrapper
program is called on the JIRA server that uses the Amazon Web Services (AWS) (for list of
acronyms see Appendix) Software Development Kit (SDK) with the credentials of the
account used to generate images on the Amazon EC2. The JIRA plugins call this server side
program by giving the issue key and the username of the developer as the input; the program
also uses the ImageCreation.properties file to take the image ID and the key name that is to
be used for the AWS SDK tools to create the image on Amazon EC2. After the commands to
make an image are given, we wait for a fixed amount of time (again picked up from the
ImageCreation.properties) and then pass the control over to other programs which will do the
installation of software’s on the machine and then send the user the email with the
instructions to connect to the machine.
All the above actions will give us a system in which we skip all the delays associated
with getting the hardware of a machine allocated to our issue, and then coordinate with the IT
department that installs all the software’s on the machine. In the case of large companies, this
delay can be significant which could waste time doing a simple action like getting the
machine ready so that the developer can create a software image in the production
environment required to reproduce, resolve and verify a fix for the issue. By using this
system we can get a machine with a single click, so that the developer can start working on
the issue with a new machine in a matter of few minutes.
In the next chapter, we will give an overview of all the technologies used to achieve
our goal of making the complete Nimbo System.
6
CHAPTER 2
BACKGROUND AND LITERATURE
Bug tracking systems are very important for the maintenance of a project [15]. When
we work on any software related project, we have to manage the bugs that exist in the
application and fix them. Some developers list them on a spreadsheet, but it becomes difficult
when a project is big and the bugs become too many, and when there are many developers
working on the project. Therefore, to maintain a project efficiently, a bug tracking or issue
tracking system is very necessary, especially during the testing or deployment phase. In
software development, bugs are always unavoidable, or we can say they are a part of the
system; hence, our choice of bug tracking [5] should be one which is simple and intuitive
with all the functionality. There are many such bug tracking systems on the market; some are
open source, others are commercial projects. JIRA is an example of such a bug tracking
system and this system will be explained in detail.
JIRA is a powerful tool, developed by Atlassian, and it can be used for various
purposes like bug tracking, maintaining the projects and also planning tools for agile
projects. JIRA is available as an open source project for free to organizations that are non-
profit, and also the full source code is available for purchase by commercial customers. JIRA
is used for issue tracking and project management by many organizations and in many
countries by various sectors. It is used in software organizations to improve the quality of
code and to speed up development. JIRA as a bug tracking system simplifies every step
involved in tracking a bug for everyone involved in the process. With its intuitive interface
creating, resolving and tracking bugs is very easy and quick. Fixing a bug for a software
development team is just like fixing code [19]. It is the most important task for any software
development team for the quality and performance of the product. To make life simple, with
Atlassian IDE Connectors, developers can interact with JIRA directly through Eclipse and
many other IDE. We can always connect JIRA bugs to the source code with the CVS
integration and plugins to any version control system like Subversion, ClearCase, and
Mercurial. Also to make the whole system more efficient, continuous integration can be
7
performed to view the builds and integrate the comments related to the JIRA bug from the
code reviews into the project. JIRA can be extremely powerful depending upon the need of
the user and the user’s knowledge about its features.
JIRA as a project management tool can be extremely useful to enable the project
managers to speed up the work and track the projects of their respective teams [3]. JIRA is
focused on the task achievement and manages all the projects and components in one
location. It tracks all the projects, manages the task list, assigns an unlimited number of tasks
and sub-tasks. It assigns the description of the project as well as about the task, can attach
files and even customize the work flow. To keep the team on track, JIRA has the ability to
make time tracking reports for each individual, set up a road map and status updates on the
dashboard, and can even export report data to Microsoft Word, Excel, XML and as RSS
feeders. JIRA also helps the developer and tester monitor the progress of the project by the
status updates on the dashboard, by the charts and reports on the personal dashboard, and by
automatic reports subscription and email updates. It also provides secure online access
through granular security schemes and by enterprise level security management. Time
tracking can also be helpful in managing the projects by setting up an estimate for each task,
time spent on the logs of each task, time tracking record of each phase of the project. Also it
can track risky tasks in a project and set up RSS feeds on risky tasks. JIRA can also be
integrated with an MS project to speed up the project manager tasks.
Agile project management with JIRA is another useful functionality for the software
development teams [4]. There are a set of tools used to extend JIRA for the agile
management, as JIRA is not explicitly designed for use in the agile environment.
GreenHopper is a full implementation plugin mainly used for Agile Developers using JIRA.
It represents all the JIRA issues in a very simple form; for example, different issues are of
different colors. Summary and other crucial information to estimate the task, time remaining
and planning are also displayed. To reorder the tasks, just drag and drop which is always
useful for any project manager to save time. GreenHopper also offers a Task Board to show
the work queue for a component or version. Chart Board will show the burn down charts for
a release. The graphs shown by the Chart Board are a burn down curve, team effort curve,
estimation accuracy curve, and many others. I have explained in brief some of the uses of the
8
JIRA for bug tracking and project management. Now, we will give some technical
introduction about the JIRA.
JIRA is a web application written in java. It is deployed as a standard Java WAR file
into a java servlet container such as Tomcat.
Figure 1 is adapted from http://confluence.atlassian.com/display/JIRA/JIRA
+Architectural +Overview [9].
Response
Request
Figure 1. JIRA architecture. Source: J. NOLEN, JIRA architectural overview. JIRA 4.4
Early Access Program, http://confluence.atlassian.com/display/JIRA/JIRA
+Architectural +Overview, accessed June 2011.
Understanding the JIRA architecture is very important for the developers who want to
extend the system and also to the testers and other users. As we know, JIRA is a Web
Application and the user interacts with it through a web browser as shown in the figure
above. We will now explain the technical details of each component used in the JIRA
architecture.
Lucene
Index Files
Lucene
JSP
Seraph
Web
Browser
Jelly
RPC
Atlassian
Scheduler
Quartz
JIRA
Service
Relational
DBMS
OSWorkflow
OfBiz Entity
Engine
PropertySet
JIRA Utility
and Manager
Classes
Web Work
Web Work
Action
Web Work
Action
Web Work
Action
Web Work
Action
OSUser
JIRA
Service
9
WEBWORK
JIRA uses OpenSymphony’s [22] Webwork1 to process the requests submitted by the
users. One of the most important task in developing the web application framework is to
separate the code, logic and presentation. If this concept of separation is not handled wisely
then it is most likely to have maintenance problems in the application. As the whole team
works on the same application, if there is no structure then it becomes really difficult to have a
clean and maintainable code. One of the most popular ways to achieve this separation is to use
a design pattern called Model-View-Controller (MVC). WebWork is a MVC framework, or
we can say service invocation framework. It is architecturally based on the design patterns and
best practices which have been proven very useful in building a stable and simple framework
while maintaining flexibility. It is mainly built on the concept of actions and views. WebWork
actions handle the entire request by the users and uses other objects such as utility and
manager classes to complete the given task.
An action in WebWork is the piece of code (normally java class but not necessarily)
containing the entry point, a method named execute () to begin its processing. Actions have a
particular lifecycle in WebWork. It uses a dispatcher to look up the entry point and take the
request itself, and then maps it to a particular action called the execute () method. This method
returns a java.lang.String, which is mapped to another Action, or a View. The view in the
WebWork generates the output and JIRA uses JSP for the view layer. Views can be anything
like JSP, Velocity, Swing, XML/XSL, and Jasper which means WebWork supports different
HTML generation technologies and gives the users the flexibility to choose their own ways to
accomplish their work. A View gets its particular request data, and it serves users as a
response to their request with the HTML generated by the views, JSP in the case of JIRA.
When views take a request data it looks for it on entire action sets and returns the first relevant
match for the data.
SERAPH
Seraph is the Atlassian’s open source web authentication framework used for most of
the authentications for JIRA. It is based on a very simple, J2EE pluggable web application
security framework so that the authentication system can be used on any application server. It
is implemented as a Servlet filter and based around the number of core elements pluggable in
10
the framework. Seraph gets a web request and associates the request with the particular user.
It supports different types of authentications including the form-based authentication, HTTP
Basic authentication, and credential authentication for users already in the session. To decide
the role required for any known request we use Security services in Seraph. The WebWork
Service is packaged with Security services to secure the WebWork 1 actions. To implement
this service we use action.xml to configure the actions in the web application and add “role-
required” as an attribute to each action component. Also there is a Path service along with
WebWork Service in the Security services package which secures URL paths and is
configured in the security-config.xml file.
To configure Seraph properly to the web application, we need to modify 2 files: one
is seraph-config.xml which resides under WEB-INF/classes directory and the other is
web.xml under WEB-INF used to add a servlet filter. There is an Authenticator class in
Seraph used to handle user authentication. To integrate JIRA with the single sign-on (SSO)
solution in your applications, custom authenticator’s can be written to implement this
concept. JIRA does not contain the SSO solution by default; therefore, to integrate, one has
to write one’s own custom authenticator according to the needs. One of the very important
purposes of the Seraph in JIRA is to allow only the users with Global Admin permission to
access the WebWork Action that allows users to manage the administration tasks. These
WebWork actions are accessed by URLs starting with "/admin."
PROPERTYSET
PropertySet is an OpenSymphony’s framework used to accumulate a set of property
(key/value) pairs against a specific entity with a unique ID. It can be used to accomplish
storage necessities for the applications which change frequently. Every property will have a
key and its corresponding value which can be java.lang.String, java.lang.Date,
java.lang.Double, java.lang.Long. Each property is always connected with one entity. In
order to store and retrieve the value, the same entity name/ID pair is used and everything
works well. It is impossible to figure out what user can store at any given time in your
application, so utilizing the Propertyset can help. A Typed key/value, supported by XML,
ofBiz, JDBC and JGB, or any other persistence mechanism can be implemented. In JIRA,
PropertySet is used by the UserPropertyManager to store the user preferences and to store
11
Application Properties, which can be used to customize the JIRA installation by changing the
configurable setting by the user and can also be used to store the preference of Portlets on the
user’s dashboard. PropertySet uses the following database tables in JIRA:
1. propertyentry - contains the entity name and ID for a property, its key and the data
type and also each entry has its own unique ID
2. propertystring - contains string values
3. propertydecimal - contains double values
4. propertydate - contains date values
5. propertynumber - contains long values.
The majority of PropertySet implementation store data in a relational database. Sometimes
the speed of querying the data on the tables is slow as it stores a wide variety of data which is
unknown at the time of development and the tables are not very optimized or in a normal
form. So creating the indices on the OS_PROPERTYENTRY table will speed up queries by
a high number factor. Other optimization techniques can also be used beside the indices; for
example, by properly and evenly distributing the key name while writing the application, the
database can have much faster access to PropertySet.
CLOUD COMPUTING
For a flexible, on-demand, and dynamically scalable computing infrastructure for
various applications, the solution which comes to mind is Cloud Computing [7, 11]. It can be
defined as a new style of computing in which dynamically scalable and often virtualized
resources are provided as services over the Internet. It has become a significant technology
trend in information technology (IT) processes and the IT marketplace. With the services
offered by the cloud computing providers, we can use different devices, including PCs,
laptops, Smartphone, and tablets to access programs, storage, and application-development
platforms over the Internet. The major advantages of this technology include cost savings,
high availability, and easy scalability. There are three types of cloud computing[20]: (1)
public cloud, (2) private cloud, and (3) hybrid cloud as described below.
In the public cloud or sometimes known as external cloud computing, resources are
dynamically provisioned over the Internet via Web applications or Web services from an off-
site third-party provider [16]. Public clouds are run by third parties, and different customers
have their applications, storage systems, and networks together on the cloud’s servers. Private
12
cloud or sometimes known as internal cloud means cloud computing on private networks. They
are used mainly for one client only, which provide full control over data, security, and the
quality of service. Private clouds can also be built and managed by a company’s own IT
organization or by a cloud provider. A hybrid cloud combines an environment from multiple
public and private cloud models. Hybrid clouds help in determining the complexity to
distribute applications across both a public and private cloud.
The ability to virtualize and share resources among different applications for better
server utilization is the main advantage of using cloud computing. We can share servers or do
virtualization for the use of operation systems and applications which results in fewer servers.
The major features of cloud computing include Scalability and on-demand services, User-
centric interface, Guaranteed Quality of Service (QoS), Autonomous systems, and Pricing.
Cloud Computing provides services and resources for users on demand, and the resources
provided are scalable over many data centers [17]. The interface that cloud provides is totally
independent and can be accessed by Web services and Internet browsers. They guarantee QoS
for users in terms of hardware, performance, and memory. The cloud computing systems are
autonomous systems and can be managed transparently to users. However, depending on the
needs of the user, the data and software inside the cloud can be reconfigured and consolidated
to a simple platform automatically. It is less expensive compared to the physical resources
since users pay for services and capacity as they need them.
Different customers use the cloud service differently, and the service provider can’t
anticipate the usage of each individual. One customer might use it as a primary development
platform for all of its applications, and another customer might use it two or three times a year
depending upon the need of his business. Due to the different needs of the customers, the
service needs to be available everyday (7 days a week, 24 hours a day), and it should be
designed to be scalable for high periods of demand and for lighter ones. Scalability can be
defined with different meanings; for example, it can mean that an application can be scaled
when the requirement of the application changes and when additional users are added into the
application. Cloud services for customers are much easier to get without going through a
lengthy process such as the services provided by data centers at any software organization.
The customers have to make a simple request for an amount of computing, storage, software,
process, or other resources from the service provider. Of course, the data center gets similar
13
requests from departments of the company and checks for other requests from other
departments to give priority and make sure of the availability of existing resources because of
the need to purchase new hardware. After new hardware is purchased, the data center staff has
to configure the data center for the new application. Depending on company policies, these
internal procurement processes can take a long time.
There are a number of cloud computing providers on the market. Not surprisingly,
some of the biggest names in cloud computing are also some of the biggest names in the
computer world, such as Google, Microsoft, Yahoo!, Salesforce.com, IBM, and others.
Basically, these providers offer similar services such as to compute your tasks on virtual
machines, store your data on remote servers, and use web based applications. As there are
many options, finding the best cloud computing vendor is not easy and you need to assess the
capabilities of the providers very carefully before choosing one. Some users don’t need too
much power serving to perform the computing jobs and they are looking for some free service
provider. Google is an example of a cloud computing vendor that provides a lot of free
services. You can use office applications and communications, store data on remote servers,
retrieve data anytime and use email services that are all web based. Free cloud computing
services would be very useful for personal use, for startup companies, and for small and
medium sized organizations. On the other hand, if you are looking for large storage and more
powerful virtual servers, then there are cloud computing vendors which charge for services.
Such web services are offered by big companies such as Amazon, IBM, and Microsoft.
Amazon is the most widely known cloud computing provider. They offer services on
many different fronts, from storage to platform to databases. The Elastic Cloud Computing
(EC2) of Amazon can provide big savings for companies. Amazon Elastic Compute Cloud is
a web service to provide virtualization of an IT environment to eliminate the need for buying
new hardware to serve your computing needs and is designed to make web scaling easier for
developers. It is a big time and money saver, since by utilizing the services of the Amazon
EC2 you can lease any number of computers and servers for $0.10 per hour in standard
instances. Also, Amazon EC2 saves time because it takes only a few minutes to obtain and
boot new server instances whenever you need to change the scale as per the requirement of the
application. For the big projects that will need large computing capabilities to perform the
tasks efficiently, renting or leasing the virtual computers on EC2 services would be the best
14
choice for big savings in terms of both time and money. Software organizations or companies
can surely get substantial cost saving benefits from EC2 cloud computing and better
performance and faster results from the ability to run computing instances simultaneously.
Amazon Web Service is collection of many web services that people can enjoy and get
benefits from. The EC2 or Elastic Compute Cloud is one of the services which people use
mostly to have their own virtual private server which they can control and manage
themselves. The S3 or the Simple Storage Service is the web service which is very timely and
provides people with safer and better storage for their personal files and documents. The
MTurk or the Mechanical Turk is known to be one of the most complicated services that
Amazon has ever provided.
Amazon EC2 or Elastic Compute Cloud is not the only service but it is the most
renowned of these kinds of web services [18]. Cloud Computing is becoming more famous
among developers and software organizations because it is flexible, cheap, and reliable.
Amazon EC2 is highly elastic, you can change your capacity needs within a few minutes to
scale your application needs. It will save your time and speed up the development process
because you will not have to wait for hours for the server to adjust to the capacity needs. If
you are using EC2 service, it can provide data storage and other Amazon services for virtual
computing. So you will have a good range of tools and servers ready to use in order to make
application development easier. Amazon can run Microsoft Windows Server 2003 and is a
way to deploy applications using the Web Platforms such as ASP.NET, Internet Information
Server (IIS) and many more. EC2 also supports SQL Server Standard and SQL Server
Express and makes those available to customers on an hourly basis. EC2 also allows you to
run the windows-based applications on the Amazon cloud computing platform. This can be
web-sites, data processing, or any other application which requires Windows software. EC2
and other computing services offer fail safe platforms to protect the applications of their
customers. They have redundant data centers and can host the applications on many servers,
and if a server breaks down, another server can come in and run the application. This process
reduces the data loss and eliminates the downtimes of the server. Also, cloud computing is
more secure as compared to the local servers because of the strong firewall technologies
implemented by the web service vendors such as Amazon. These technologies protect the data
from hacking and the attacks that come from the Internet.
15
Amazon provides a simpleDB for database services to store data for its customers. It
provides basic and important functions of the data indexing and querying. This database
service works closely with Amazon EC2 and Amazon Simple Storage Service (Amazon S3).
Amazon S3 storage service can be used by developers and online organizations to get web-
based solutions for their needs of storing data. This provides the ability to store, process, and
query the data sets in the cloud. The traditional relational database requires a considerable
upfront expense and also the design is too complex to understand and requires a database
administrator. Amazon S3 is very important and useful because it is very simple to use and
requires no schema, automatically indexes data, and provides a simple API for storage and
access. With S3, this process becomes easier to manage and eliminates the administrative load
of data modeling, performance tuning, and index maintenance. Amazon’s data storage
facilities are reliable and S3 outages are very rare and usually last for only few minutes.
Amazon S3 utilizes a simple web service to store and retrieve any amount of data from
anywhere on the web. This type of utility gives developers a chance to access the same data
storage infrastructure that Amazon uses to run its applications. Moreover, on a S3 server your
data would be protected because its security features are very strong and can be relied on to
protect the integrity of your data. Lastly, Amazon S3 is cheaper and only costs for the space
and bandwidth used.
Amazon S3 can also be used as a content delivery network (CDN) [21]. A content
delivery network or content distribution network (CDN) is a system of computers containing
copies of data placed at various nodes of a network. This system is used to achieve improved
performance, a wider scale of delivery and reduced cost of delivery. All these benefits are
useful for the customers. Amazon S3 is one of the few Amazon web services that can be
deployed to work with a group of connected computer systems in order to perform the
delivery of content to all the people that are getting service from them. Very few companies in
the market are using this content delivery network due to the lack of information about its
usage and benefits. However, the number of companies using this technology will grow
eventually because of the rise in the demand for the effective delivery of contents.
There are numerous benefits of using cloud computing on Large Scale Web
Applications. Software designed for the commercial development of large users usually
belongs to the category of Large Scale Web Applications. These types of applications will
16
need a large amount of server space and disk space. They also require powerful CPU
performance and fast memories. Large scale applications would also require purchasing a lot
of new hardware and upgrading the software for their development. This could cost a lot and
might be an obvious reason not to start with such applications. But with cloud computing, it
becomes so easy to start large scale web applications without worrying about the new
hardware. You can rent/lease thousands or any number of computers required for the
development of applications. Also you can benefit from the huge data storage and fast servers,
with minimal overhead cost, if you are developing your application through cloud computing.
There are virtual infrastructures available on clouds which are used to host the applications on
the web. So, a company doesn’t need to buy expensive servers to host their applications on the
internet. Cloud computing makes large scale web application development successful and
saves a lot of money which promotes the growth because of improved profitability.
The Amazon EC2 provides scalability, but the management of EC2 requires a lot of
supervision from personnel. It can be labor demanding in terms of making sure that high
traffic levels are served and other changes are in tune accordingly. Website failures or
application crashes could happen, so personnel are needed to control and look into these
matters to run the operations smoothly. For easier EC2 management there are various
extensions and applications offered such as ElasticFox. EC2 users are assisted by this
extension which allows them to perform basic tasks of EC2 management. By setting up the
ElasticFox, EC2 users can assign IP addresses, spawn instances and open up ports. There are
also applications that can be used alternatively to allow automated EC2 management. Such
applications provide service to EC2 users by scaling automatically their systems depending
upon the usage. Sudden growth and spikes can be easily handled by the use of these
applications. EC2 management becomes more efficient and the need for personnel to handle
all these operations is minimized because problems and adjustments are handled gracefully.
Due to the unstable IP environments, there is always a temporary loss of websites but
with the more stable IP address environment these problems can be easily solved and websites
can be revived again within a few seconds. These applications are very efficient and many
EC2 users benefit from them. Depending upon the user’s requirement, it will always be an
option to manage the EC2 manually or use the extensions and applications. If the scale of
operation is big then it is always better to use EC2 management applications to get efficient
17
results. EC2 users also want to have easier management administration. EC2 allows flexibility
in order to add more users and capacity depending upon the usage, in short, scalability. These
actions require good EC2 administration. In order to get increased capacity and cut down on
costs, EC2 users need to balance loads very efficiently and smartly. Again with dynamic IPs
assigned to run the websites and downtime problems, EC2 automation applications have been
developed for load balancing. These EC2 applications will do everything for the users, and
scaling capacity will be done automatically, therefore, diminishing the need for the
administrator.
With EC2 automation, most of the problems are solved, and it becomes a more
powerful and more robust product to serve the needs of the customers. With these automated
applications, the IP environment becomes more stable and downtime problems are solved as
uptime is just a few seconds. Manual management and administration has been replaced by
the efficient and reliable extensions and applications. It certainly becomes a most popular
product nowadays.
In our product, we have used Amazon EC2 for the creation of images, and we will
briefly describe the importance of creating EC2 images. Amazon Machine Image or AMI is a
special type of encrypted file that is stored in S3 [2] of Amazon and used to instantiate a
virtual machine within the Amazon EC2. AMI contains all the essential information required
in booting the instance of your application or software. The instance is nothing but a running
system based on the AMI. Amazon EC2 provides different types of instances and the
flexibility to choose among them based on the requirements; each instance provides different
computing capacity and is charged on a per-hour basis. The instances with the identical AMI
are also executed in the same manner, so any information that might be lost will be
terminated. Hence, to avoid instances of termination and failures, creating EC2 images are
very important. There is a large amount of EC2 images. There are very steep chances of
finding the instances as there are lots of images available. Every EC2 image has a unique
identity which is helpful in finding the image once booting is done. Amazon assigns an API
called REST which provides each instance an identity and all the information about itself.
You can create new AMI either by creating a new one from scratch or modifying and
extending the ones which are already available and very close to what you are looking for.
The steps which you need to keep in mind are the modification of the existing image based
18
on the original image, preparing for bundling once the system has been modified
successfully, and creating a snapshot which needs to be packaged into the AMI by using the
ec2-bundle-vol utility. This utility encrypts and protects the image from any tampering or
damage such that only you and Amazon EC2 can decrypt it. Bundling the image requires
uploading the certificate. To access later you have to use your AWS [14] account ID as your
username. The AMI is then uploaded directly to the S3 or Amazon storage. The uploaded
image will be used in signing up for Amazon’s S3. This upload process can be lengthy, but
you will get continuous feedback about the upload till the time the image is successfully
uploaded. Registration of AMI is also necessary so that you can trace the image in order to
run the instance later on. You can now finally run the instance of the modified AMI by
specifying the image identifier you received at the time of image registration. You can
monitor the status of the image until it’s ready and then connect the image to your browser to
confirm the changes you have made to the original image.
19
CHAPTER 3
METHODOLOGYAND DESIGN
INTRODUCTION
The motivation of this study is to explore the idea of an on demand, scalable,
automated framework that automates the setup of development environments for the software
industry. Public and Private Virtualization solutions such as Amazon’s EC2 or Eucalyptus
lay the foundation for such an environment to be on demand and scalable. Through the study
of various research papers and articles, we have realized that there are very few such
solutions in the market today. This chapter is dedicated to describing the methodology and
design of our product and its various components in order to illustrate how this idea could be
useful for the software industry in the future.
My part was to engineer a push button and administration interface in the form of a
java plugin into an issue tracking system. The push button interface will spawn an image by
the click on a single button through an issue screen of the issue tracking system. The
administration interface provides user access to the push button interface. One time mapping
of projects to configuration and automation scripts, source repository location and package
repository location are all part of the functionality that is provided by the administration
interface. I have used JIRA as a bug tracking system for the implementation of the study. To
understand completely the design and methodology of the product, I will explain the JIRA
Plugin and what software is required to begin the plugin writing.
JIRAPLUGIN OVERVIEW AND SOFTWARE
REQUIREMENTS
JIRA is an extendable issue tracking system built by Atlassian. It is used by software
engineering firms to keep track of requirements and bugs. JIRA can be extended by
developing java plugins. These plugins are simply jar files which contain all the codes,
configuration files and resources that can be installed into the Atlassian product to add new
features or functionality into it or to change the behavior of the existing system. There are
two types of Atlassian plugins: First is “Version 1” plugin which is compatible with the old
20
plugin framework, and was used in JIRA prior to version 4.0; it is also compatible with the
new plugin framework of JIRA version 4.0 or later. The installation directory of “Version 1”
plugin is atlassian-jira/WEB-INF/lib/ and it must be installed and uninstalled manually. The
second plugin is “Version 2” which is compatible with the new plugin framework which is
based on the open social java framework applicable to JIRA version 4.0 and later. It is
installed automatically using the JIRA administration interface. Also the version 2 plugin is
installed into the JIRA home directory under plugins/installed-plugins/. ‘Version 2’ plugins
are deployed into the internal OSGi container which provides a service-oriented, component-
based environment for developers and offers standardized ways to manage the software
lifecycle. These capabilities greatly increase the value of a wide range of computers and
devices that use the Java platform[12]. They isolates the plugin from the core code and from
the other plugins, it also allows dependency management. Also, Version 2 plugins have to be
specifically declared by defining one of their attributes as plugin-version=”2” in plugin’s
configuration file.
To set up the environment for the development of plugins, a java development kit is
required because Atlassian applications support JDK 6. We need to download and install
JDK 6 and set the JAVA_HOME environment variable. We even need to install Atlassian
plugin SDK; this SDK includes Maven with a well-configured settings.xml file and also
some scripts which are useful to speed up and simplify the plugin development. After that,
we have to add the SDK bin’s directory to the path environment variable. The next step is to
decide on the code development tool because we need one that supports Maven 2. Atlassian
Plugin development process depends heavenly on Maven, so we have to make sure our IDE
supports that. After considering all the above mentioned criteria, we have decided to use
Eclipse for the plugin development process.
Writing your own plugin will require the developer to call or use JIRA code to
accomplish certain tasks; for example, to retrieve the list of projects, to get a list of users and
add new data to the issues or make changes in workflow. Understanding the JIRA
architecture and its functionality is another important thing to be completed before beginning
the plugin development. Customizing the JIRA sometimes requires modifications in the
code. There are some default classes in JIRA, which conform to an interface like Manager
classes or the Webwork action classes, so modifying the classes and then plugging modified
21
classes into JIRA can be made easy by writing your own implementation of JIRA interface. It
is always a problem to integrate modified classes into an existing system. In this section, I
will describe methods to use JIRA resources and functionality into our own implementation
to address this problem.
PICOCONTAINER AND DEPENDENCY INJECTION
Software components depend on each other; for example, component X depends on
component Y, so for X to successfully complete its task, the correct execution of Y is very
important. Dependency Injection [13] (DI) is where components are given their dependencies
through their constructors, methods, or directly into fields. Now there are some cases where
components depend on each other rather than the normal way in source code where various
references are made to other components. Component interdependency makes it harder to
modify the components because no one can predict what will be the effects of the changes on
depending components. Also, maintaining a larger project becomes a problem because fixing
these dependency problems invites new problems in other parts of the software product.
Lastly, it makes unit testing possible by keeping the instantiation of objects outside separate
from other objects. That is why it is always recommended to use Dependency Injections in
your applications. JIRA uses PicoContainer as it is a lightweight dependency injection
framework. One of the important features of the PicoContainer is its ability to instantiate
arbitrary objects and resolves their constructor dependencies. This is accomplished by using
its API, which works very similar to a hash table. PicoContainer is very useful when there are
larger numbers of classes and interfaces with complex dependencies between each other.
JIRA uses PicoContainer [10] to manage object creation throughout the system or as a central
object factory. There are different types of dependency injections that are supported by
PicoContainer. One of the recommended methods of injection is through the constructor
where the constructor of a class is used to pass dependencies into it. In Table 1 are some of
the regular types and their descriptions.
Understanding the concept of dependencies injection is very necessary in order to
resolve the uninvited conflicts between the classes and interfaces while plugging the
modified code into it. Using the PicoContainer in JIRA plugin development as a DI is very
22
Table 1. Types of Dependency Injection
Regular Types Description
Setter Multiple Setter methods on the class are used to pass dependencies in.
Annotated
Method
Methods are marked with an annotation to indicate that they
should be used for injecting dependencies
Field Fields are injected into for dependencies
Annotated Field Fields are marked with an annotation to indicate that they should be
injected into
Named Field Fields of a certain name should be injected into
helpful in simplifying the code, as any PicoContainer instantiated object of class X can
obtain an instance of another class Y by simply requesting one in its construction or by any
other means mentioned in above table. Construction injection is one of the best DI
implementation approaches in the PicoContainer.
JIRAINTERACTION WITH DATABASE
JIRA uses Entity-Engine module of OfBiz [6] (Open for Business Entity Engine) to
communicate with the database. OfBiz is a set of tools used to model and manage entity
specific data. An entity is defined by a set of fields and relations to other entities; it is a piece
of data managed by certain rules defined in xml. The entity engine uses various patterns such
as Business Tier patterns and Integration patterns which include: Business Delegates or
normal Delegates, Value Objects (VO), Data Access Object (DAO), and Service Locator.
The main purpose behind using an entity engine is to reduce the requirement of persistence
code related to entity from as many layers as possible. This kind of framework will make a
sort of abstraction that saves a lot of development effort. It does that by reducing the
persistence related issues in the system while developing the project. Any change in one
piece of code will not affect the rest of the code because of the layered architecture which
separates the business logic from the presentation logic.
The Entity Engine provides all the utilities and tools necessary to securely access an
unlimited number of databases regardless of their physical location, as shown in the
Figure 2. OfBiz framework
The Entity Engine provides OfBiz applications with all the utilities, tools, and an
application programming interface (API) to easily read and write data to all the configured
data sources in a consistent manner without the
or for the database connection. As shown in the above figure, a remote database can also be
connected to the Ofbiz engine. A remote database can be network connected or installed on a
local server. Remote database requ
In JIRA, the database schema is described in the entitymodel.xml file under WEB
INF/classes/entitydefs directory. This entitymodel.xml file contains the XML definition of all
the JIRA database tables, table columns, and their data types.
the tables is also found in this file. JIRA database can be accessed through APIs or through
jdbc connections.
The Entity Engine provides all the utilities and tools necessary to securely access an
unlimited number of databases regardless of their physical location, as shown in the
ramework.
ngine provides OfBiz applications with all the utilities, tools, and an
application programming interface (API) to easily read and write data to all the configured
data sources in a consistent manner without the limitation of the physical location of the data
or for the database connection. As shown in the above figure, a remote database can also be
connected to the Ofbiz engine. A remote database can be network connected or installed on a
local server. Remote database requires a datasource entry in the entityengine.xml file.
In JIRA, the database schema is described in the entitymodel.xml file under WEB
INF/classes/entitydefs directory. This entitymodel.xml file contains the XML definition of all
table columns, and their data types. Relationships
the tables is also found in this file. JIRA database can be accessed through APIs or through
23
The Entity Engine provides all the utilities and tools necessary to securely access an
unlimited number of databases regardless of their physical location, as shown in the Figure 2.
ngine provides OfBiz applications with all the utilities, tools, and an
application programming interface (API) to easily read and write data to all the configured
physical location of the data
or for the database connection. As shown in the above figure, a remote database can also be
connected to the Ofbiz engine. A remote database can be network connected or installed on a
ires a datasource entry in the entityengine.xml file.
In JIRA, the database schema is described in the entitymodel.xml file under WEB-
INF/classes/entitydefs directory. This entitymodel.xml file contains the XML definition of all
entry between
the tables is also found in this file. JIRA database can be accessed through APIs or through
24
Here is an example of one of the entries of the tables in the entitymodel.xml file.
<entity entity-name="TrackbackPing" table-name="trackback_ping" package-name="">
<field name="id" type="numeric"/>
<field name="issue" type="numeric"/>
<field name="url" type="long-varchar"/>
<field name="title" type="long-varchar"/>
<field name="blogname" type="long-varchar"/>
<field name="excerpt" type="long-varchar"/>
<field name="created" type="date-time"/>
<prim-key field="id"/>
<relation type="one" title="Parent" rel-entity-name="Issue">
<key-map field-name="issue" rel-field-name="id"/>
</relation>
</entity>
Each table is treated as a single entity and field names and their data types can be
defined as showed above, along with those relationships between the entities. There are many
ways to access the JIRA database while creating plugins. There are JIRA APIs which can be
used to access the database. If APIs are used, then a lot of code deals with GenericValue
objects. Each GenericValue object represents the record in the database; it is also an object of
the OfBiz entity engine. We can always edit the entitymodel.xml and put that under the
different directory such as the webapp/WEB-INF/classes/entitydefs/entitymodel.xml and
create an entry of the table required in the plugin development.
The functionality of the OfBiz framework can be divided into the following distinct
layers:
Presentation Layer
OfBiz uses the concepts of screens to represent the pages. A page in OfBiz consists of
components which can be header, footer, etc., and all these components have to be put into a
single directory. These components can be Java Server Pages (JSPs), Forms and Widgets and
are a part of framework OfBiz components, responsible for rendering the screen, accessing
25
the database, caching, managing transactions and several low level tasks used by the
application components.
Business Layer
The business layer or application layer defines services provided to the users. The
service engine as shown in the figure above is responsible for the invocation and transaction.
OfBiz uses several types of services such as SOAP, Java methods, and workflows.
Data Layer
The data layer is responsible for the database access, storage and providing a common
interface to the application layer. Data is accessed in such a way that each entity is provided
to the business layer as a set of generic values. Generic values are not type specific, so fields
of the entity are accessed by column name.
DEVELOPMENT PHASE
We will now explain the whole development phase, starting with the skeleton of the
plugin. First of all, we need to make a new directory in which the plugin would be created.
To do that, we need to open a command window and navigate to the newly created folder.
On the command prompt, we have to then run the command “atlas-create-APPLICATION-
plugin,” where APPLICATION is the Atlassian application for which we are developing our
plugin. In this case, we are working with the JIRA product from Atlassian, hence we have to
use the command “atlas-create-jira-plugin.” It will prompt for the following values: groupID,
artifactId, version and package. This process will involve creating a lot of directories and
subdirectories, XML files and java files. While creating the plugin skeleton, Maven will
download all the dependencies from the Internet. It will create a new directory structure as
shown in Figure 3.
The following describes the files and directories in the JIRA plugin directory:
 The pom.xml will be created in the root folder of the plugin. POM [1] stands for
Project Object Model definition file.
 Directories for the source code files will be in the “src/main” directory, and the test
source code files will be in “src/test” directory. All the java files will go into the
directory “src/main.”
 Directories for the templates, javascript, css, images, etc., will be under resources.
26
Figure 3. Example of pom.xml.
 There is a package under test called “it,” which is for integration testing.
An atlassian-plugin.xml file in “src/main/resources” directory is the plugin descriptor
file; this descriptor tells the application all the plugins and the modules within it and
is located in the root of the plugin’s jar file. Here is the sample of atlassian-
plugin.xml as an example below.
<atlassian-plugin key="${project.groupId}.${project.artifactId}"
name="${project.name}" plugins-version="2">
<plugin-info>
<description>${project.description}</description>
<version>${project.version}</version>
<vendor name="${project.organization.name}" url="${project.organization.url}"
/>
</plugin-info>
</atlassian-plugin>
27
“Atlassian-plugin” element is the root element for the plugin descriptor. The key must be
present and is used to identify the plugin uniquely and its name is used to display the plugin
in the menus. The plugin version is to notify that version of plugin which we are using in our
application. As explained earlier in this chapter, if we are using ‘version 2’ plugins, then it
has to be explicitly mentioned in the code as above. The plugin-info block inside the
atlassian-plugin is used to provide more information about the plugin. Description block is
used for the details about the plugin, which describes what the plugin has inside it. Version is
displayed in the application’s plugin manager and “vendor” supplies the information about
the developer of the plugin.
The Project Object Model, pom.xml also plays a very important role by describing
everything about the dependencies, deployment location, tests, author, and source control
that Maven needs to know about the plugin. After successfully building the skeleton, we have
to go to the command prompt and navigate to the folder that contains pom.xml and run the
command “atlas-run.” This command will be slow if running for the first time, but after
completion, we can access our JIRA plugin through the browser with the URL mentioned in
the command prompt.
VELOCITY OVERVIEW
User interacts with JIRA using the web browser. It is a web application and uses
OpenSymphony’s WebWork1 [8]. WebWork1 is a MVC framework similar to struts. When
an HTTP request is made by a browser at the client-side for a URL, the request is passed to
the JIRA web application which returns the HTML. These HTML pages can be static pages
or dynamic pages generated at the time of request. JIRA uses JSP for the presentation layer.
A lot of business logic in JIRA is implemented in many java files that are present in various
places inside the hierarchy of the plugin. We have used a velocity template as a presentation
layer while implementing the JIRA plugins.
Velocity is a template language which is used to present dynamic information in an
easy way to the users of a web site or an application. A collection of java based classes is
used to create a separation between the model and view components of the Model-View-
Controller (MVC) model. One of the attractive features of velocity is the clear separation
between the view classes and the rest of the code with only a simple set of syntax. The output
28
yield by the velocity can be HTML or any other content, like SQL, XML, or source code.
Velocity also supports servlets directly and has access to java object methods. It can easily
transform XML to any other content with the help of Anakia which is an application that
allows XML to be processed using velocity instead of XSL (Extensible Stylesheet
Language). There are many more features of velocity: It uses the log4J system for logging.
Velocity macros allow the building of macros for HTML and scripting elements which are
very useful from the developer point of view. It even supports all the major application
servers and servlets.
The obvious reason of choosing velocity over the JSP in our code implementation is
that JSP files are converted in java files, which are compiled to java class files and reside
inside the classes folder of the JIRA structure. If there are any changes to be done in the
JIRA we have modify or add new JSP files to that class folder. This would need a
recompilation of JSP files which would lead to starting up JIRA again and again. Also, if
there is any JIRA upgrade to the system, then all the changes are lost and we have to repeat
the same steps again to make our code work with the new version of JIRA. On the other
hand, velocity templates are passed to a velocity engine code for evaluation and there is no
need of restarting JIRA to check the affect after changing anything to the velocity template.
Once the plugin is ready, we make a jar file and deploy it to any system without any changes.
But if there are some modified JSP files then we have to put those files into a particular
directory with the jar file. With the velocity template as a presentation layer, we do not need
to do that extra step. Velocity is also much lot easier to learn and implement due to its simple
syntax, and it’s even faster than JSP. As we are using xml files to map the action files with
the views, the velocity template is best suited to be used with xml. Moreover, using JSP is
troublesome because everything is statically typed and working with very a big project with
lots of custom tags, which causes the development deployment cycle to slow down
considerably, and it also results in inefficiently resolving dependencies.
Both JSP and velocity templates have their own advantages and disadvantages. So, if
we look from the JIRA plugin development side, we need something which is portable and
does not change if there are any upgrades in the JIRA system. With the advent of new
technologies and newer versions coming out at a rapid speed, we need something stable for
our customers, so the velocity template is the right choice for these kinds of plugins.
Here is the procedure
actual implementation was started. The main goal is to make
avoid any mistakes in the development
simple to use and intuitive for the user. The use of a link in the operations
issue screen should suffice our need. This push button is linked with the Admin Screen and
can be seen in Figure 4. Once this button is clicked, the action will spawn the image, install
all packages and send out an email to the user wi
manual steps that are still required to complete the setup, ssh key and
role of the image.
Figure 4. Mock up diagram.
MOCKUP DESCRIPTION
procedure of the user interaction of the project that was designed before
ctual implementation was started. The main goal is to make the design clear
avoid any mistakes in the development stage. The push button interface should be extremely
simple to use and intuitive for the user. The use of a link in the operations section of
issue screen should suffice our need. This push button is linked with the Admin Screen and
. Once this button is clicked, the action will spawn the image, install
all packages and send out an email to the user with information on how to log
manual steps that are still required to complete the setup, ssh key and the Properties of the
Figure 4. Mock up diagram.
29
of the user interaction of the project that was designed before
design clear in order to
. The push button interface should be extremely
section of the JIRA
issue screen should suffice our need. This push button is linked with the Admin Screen and
. Once this button is clicked, the action will spawn the image, install
th information on how to log in, display the
Properties of the
30
FUNCTIONALITY OF THE SYSTEM
Figure 5 is a screen shot of the Admin screen which has a link called “Nimbo Admin
Screen” under the global settings. On clicking that link, the Admin screen will be opened and
the user can enter relevant information as explained above. This screen is an extension of the
JIRA system which gives the user an option of editing and deleting the information. If an
administrator of a bug tracking system wants to configure an application to be spawned using
EC2, they would need to configure the bug tracking system by entering the relevant
information for that project in the screen shown below. This can be done by defining a screen
in the bug tracking system that allows the configuration of Nimbo. All the components of this
project were created in java in the Eclipse IDE; however, to write a JIRA plugin, one should
have a specific development environment as explained above.
Figure 5. Screen shot of admin screen.
We need to log in to the JIRA bug tracking system with the particular credentials of
the user. Next, select the issue to work on, which could be a bug fix, some task to be
completed or anything related to the particular project. In JIRA, issues are linked with
projects. To fix the bug or the problem, the developer needs a system to work on; on issue
screen there will be a Launch button which will spawn an image on demand to the Amazon
EC2 server with all the application related software on it. This Launch button is interlinked
31
with the Admin screen under the administrator section on the JIRA web application. The
Admin screen will have the information of all the projects configured in the JIRA system. So,
each project can be mapped to a single role (mysql, LAMP, etc.) from that particular screen.
The administrator needs the ability to define the type of version control mechanism used and
repository location for the applications that will be used in the image and map it to a
project/component. The role of an image is a script that is in a revision control mechanism.
The administrator wants to put the url into the source control mechanism where these
cookbooks are found. All this information will be stored in the table in the JIRA database.
Now every issue related to that particular project which has the “NIMBO” functionality
enabled will have the “LAUNCH NIMBO” button as shown in Figure 6.
Figure 6. Screen shot of issue screen with launch button.
An engineer is looking through a bug record in an issue tracking system. He/she
decides to work on this issue but needs to set up a “production like” system. The engineer
does that by clicking the “Launch Nimbo” button on the issue screen. Once that is done, the
issue key and username of the user that clicked the launch button is sent to a server side Java
32
program which creates the image on Amazon EC2. To create the image on EC2, we have
used the Amazon Web Services (AWS) Software Development Kit (SDK), a set of developer
tools which can be integrated into Java based programs which can then directly talk to
Amazon Web Services Frameworks on their servers. The tools in the SDK have the
capability of controlling almost every aspect of the image creation process, some of which
are the zone which would be used to create the image, the image ID that is to be used for
image creation and the keyname of the key that would be used to connect to the machine.
The image ID is the ID of the bare bones skeleton which should be configured exactly like
the machine that will be created by the AWS SDK tools later on by the user. Once the
machine is an exact copy of the image required, there is a set of tools from Amazon called
Amazon Machine Identifier (AMI) tools which create the files that will be used by Amazon
to create an image. The files are then bundled together in an Amazon Bundle, and the bundle
is then uploaded to Amazon S3 servers with a particular bundle or bucket name. Once the
files are uploaded to a bucket, these set of files can then be registered by the Amazon API
tools and are issued an image ID which can be either a public or private ID. If the user
chooses to create a private Image ID, the user is restricting the image creation from the image
ID only to the users which use the same credentials that were used to make the image in the
first step. If a public image ID is used by users, the user can contribute this image to the vast
amount of an already public repository of images so that other users don’t have to worry
about creating images, and can use this image ID directly for creating a machine that can be
used for their purposes.
Eclipse is one of the most popular Integrated Development Environments (IDE) for
creating Java based programs and is used extensively by Java programmers. As a result,
Amazon Provides the AWS toolkit for Eclipse, which gives users access to AWS SDK for
Java, AWS Elastic Benastalk Deployment tools, Amazon SimpleDB Management tools, and
Amazon EC2 Management Tools. AWS SDK for Java is a set of libraries which can be
integrated into the existing Java projects of different users, and the users can use these
libraries to gain access to different classes which can control all the details of the Image
creation process which would be explained later in the chapter. AWS Beanstalk
Development tools are another set of libraries which can deploy Java based web applications
to be deployed to the rigorously tested and stable Beanstalk Development framework
33
provided by Amazon. Once a web based application is deployed to the AWS Elastic
Beanstalk environment, this framework handles all the deployment details so that the
programmers of the application can optimize the application to increase user satisfaction and
deduce ways to prevent service attacks on the application. These deployment details include:
load balancing of application; auto-scaling which can control different resources available to
the application like reducing or increasing the amount of bandwidth available to the
application depending upon the amount of requests coming in from different users; capacity
provisioning; and active application health monitoring which gives users access to look at
different metrics like number of user requests, disk space used, and bandwidth allocated to
the application at different times.
When a user or company signs up to use Amazon EC2 services, each user assigned is
given some AWS Credentials. The two important properties in these credentials are the
access key and the secret key associated with that account. For using the AWS SDK and all
its associated libraries, it is necessary to define the above-mentioned credentials in a file
called “AwsCredentials.properties” and then include this file in the root of the Java project.
We use the “AwsCredentials” class to instantiate an object (let’s call this object
“credentials”) that is given the above-mentioned file “AwsCredentials.properties.” as an
input. Now, when we call our Java project, the credentials in this object is authenticated by
the Amazon framework, and after the successful authentication, this object now becomes a
direct connection to the Amazon framework. We use another class called “AmazonEC2” and
instantiate its object, which in our case is “EC2.” This “EC2” object is then made the
Amazon EC2 client object by using the “AmazonEC2Client” function of the above
mentioned “AmazonEC2” class. Now the “EC2” object can be used to control all the
subsequent image creation activities. In our project, we have made a file called
“ImageCreation.properties” which is used to store the Image ID of the image that has to be
created on Amazon EC2, Keyname of the key that would be used to connect to that machine
by using the ssh client, and the interval of time that the server side Java program should wait
after issuing the command to create the image. After which this file should call another
server side program that connects to this new machine and then installs the software defined
in the role associated with the project by the administrator of the company. Once we read all
the above-mentioned properties from the “ImageCreation.properties,” we use the
34
“RunInstancesRequest” class to instantiate an object that associates the Image ID and the key
name read above with the “EC2” object. We then call the “runInstances” function of the
“AmazonEC2” class to create a new machine on the Amazon EC2 infrastructure using the
above mentioned image ID and key name. The “runinstanceResult” of the
“RunInstancesRequest” class then gives us the result of the image creation process. Once the
image creation request comes into the Amazon EC2 framework, it assigns an instance ID to
this request and then begins with the image creation process. AWS SDK gives us another
class called “Reservation” which queries the Amazon EC2 framework during this instance ID
assignment process, and once an instance ID is assigned to the incoming request, it provides
access to the value of this instance ID.
Once the image creation request is given to the Amazon EC2 framework, we have to
query the framework about the state of the image creation process, so that we can extract the
details like hostname, instance ID, and availability zone, which are details associated with the
newly created image. The AWS SDK provides the user with a class
“DescribeInstancesRequest” which has to instantiate an object and then run the
“setInstanceIds” function which associates the instance ID of the newly created machine with
this object. The “EC2” Amazon client object referred to above then calls the
“describeInstances” function of the Amazon EC2 class to find out all the above mentioned
machine details like hostname, zone, etc., which are used in the email that is sent to the user
who originally clicked the “Launch Nimbo” button.
Once all the above mentioned actions are done, this Java program then calls another
server side Java wrapper program that talks to the JIRA server to fetch the Nimbo related
attributes of the project and then uses all these attributes to install the software in the newly
created machine, and send an email to the user who clicked the button to create the image.
All the above mentioned actions of this server side Java program are logged in a file
called “NimboJiraPluginLog.txt.” This file gives us the output of every step in the above
mentioned program; for example, it gives out the image ID and the keyname read from the
“ImageCreation.properties” file. If that file does not exist, it gives us the output of the failure
to read the file. It then gives us details of the user that clicked the “Launch Nimbo” button by
giving out the username and the issue key in the log file. Once we enter the AWS SDK
portion of the image creation, it displays the big banner of entering that stage. In case we
35
cannot initialize the SDK, it even gives out the error report for that. Every function
associated with the image creation mentioned in the above procedure also logs all its output
in the same file. The final output then mentions the hostname and other attributes associated
with the newly created machine, and then the last line to call is the other server side wrapper
program to install the software. All these log messages will be displayed in Chapter 4, and
every line will be explained in detail in that chapter.
An image has been started via the click of a button on a web interface. Once the
image is fully configured and running, the engineer will receive an email with the IP address
and/or name of the machine, along with the location of the ssh key link to download the key
of the file server. A clear set of steps will be included in the email communication on how to
log in with which utilities. The notification email needs to list the additional manual steps
required to fully configure the development and or test environment. The engineer will
receive an email after the start up of the image, about the name of the image, Operating
system, Version, Architecture, and a list of dependent installed packages and applications.
In the next chapter we will show the results of the implementation and their
importance in the context of the users and developers.
36
CHAPTER 4
RESULTS
The main objective of this project was to give a client company using the Nimbo
product the independence to create different kinds of machines automatically without the
involvement of any IT department personnel. Once the machine is created, all the software
required on that machine would also be installed automatically. Users would get a really
powerful machine with a huge amount of resources within a short span of time. Also, a large
amount of such machines could be created at the same time, giving that company access to
resources beyond their current hardware and software budget resource. A team of developers
working on different issues can create “production like” machines that would be used to
reproduce the issues that they are trying to solve and once that is done, find the solution by
changing either the configuration or their codebase to find the resolution to the problem.
This project starts with a user (normally a developer) logging into a JIRA bug
tracking system to work on a particular issue. The administrator of the JIRA system ensures
that all the user properties like username, email address, First Name, and Last Name are all
set before he or she is issued a username and password. The administrator also makes sure
the project on which this user is working is mapped to a particular role and all the relevant
Nimbo related attributes mentioned above all correctly filled. Unless all the Nimbo related
attributes are populated for a project, the project is not “Nimbo enabled” and the developers
of the project don’t get a “Launch Nimbo” button on the issue screen.
In addition to the user working on projects, we have made another screen in which the
administrator logs in to set all the Nimbo related attributes. The screen is called “Admin
Nimbo Screen” and can be found under the “Administration” tab of the JIRA bug tracking
system. This screen is only visible to users with the “JIRAadministrators” group. Once the
administrator logs in, he or she can set these attributes according to the requirements of the
project. The administrator would have to talk to the Nimbo programmer to decide the kind of
image that needs to be created on the Amazon EC2, so that the Nimbo programmer can make
a bare bones skeleton image and give the administrator the image ID for that particular role.
37
In addition, the administrator needs to set the APT repository and the cookbook repository
location, so that the cookbooks for that role can be checked out once the image is created on
the Amazon EC2. The users also have an option to install their custom software by giving the
location of the apt repo. The administrator can provide the installers with their custom
software by making the apt specific installers and then giving the location in this field.
Figure 7 is a screenshot of the Nimbo Admin Screen:
Figure 7. Screen shot of add admin screen.
All the projects that do not have the Nimbo functionality enabled are present in the
dropdown of the Projects category. We have the validation enabled in the above form in
which the different fields are checked to make sure that none of them have a blank value. In
addition, we check the locations for the apt and cookbook repos to make sure that they have a
valid web address value.
Now we go through each of the Nimbo related attributes and make sure that we go
through all the validations. If the Role is given no value, we get the following validation error
in Figure 8 which tells us that the role cannot be blank:
38
Figure 8. Screen shot of role validation.
We even check the attributes to make sure that the apt repo should have a valid web
address. We do that by making sure that the apt repo value should begin with an http or an
https value. If we do not do that, we get the following error in Figure 9:
Figure 9. Screen shot of APT repo validation.
We do the same validation for the value of the cookbook repo and make sure it also
has a value with a valid Internet address beginning with an http or https. If we do not do that,
we get the following validation error in Figure 10:
Figure 10. Screen shot of cookbook repo validations.
In addition to the initial entering of values by the administrator, we can edit the roles
by using the “Edit” button on the right side of the role. Once we click that button, we go to
the “Nimbo Edit” screen, which is shown in Figure 11 below:
39
Figure 11. Screen shot of edit admin screen.
All the above mentioned validations for the role, apt repo and cookbook repo
designated in the “Admin Nimbo Screen” also apply to the edit screen. We check for the
blank values and also valid Internet addresses for those attributes.
For deleting the roles, we can use the “Delete” option in front of the roles which takes
the user to the “Nimbo Delete” screen. Figure 12 is the screenshot for that screen:
Figure 12. Screen shot of delete admin screen.
For the image creation Server side wrapper Java program, we have used a log file to
list the steps of the entering and exiting stage of a function. In addition, we have used
40
exceptions to catch all the possible errors that could occur during program execution, all of
which are described in this section.
Below is the output of the log file after the user clicks the “Launch Nimbo” button:
DEBUG - Start NimboIssue
DEBUG - End NimboIssue
DEBUG - NimboIssue Start doExecute
DEBUG - Start Create Image With parameter
DEBUG - issukey : NMB-2
DEBUG - username : shilpa
DEBUG - NimboIssue End doExecute
DEBUG - Run the image creation
DEBUG - ---------[java, -jar, c:NIMBOImageCreation.jar, NMB-2, shilpa]
DEBUG - ---------Properties are
DEBUG - ---------imageid : ami-c0f906a9
DEBUG - ---------keyname : shilpa-ec2-keypair
DEBUG - ---------interval : 120000
DEBUG - ---------issukey : NMB-2
DEBUG - ---------username : shilpa
DEBUG - ---------===========================================
DEBUG - ---------Welcome to the AWS Java SDK!
DEBUG - ---------===========================================
DEBUG - ---------May 6, 2011 6:09:12 PM com.amazonaws.http.HttpClient execute
DEBUG - ---------INFO: Sending Request: POST https://ec2.amazonaws.com / Parameters:
(ImageId: ami-c0f906a9, Action: RunInstances, MinCount: 1, SignatureMethod:
HmacSHA256, AWSAccessKeyId: AKIAI3NLGY5ZWOMITILA, SignatureVersion: 2,
Version: 2011-02-28, Signature:
fKF+S/0QqPamdWQDnE7ymkGgRAKSA+16ZABsZ7Vnx5M=, Timestamp: 2011-05-
07T01:09:12.107Z, KeyName: shilpa-ec2-keypair, MaxCount: 1,)
DEBUG - ---------May 6, 2011 6:09:13 PM com.amazonaws.http.HttpClient handleResponse
DEBUG - ---------INFO: Received successful response: 200, AWS Request ID: 45de916c-
b7f4-4379-9309-02cc86cf64cb
41
DEBUG - ---------runinstanceResult : {Reservation: {ReservationId: r-4bfc2f27, OwnerId:
622506625911, RequesterId: null, Groups: [{GroupName: default, GroupId: sg-a011cac9, }],
GroupNames: [default], Instances: [{InstanceId: i-6e6e6101, ImageId: ami-c0f906a9, State:
{Code: 0, Name: pending, }, PrivateDnsName: , PublicDnsName: , StateTransitionReason: ,
KeyName: shilpa-ec2-keypair, AmiLaunchIndex: 0, ProductCodes: null, InstanceType:
m1.small, LaunchTime: Fri May 06 18:08:35 PDT 2011, Placement: {AvailabilityZone: us-
east-1a, GroupName: , Tenancy: default, }, KernelId: aki-6603f70f, RamdiskId: null,
Platform: null, Monitoring: {State: disabled, }, SubnetId: null, VpcId: null, PrivateIpAddress:
null, PublicIpAddress: null, StateReason: {Code: pending, Message: pending, },
Architecture: null, RootDeviceType: instance-store, RootDeviceName: null,
BlockDeviceMappings: null, VirtualizationType: null, InstanceLifecycle: null,
SpotInstanceRequestId: null, License: null, ClientToken: , Tags: null, SecurityGroups:
[{GroupName: default, GroupId: sg-a011cac9, }], SourceDestCheck: null, }], }, }
DEBUG - ---------New Instance ID: i-6e6e6101
DEBUG - ---------May 6, 2011 6:11:13 PM com.amazonaws.http.HttpClient execute
DEBUG - ---------INFO: Sending Request: POST https://ec2.amazonaws.com / Parameters:
(Action: DescribeInstances, SignatureMethod: HmacSHA256, InstanceId.1: i-6e6e6101,
AWSAccessKeyId: AKIAI3NLGY5ZWOMITILA, SignatureVersion: 2, Version: 2011-02-
28, Signature: 92bzskBuyEUYSQbOWxqqxbL0ymcNU9HjYu4HVNu7BLs=, Timestamp:
2011-05-07T01:11:13.793Z, InstanceId.2: i-40fdd02f, )
DEBUG - ---------May 6, 2011 6:11:14 PM com.amazonaws.http.HttpClient handleResponse
DEBUG - ---------INFO: Received successful response: 200, AWS Request ID: 241144fa-
1eba-4c95-93ec-3df44e24b8c8
DEBUG - ---------reservation : {ReservationId: r-4bfc2f27, OwnerId: 622506625911,
RequesterId: null, Groups: [{GroupName: default, GroupId: sg-a011cac9, }], GroupNames:
[default], Instances: [{InstanceId: i-6e6e6101, ImageId: ami-c0f906a9, State: {Code: 16,
Name: running, }, PrivateDnsName: ip-10-83-11-163.ec2.internal, PublicDnsName: ec2-50-
19-66-249.compute-1.amazonaws.com, StateTransitionReason: , KeyName: shilpa-ec2-
keypair, AmiLaunchIndex: 0, ProductCodes: null, InstanceType: m1.small, LaunchTime: Fri
May 06 18:08:35 PDT 2011, Placement: {AvailabilityZone: us-east-1a, GroupName: ,
Tenancy: default, }, KernelId: aki-6603f70f, RamdiskId: null, Platform: null, Monitoring:
42
{State: disabled, }, SubnetId: null, VpcId: null, PrivateIpAddress: 10.83.11.163,
PublicIpAddress: 50.19.66.249, StateReason: null, Architecture: i386, RootDeviceType:
instance-store, RootDeviceName: null, BlockDeviceMappings: null, VirtualizationType:
paravirtual, InstanceLifecycle: null, SpotInstanceRequestId: null, License: null, ClientToken:
, Tags: null, SecurityGroups: [{GroupName: default, GroupId: sg-a011cac9, }],
SourceDestCheck: null, }], }
DEBUG - ---------Host name : ec2-50-19-66-249.compute-1.amazonaws.com
DEBUG - ---------Instance Id : i-6e6e6101
DEBUG - ---------Start sending the issue key and username to server
DEBUG - ---------issukey : NMB-2
DEBUG - ---------username : shilpa
DEBUG - ---------hostname : ec2-50-19-66-249.compute-1.amazonaws.com
DEBUG - ---------instanceId : i-6e6e6101
DEBUG - ----wait over--- 0
Once the user logs in and assigns issues to him or her, the user gets a “Launch
Nimbo” button on the top left-hand corner of the issue screen. When that button is clicked,
the form calls the program NimboIssue.java, which gives us the output of executing the
“doExecute” function. This function calls the java program “CreateAmazon.java” which is
called with the value of the username of the user who clicked the “Launch Nimbo” button
and with the issue key of that issue screen on which the user is at that moment. In addition,
all the other properties like the image ID, keyname of the image, and the delay interval are
picked up from the ImageCreation.properties file. The delay interval controls the amount of
time needed to wait after issuing the command to make the image and get the hostname of
the newly generated machine. We can control these parameters and make them different for
different client requirements. The AWS SDK now creates the image and gives all the
relevant output until we get the instance ID associated with that image. All the functions
visited in the ImageCreation.java are explained in the above chapter. This is the main
program which creates the image on the Amazon EC2 using the AWS SDK and then gives
control to other programs which first retrieve all the parameters from the JIRA bug tracking
system, install software on the newly made machine and then finally send an email to the
user that clicked the “Launch Nimbo” button to make that image.
43
CHAPTER 5
CONCLUSION AND FUTURE WORK
By using Amazon EC2 and expanding the bug tracking system, it is conceivable to
automate the process of creating development and testing environments on demand by a
simple click of a button. Initially, we were not sure if this idea of automating the systems on
demand by the click of button would be possible. We spent a lot of time in discussing and
writing the design and requirements for the project. To come up with the tools and design is
the most time consuming process for any project and so it was in this project. After studying
various articles and research papers, we agreed on the technology needed to implement this
project. We chose JIRA as a bug tracking system and Amazon EC2 as our platform in order
to achieve a successful project. This project can be far reaching and very useful for any
software organization which has a team that wants to reduce the environment set-up time for
each developer and understands that the organization will save a significant amount of time.
Different people will have different requirements depending on the project they are working
on, so their need for a system as quickly as possible with all the software they need to start
with the project is different. Keeping this in mind there will be many use cases to implement,
to make this project successful for each individual. To start with, we have to focus on
implementing only one development/test environment for now.
On the JIRA front, we expanded the system and wrote a plugin which will store
project specific information into the database which will be helpful in identifying the
software requirements to install in the system. Each project is role specific which means one
project will have only one specific role and cannot be enrolled in multiple roles. Role is the
name of the environment which needs to be installed in the system on the user’s demand. We
have the URL information stored for cookbook repo’s and APT repo’s. This information will
be useful to check out the recipes from the cookbook and packages that need to be installed
from the APT repo and can be identified by the name of the role mapped with the project. All
these packages and software will then install in the system by a click on the button from the
project issue screen. The user can be a developer who wants to add new features in the
44
system or a tester who wants to fix the bugs. Our aim is to reduce the environment set-up
time so that any developer or tester can come in and by just a click of the button spawns an
image with the entire project specific software environment on it. Sometimes it takes a week
or two to set up an accurate development/testing environment for only a couple of hour’s
work, which is sometimes very frustrating and disappointing to the developer and team. Also,
having physical machines available immediately is also a problem and waiting for resources
to be available is a waste of time for the company and the team. The implementation of
automating the systems will help to overcome these kinds of problems which software
companies have.
We have designed our system with the limitation of having just one operating system,
Ubuntu, as a base image for now and we will make our product available for the companies
who are using Ubuntu to make their products. Some of the advantages of using Ubuntu are
that it is a free operating system and users can download software and applications from huge
repositories containing thousands of free software and applications. There are also many
commercial alternatives to the free software available, such as OpenOffice and Firefox.
Ubuntu is very easy to use and even a non-technical person is very comfortable in using
Ubuntu and installing applications and software. One does not have to accept the agreements
and simply clicks 10 times to install an application as most of the applications are free, and it
is not necessary to worry about the license. Ubuntu is very secure as there are very few
viruses on Linux and one need not worry about losing data. Changing the settings are also
very simple as you need to be logged in as an administrator and change your settings as you
want to for a particular user, you just have to enter administrator password and you are done.
There is a lot of help available on the Internet in the form of forums and blogs; if you are
looking for any kind of support whether technical or nontechnical, related to Ubuntu. Users
don’t need to call customer service and waste their time waiting for a technician to help
them. Everything is open and available for free. We can enhance our system by giving the
flexibility to the user to choose any operating system he or she wants to work with based
upon the requirements of the project. We are thinking of supporting Red Hat Enterprise and
Centos Linux operating systems. Once we are comfortable with Linux based systems, we
will study and understand the licensing issues with installing a Windows based system and
will try to implement that into our project.
45
For now the user can have three development systems ready to work on demand with
one operating system. We have given the user the flexibility to have three development
environments, such as Web Server (apache) installed on the base AMI, at present Ubuntu;
PHP Server (PHP and apache); and Database Server (mysql). If a user is working on a
project related only to mysql, then he or she can log into the JIRA system and check the issue
assigned to him or her and click on the “LAUNCH” button (which is an expansion of the
JIRA system) and an image will be spawned on the Amazon EC2 with the base AMI and
mysql installed on it. The user will get an email with the detailed information of username,
password, key, image ID and other information required to log into the system. The user can
then log into the system and directly work on the assigned task as the system will be ready to
work on demand with all the project specific software installed on it. By customizing the
JIRA, the administrator can store all the information by Web UI interface. We have a Web
interface, which has access to only the administrator of the JIRA who is responsible for
storing all the information of the project and whose role is required for the project with all
the repo information.
In big companies, there are hundreds or thousands of projects going on with different
teams and departments and all of them are working with different technologies. They require
different platforms to work on the projects. Most of the companies today are using JIRA as a
bug tracking or as a project management system and also expanding JIRA by writing plugins,
and the capability to customize JIRA is an important advantage. We can get a list of all the
projects of an organization from JIRA, and with our plugin the JIRA administrator can map
the projects with the role (PHP Server [php and apache], Database Server [mysql], and Web
Server [apache]) and give the information about the repo’s from where the packages need to
install. By this process, we can map different projects with different roles and the
developer/tester can get a system on demand by a click of the button. In future, we can
provide more roles depending upon the customers’ requirements and improve our system to
be more flexible and user friendly. Currently, on the Admin screen, we have provided a
combo box with a list of projects and a text box to enter the roles associated with that. We
can improve this system as we add more roles into our project and convert the text box for
entering the roles into the combo box and map them in such a way that all the configured
roles in the cookbook on the server can be fetched directly and we need not manually enter
46
the name of the role any more. Also, based on the selected role, the URL of the cookbook
repo can come automatically. Doing this extension of our system will give more accuracy
and reduce the chances of making mistakes while manually entering the name of roles
associated with the project and cookbook repo.
For this version of the project, we have made our system to be simple and work with
limited tools and software because this will enable our system to function well without
having to add the overhead of other tools. Also, by not making its design complex and
complicated, we can test our product on this level more efficiently. The scope of this product
is project based. But in JIRA we can subcategorize our projects and in future we can even
add more fields in our Admin screen to select the project category, and, based on the project
category, we can assign roles and other information. Then, our product will not be like one
project mapped to one role. There could be a requirement where one big project is divided
into two modules each of them using different technology and platforms. We could then
handle these types of uses by enhancing our UI to make it flexible enough to map projects
with multiple roles. This will increase the scope of our product usage and the visibility of our
project as a product to be used commercially.
We could even add custom software and applications if customers want to install their
own software and packages on the system. As we are only supporting standard software and
packages at this time, we could even extend and modify our system to install custom
software as per the requirement of the customer. In the JIRA Admin screen we have given
the provision of adding APT repo information to be stored in the database. With the use of
that field, we can modify our system to install custom packages, and we would put the
packages under APT repo and pull them out from a URL stored into the JIRA database and
our program will install them. We can support any custom software and packages as long as
they can be accessible through APT. The Advanced Packaging Tool (APT) simplifies the
process of installing the software packages, and managing and retrieving the software by
automating them. Different customers will have different needs for the development
environment of their projects, and we have implemented our product to be simple and open
in design so that we can easily extend the system without making much change in the design
and implementation part of our present system. There is always room for improvement in any
project, and as more and more users and customers use your product, you get to know the
47
bugs and limitations in the product. We are open to support and fix any bugs, add new
features, and improve present functionality for the future. We started the requirements and
design of the project to explore the idea of an on demand, scalable, automated framework
that automates the set up of development environments for the software industry. We studied
a lot of research papers, which proved to be helpful in deciding the technologies and tools to
be used in our project.
We have used the JIRA bug tracking system with the Amazon EC2 as the platform to
build our product. We could look for other bug tracking systems other than JIRA to
implement the concept of spawning an image on a click of the button. We are calling the Java
program when the user clicks on a button for image creation, so we can implement our
product just by extending any other bug tracking system, which has all the functionality we
require completing the process as in JIRA. That will require some work at the Web UI
interface front with no changes in the server program and client program. With the advent of
new technologies and products, we will keep our project updated and will incorporate future
modifications and improvements, whether they are suggestions from the customer or any
further changes which can make our system more efficient and powerful. We will continue to
work on this project to make a successful product in the software market that will save a lot
of time for developers and testers and automate the process of installing the software and
applications to set up the development environment.
48
REFERENCES
[1] APACHE SOFTWARE FOUNDATION, Introduction to the POM. Apache Maven Project,
http://maven.apache.org/ guides/introduction/introduction-to-the-pom.html, accessed
June 2011.
[2] J. BROBERG , R. BUYYA AND Z. TARI, MetaCDN: Harnessing ‘storage clouds’ for high
performance content delivery, J. Net. Comp. App., 32 (2009), pp. 1012-1022.
[3] Q. W. FLEMING AND J. M. KOPPELMAN, Earned Value Project Management, Project
Management Institute, Inc., Pennsylvania, 2000.
[4] N. HANAKAWA AND K. OKURA, A project management support tool using
communication for agile software development, Proceedings of the 11th Asia-Pacific
Software Engineering Conference, Japan, 2004.
[5] N. JALBERT, AND W. WEIMER, Automated duplicate detection for bug tracking systems,
Proceedings of the Dependable Systems and Networks Conference, Alaska, 2008.
[6] D. E. JONES, The open for business project: Entity engine guide. Ofbiz, http://ofbiz.
apache.org/ docs/entity.html, accessed June 2011.
[7] W. LU, J. JACKSON AND R. BARGA, AzureBlast: A case study of developing science
applications on the cloud, Proceedings of the 1st Workshop on Scientific Cloud
Computing, Chicago, 2010.
[8] N.A., OpenSymphony RIP 2000-2011. http://www.opensymphony.com/, accessed June
2011.
[9] J. NOLEN, JIRA architectural overview. JIRA 4.4 Early Access Program,
http://confluence.atlassian.com/display/JIRA/JIRA +Architectural +Overview,
accessed June 2011.
[10] J. NOLEN, PicoContainer and JIRA. JIRA 4.4 Early Access Program, http://confluence
.atlassian.com/display/JIRA/ PicoContainer+and+JIRA, accessed June 2011.
[11] D. NURMI, R. WOLSKI, C. GRZEGORCZYK, G. OBERTELLI, S. SOMAN, L. YOUSEFF AND D.
ZAGORODNOV, The eucalyptus open-source cloud-computing system, Proceedings of
the International Symposium on Cluster Computing and the Grid, China, 2009.
[12] OSGI ALLIANCE, OSGI main home page. http://www.osgi.org/Main/Home Page,
accessed June 2011.
[13] PICOCONTAINER, Dependency injection. PicoContainer, http://picocontainer.org
/injection.html, accessed June 2011.
[14] S. RICHTER AND C. THIELE, Benchmarking the cloud - A comparison of lokad on azure
and elastic MapReduce. Unpublished report, 2010.
[15] N. SERRANO AND I. CIORDIA, Bugzilla, ITtacker, and other Bug Trackers, IEEE
Software, 22 (2005), pp.11-13.
49
[16] S. TILLEY, J. GERDES, T. HAMILTON, S. HUANG, H. MULLER AND K. WONG, Adoption
challenges in migrating to web services, Proceedings of the Fourth International
Workshop on Web Site Evolution, Washington D.C., 2002.
[17] M. A. VOUK, Cloud computing–Issues, research and implementations, J. Comp. Info.
Tech – CIT, 16 (2008), pp. 235-246.
[18] G. WANG AND T.S.E. NG, The impact of virtualization on network performance of
Amazon EC2 data center, INFOCOM Proceedings, San Diego, 2010.
[19] C. WEISS, R. PREMRAJ, T. ZIMMERMANN AND A. ZELLER, How long will it take to fix
this bug? Proceedings of the Fourth International Workshop on Mining Software
Repositories, Minnesota, 2007.
[20] WIKIPEDIA, Cloud computing. Wikipedia, http://en.wikipedia.org/wiki/Cloud
_computing, accessed June 2011.
[21] WIKIPEDIA, Content delivery network. Wikipedia, http://en.wikipedia.org/wiki/Content
_delivery_ network, accessed June 2011.
[22] WIKIPEDIA, OpenSymphony. Wikipedia, http://en.wikipedia.org/wiki/OpenSymphony,
accessed June 2011.
50
APPENDIX
ACRONYMS
51
Acronyms
GUI Graphical User Interface
AWS Amazon Web Services
EC2 Elastic Cloud Computing
S3 Simple Storage Service
IDE Integrated Development Environment
AMI Amazon Machine Identifier
SaaS Software as a Service
IaaS Infrastructure as a Service

Sood shilpa

  • 1.
    AUTOMATED INFRASTRUCTURE CREATIONTHROUGH JIRA BUG TRACKING SYSTEM _______________ A Thesis Presented to the Faculty of San Diego State University _______________ In Partial Fulfillment of the Requirements for the Degree Master of Science in Computer Science _______________ by Shilpa Sood Summer 2011
  • 3.
    iii Copyright © 2011 by ShilpaSood All Rights Reserved
  • 4.
    iv DEDICATION This work isdedicated to my family, my mentor, who guides me to accomplish my goals and always motivated me for my hard work.
  • 5.
    v ABSTRACT OF THETHESIS Automated Infrastructure Creation through JIRA Bug Tracking System by Shilpa Sood Master of Science in Computer Science San Diego State University, 2011 It takes tremendous amount of time for developers to setup the environment to fix a bug in an application. Depending upon the complexity of application, it may take a few days to several weeks for developer to replicate a production environment. Very few solutions exist today that help reduce that time. With the help of Amazon EC2 you can in theory set up instantaneous infrastructure through a bug tracking system (JIRA).For each application, the developers usually need a machine for the old platform on which the bug exists, and a separate machine to install the new environment, which has the bug fix. In some situations, the QA environment could need a separate machine for all the testing. If the total number of applications increases to say 40, then this leads to a maintenance nightmare, and the whole solution would be really expensive. The solution will consist of extending an issue tracking system to capture the configuration of a specific environment through a plugin and automatic installation and configuration of a system through templating of a system.
  • 6.
    vi TABLE OF CONTENTS PAGE ABSTRACT...............................................................................................................................v LISTOF TABLES................................................................................................................. viii LIST OF FIGURES ................................................................................................................. ix CHAPTER 1 INTRODUCTION .........................................................................................................1 2 BACKGROUND AND LITERATURE........................................................................6 WebWork.................................................................................................................9 Seraph ......................................................................................................................9 PropertySet.............................................................................................................10 Cloud Computing...................................................................................................11 3 METHODOLOGY AND DESIGN.............................................................................19 Introduction............................................................................................................19 JIRA Plugin Overview and Software Requirements .............................................19 PicoContainer and Dependency Injection..............................................................21 JIRA Interaction with Database.............................................................................22 Presentation Layer ...........................................................................................24 Business Layer.................................................................................................25 Data Layer........................................................................................................25 Development Phase................................................................................................25 Velocity Overview.................................................................................................27 Mockup Description...............................................................................................29 Functionality of the System ...................................................................................30
  • 7.
    vii 4 RESULTS ....................................................................................................................36 5CONCLUSION AND FUTURE WORK ....................................................................43 REFERENCES ........................................................................................................................48 APPENDIX ACRONYMS.....................................................................................................................50
  • 8.
    viii LIST OF TABLES PAGE Table1. Types of Dependency Injection .................................................................................22
  • 9.
    ix LIST OF FIGURES PAGE Figure1. JIRA architecture........................................................................................................8 Figure 2. OfBiz framework......................................................................................................23 Figure 3. Example of pom.xml. ...............................................................................................26 Figure 4. Mock up diagram......................................................................................................29 Figure 5. Screen shot of admin screen.....................................................................................30 Figure 6. Screen shot of issue screen with launch button........................................................31 Figure 7. Screen shot of add admin screen. .............................................................................37 Figure 8. Screen shot of role validation...................................................................................38 Figure 9. Screen shot of APT repo validation.........................................................................38 Figure 10. Screen shot of cookbook repo validations..............................................................38 Figure 11. Screen shot of edit admin screen............................................................................39 Figure 12. Screen shot of delete admin screen. .......................................................................39
  • 10.
    1 CHAPTER 1 INTRODUCTION Today, almostevery company, large or small, uses a bug tracking system. It is one of the most basic components of a development process. No system is made perfect in the first iteration. Bugs always exist in software’s, and a bug tracking system gives an organization an estimate of the amount of work that needs to be done to approach the goal of perfection. Server machines are the other component of the development process. Sizable software needs big machines with a large amount of processing power and memory so that thousands of users can be connected to it at the same time. Our motivation was to combine the above two ideas and make them into one synchronized system. We have integrated an open source bug tracking system (JIRA) with one of the best virtualized server providers (Amazon) so that the developers can call from one system to another, and can have a very powerful server machine with the single click of a button. There are various companies offering bug tracking systems, but JIRA is one of the best open source products available in the market today. Its open source nature makes it possible for thousands of developers to use and work on it on a daily basis, thereby enabling much more rapid updating than its closed code competitors. The open source model also makes sure that all the security vulnerabilities of the bug tracking system are quickly identified, and are fixed with the next possible release. As the source code is available to the public, companies can use the Java based source code, modify the front end or backend according to their requirements, and create a completely customized product for their company. They can even release that product back into the development branch of JIRA and contribute to the continuous development of the project. JIRA is used by many big and small companies for their day-to-day operations; therefore, one can imagine the scalability of the software, which can shrink and grow quickly depending upon the daily requirements. There are hundreds of plugins of JIRA available on the market, some of them cost and some are free. Different companies can use these add-ons to connect to different kinds of systems.
  • 11.
    2 Cloud computing isone of the buzz words in the computing industry today. The kind of flexibility offered by cloud computing makes it an ideal candidate for use in all kinds of companies, from dynamic start-ups that require fast infrastructure to big companies which are trying to consolidate their huge infrastructure by going towards the virtual route. In this way they can save money by creating virtual machines that require less space and money when compared to a situation in which they require a physical hardware machine for every situation. If we have to assign a physical machine for every role in the project, that would involve considerable investment in power for the machine, cooling equipment and the personnel to configure and maintain all those machines. As the inventory of physical machines grow over time, all the above mentioned resources would keep on increasing. With a virtual infrastructure, all the above mentioned investments would be considerably lower, and that would save a company a lot of money which could be shown as profit at the end of the year. If a company uses a third party virtualization provider like Amazon, the company does not need to worry about all the above mentioned resources, and their engineering activities could take preference. There are a lot of virtualization providers in the market today, each of which has their own unique selling point. Every company is trying to move their services to the cloud to harness the power of mobile computing, so that all the users of the company can use this service from any place. As long as the user has a desktop computer and a browser with an active internet connection, the user is given power to connect to really powerful machines and either use the machine directly for their own use or connect to those machines and use some of the services hosted by the cloud provider. Let’s take an example of a big company like Google, which offers one of the most used mail services, called Gmail, by harnessing the power of the cloud. All the servers that have the Gmail service hosted are on the cloud and have some sort of clustering scheme thus ensuring that the service is not affected by one of the server machines going down because the data on that server is quickly replicated to one of the other servers and the other machine can take the place of the machine which went down initially. All the above mentioned steps should be transparent to the user so that he has absolute confidence in the availability of the service. Similarly, there are other web services provided by companies like Yahoo, Microsoft, and Salesforce, etc., which provide a unique
  • 12.
    3 functionality to theuser. All these vendors are said to be using the “Software as a Service” (SaaS) functionality of cloud computing. There are other cloud computing providers that provide the “Infrastructure as a Service” (IaaS) of cloud computing. One of the most popular companies providing that service is Amazon which offers its suite called “Amazon Web Services.” This suite consists of different services such as Amazon Simple Storage Service (S3) which provides a huge amount of storage to the users so that they can upload different things like their personal information, as well as other things that are to be used by other services in the same suite. The users of this service first have to make an entity called “bucket” which is the encapsulating identity for the information. The users can then use tools provided on the Amazon Management web interface to upload the data, or can even use Amazon Command line AMI and API tools to upload the different kinds of data in the bucket. This data can then be retrieved either by using the same suite of command line tools, or using the Web interface to download the items on some other computer at a different location. Amazon Elastic Cloud Computing (EC2) is another service that the users can use to get (rent) powerful machines, which can be used to host different kinds of services required by the user. A user is given an option of choosing a small, medium or large instance, each of which is different in terms of the processing power and the amount of memory available on the machine. A small instance usually has 4 GB of RAM with a dual core processor (this configuration can be slightly different depending upon the user), which is a good enough machine and powerful enough to run a small service like a web server which can be used to service hundreds of users with a simultaneous connection. The machines can have different kinds of Operating systems such as Windows or Linux, each of which can have different features and different architecture, and the user has the option of choosing an operating system which best suits his requirements. All of these different kinds of machines can be hosted in different zones. The United States has different zones available; the same applies to other different geographic parts of the world like Europe and Asia. Choosing machines in different zones and then replicating information between them ensures that the machines have a high resistance to different natural disasters. And if the machines in a particular zone are affected due to some uncontrollable circumstances, there are other machines always available in other parts of world which ensures that the service provided by the user using Amazon’s infrastructure has
  • 13.
    4 a readily availableservice resistant to different kinds of disasters. There are a lot of other services provided by Amazon Web Services suites, such as Amazon Elastic Map Reduce, CloudFront, and Elastic Benstalk. All of these services are tightly integrated and work closely with each other to give complete control over every aspect of the cloud computing experience. In this project, we have used different technologies like Velocity, Ofbiz and JDBC on the JIRA side to create a different screen that will be used by the administrator of the system to program different “Nimbo” related attributes in the standard JIRA bug tracking system. We have created a web page called “Admin Nimbo Screen” that will be used to associate a project with the Nimbo functionality. We have to first log in using the administrator user account that has the privilege to access all the projects of the company mentioned in the JIRA bug tracking system. Once we do this, and we have logged in using the administrator user account, we get a list of all the projects in the company. We can then select a single project and fill in the values for all the Nimbo related attributes like Role, ImageId, Apt Repo and Cookbook Repo and then click the submit button to store all of these values. Once the submit button is clicked, we validate all these attributes for their correct values so that we do not have any problems while the image is being generated on Amazon EC2 for these projects. Different checks like the NULL value check are done to make sure that we don’t insert any blank values into these attributes. We also check to make sure that the value for cookbook repo and apt repo has a web address inside them to ensure that they have an http or an https value in those attributes. Once all the checking is done, and the values entered are valid, we then store these values using a SQL query (using JDBC connection) in a separate table called PROJECTNIMBO that has the “Project Name” as the primary key of the table. Once the Nimbo functionality is enabled on a particular project, and a role is associated with the project, all the issues in that project get a “Launch Nimbo” button on the top left-hand side of the issue screen, which can be clicked by the developer working on that issue to generate an image for that particular role by just clicking that button. Once that is done, the user is taken to another screen which shows that the image is being created on Amazon EC2, and then all the software’s associated with that role will be automatically installed on that machine. At the end, he will be sent mail to the email address associated with his user account on the JIRA bug tracking system with the instructions to connect to the
  • 14.
    5 machine using aSSH client. The administrator of the JIRA system has to make sure that the key that is used to connect to those machines is already given to the developers, and that they know all the steps to follow in order to connect to that machine effectively. When the user clicks the button to launch the image, a server side Java wrapper program is called on the JIRA server that uses the Amazon Web Services (AWS) (for list of acronyms see Appendix) Software Development Kit (SDK) with the credentials of the account used to generate images on the Amazon EC2. The JIRA plugins call this server side program by giving the issue key and the username of the developer as the input; the program also uses the ImageCreation.properties file to take the image ID and the key name that is to be used for the AWS SDK tools to create the image on Amazon EC2. After the commands to make an image are given, we wait for a fixed amount of time (again picked up from the ImageCreation.properties) and then pass the control over to other programs which will do the installation of software’s on the machine and then send the user the email with the instructions to connect to the machine. All the above actions will give us a system in which we skip all the delays associated with getting the hardware of a machine allocated to our issue, and then coordinate with the IT department that installs all the software’s on the machine. In the case of large companies, this delay can be significant which could waste time doing a simple action like getting the machine ready so that the developer can create a software image in the production environment required to reproduce, resolve and verify a fix for the issue. By using this system we can get a machine with a single click, so that the developer can start working on the issue with a new machine in a matter of few minutes. In the next chapter, we will give an overview of all the technologies used to achieve our goal of making the complete Nimbo System.
  • 15.
    6 CHAPTER 2 BACKGROUND ANDLITERATURE Bug tracking systems are very important for the maintenance of a project [15]. When we work on any software related project, we have to manage the bugs that exist in the application and fix them. Some developers list them on a spreadsheet, but it becomes difficult when a project is big and the bugs become too many, and when there are many developers working on the project. Therefore, to maintain a project efficiently, a bug tracking or issue tracking system is very necessary, especially during the testing or deployment phase. In software development, bugs are always unavoidable, or we can say they are a part of the system; hence, our choice of bug tracking [5] should be one which is simple and intuitive with all the functionality. There are many such bug tracking systems on the market; some are open source, others are commercial projects. JIRA is an example of such a bug tracking system and this system will be explained in detail. JIRA is a powerful tool, developed by Atlassian, and it can be used for various purposes like bug tracking, maintaining the projects and also planning tools for agile projects. JIRA is available as an open source project for free to organizations that are non- profit, and also the full source code is available for purchase by commercial customers. JIRA is used for issue tracking and project management by many organizations and in many countries by various sectors. It is used in software organizations to improve the quality of code and to speed up development. JIRA as a bug tracking system simplifies every step involved in tracking a bug for everyone involved in the process. With its intuitive interface creating, resolving and tracking bugs is very easy and quick. Fixing a bug for a software development team is just like fixing code [19]. It is the most important task for any software development team for the quality and performance of the product. To make life simple, with Atlassian IDE Connectors, developers can interact with JIRA directly through Eclipse and many other IDE. We can always connect JIRA bugs to the source code with the CVS integration and plugins to any version control system like Subversion, ClearCase, and Mercurial. Also to make the whole system more efficient, continuous integration can be
  • 16.
    7 performed to viewthe builds and integrate the comments related to the JIRA bug from the code reviews into the project. JIRA can be extremely powerful depending upon the need of the user and the user’s knowledge about its features. JIRA as a project management tool can be extremely useful to enable the project managers to speed up the work and track the projects of their respective teams [3]. JIRA is focused on the task achievement and manages all the projects and components in one location. It tracks all the projects, manages the task list, assigns an unlimited number of tasks and sub-tasks. It assigns the description of the project as well as about the task, can attach files and even customize the work flow. To keep the team on track, JIRA has the ability to make time tracking reports for each individual, set up a road map and status updates on the dashboard, and can even export report data to Microsoft Word, Excel, XML and as RSS feeders. JIRA also helps the developer and tester monitor the progress of the project by the status updates on the dashboard, by the charts and reports on the personal dashboard, and by automatic reports subscription and email updates. It also provides secure online access through granular security schemes and by enterprise level security management. Time tracking can also be helpful in managing the projects by setting up an estimate for each task, time spent on the logs of each task, time tracking record of each phase of the project. Also it can track risky tasks in a project and set up RSS feeds on risky tasks. JIRA can also be integrated with an MS project to speed up the project manager tasks. Agile project management with JIRA is another useful functionality for the software development teams [4]. There are a set of tools used to extend JIRA for the agile management, as JIRA is not explicitly designed for use in the agile environment. GreenHopper is a full implementation plugin mainly used for Agile Developers using JIRA. It represents all the JIRA issues in a very simple form; for example, different issues are of different colors. Summary and other crucial information to estimate the task, time remaining and planning are also displayed. To reorder the tasks, just drag and drop which is always useful for any project manager to save time. GreenHopper also offers a Task Board to show the work queue for a component or version. Chart Board will show the burn down charts for a release. The graphs shown by the Chart Board are a burn down curve, team effort curve, estimation accuracy curve, and many others. I have explained in brief some of the uses of the
  • 17.
    8 JIRA for bugtracking and project management. Now, we will give some technical introduction about the JIRA. JIRA is a web application written in java. It is deployed as a standard Java WAR file into a java servlet container such as Tomcat. Figure 1 is adapted from http://confluence.atlassian.com/display/JIRA/JIRA +Architectural +Overview [9]. Response Request Figure 1. JIRA architecture. Source: J. NOLEN, JIRA architectural overview. JIRA 4.4 Early Access Program, http://confluence.atlassian.com/display/JIRA/JIRA +Architectural +Overview, accessed June 2011. Understanding the JIRA architecture is very important for the developers who want to extend the system and also to the testers and other users. As we know, JIRA is a Web Application and the user interacts with it through a web browser as shown in the figure above. We will now explain the technical details of each component used in the JIRA architecture. Lucene Index Files Lucene JSP Seraph Web Browser Jelly RPC Atlassian Scheduler Quartz JIRA Service Relational DBMS OSWorkflow OfBiz Entity Engine PropertySet JIRA Utility and Manager Classes Web Work Web Work Action Web Work Action Web Work Action Web Work Action OSUser JIRA Service
  • 18.
    9 WEBWORK JIRA uses OpenSymphony’s[22] Webwork1 to process the requests submitted by the users. One of the most important task in developing the web application framework is to separate the code, logic and presentation. If this concept of separation is not handled wisely then it is most likely to have maintenance problems in the application. As the whole team works on the same application, if there is no structure then it becomes really difficult to have a clean and maintainable code. One of the most popular ways to achieve this separation is to use a design pattern called Model-View-Controller (MVC). WebWork is a MVC framework, or we can say service invocation framework. It is architecturally based on the design patterns and best practices which have been proven very useful in building a stable and simple framework while maintaining flexibility. It is mainly built on the concept of actions and views. WebWork actions handle the entire request by the users and uses other objects such as utility and manager classes to complete the given task. An action in WebWork is the piece of code (normally java class but not necessarily) containing the entry point, a method named execute () to begin its processing. Actions have a particular lifecycle in WebWork. It uses a dispatcher to look up the entry point and take the request itself, and then maps it to a particular action called the execute () method. This method returns a java.lang.String, which is mapped to another Action, or a View. The view in the WebWork generates the output and JIRA uses JSP for the view layer. Views can be anything like JSP, Velocity, Swing, XML/XSL, and Jasper which means WebWork supports different HTML generation technologies and gives the users the flexibility to choose their own ways to accomplish their work. A View gets its particular request data, and it serves users as a response to their request with the HTML generated by the views, JSP in the case of JIRA. When views take a request data it looks for it on entire action sets and returns the first relevant match for the data. SERAPH Seraph is the Atlassian’s open source web authentication framework used for most of the authentications for JIRA. It is based on a very simple, J2EE pluggable web application security framework so that the authentication system can be used on any application server. It is implemented as a Servlet filter and based around the number of core elements pluggable in
  • 19.
    10 the framework. Seraphgets a web request and associates the request with the particular user. It supports different types of authentications including the form-based authentication, HTTP Basic authentication, and credential authentication for users already in the session. To decide the role required for any known request we use Security services in Seraph. The WebWork Service is packaged with Security services to secure the WebWork 1 actions. To implement this service we use action.xml to configure the actions in the web application and add “role- required” as an attribute to each action component. Also there is a Path service along with WebWork Service in the Security services package which secures URL paths and is configured in the security-config.xml file. To configure Seraph properly to the web application, we need to modify 2 files: one is seraph-config.xml which resides under WEB-INF/classes directory and the other is web.xml under WEB-INF used to add a servlet filter. There is an Authenticator class in Seraph used to handle user authentication. To integrate JIRA with the single sign-on (SSO) solution in your applications, custom authenticator’s can be written to implement this concept. JIRA does not contain the SSO solution by default; therefore, to integrate, one has to write one’s own custom authenticator according to the needs. One of the very important purposes of the Seraph in JIRA is to allow only the users with Global Admin permission to access the WebWork Action that allows users to manage the administration tasks. These WebWork actions are accessed by URLs starting with "/admin." PROPERTYSET PropertySet is an OpenSymphony’s framework used to accumulate a set of property (key/value) pairs against a specific entity with a unique ID. It can be used to accomplish storage necessities for the applications which change frequently. Every property will have a key and its corresponding value which can be java.lang.String, java.lang.Date, java.lang.Double, java.lang.Long. Each property is always connected with one entity. In order to store and retrieve the value, the same entity name/ID pair is used and everything works well. It is impossible to figure out what user can store at any given time in your application, so utilizing the Propertyset can help. A Typed key/value, supported by XML, ofBiz, JDBC and JGB, or any other persistence mechanism can be implemented. In JIRA, PropertySet is used by the UserPropertyManager to store the user preferences and to store
  • 20.
    11 Application Properties, whichcan be used to customize the JIRA installation by changing the configurable setting by the user and can also be used to store the preference of Portlets on the user’s dashboard. PropertySet uses the following database tables in JIRA: 1. propertyentry - contains the entity name and ID for a property, its key and the data type and also each entry has its own unique ID 2. propertystring - contains string values 3. propertydecimal - contains double values 4. propertydate - contains date values 5. propertynumber - contains long values. The majority of PropertySet implementation store data in a relational database. Sometimes the speed of querying the data on the tables is slow as it stores a wide variety of data which is unknown at the time of development and the tables are not very optimized or in a normal form. So creating the indices on the OS_PROPERTYENTRY table will speed up queries by a high number factor. Other optimization techniques can also be used beside the indices; for example, by properly and evenly distributing the key name while writing the application, the database can have much faster access to PropertySet. CLOUD COMPUTING For a flexible, on-demand, and dynamically scalable computing infrastructure for various applications, the solution which comes to mind is Cloud Computing [7, 11]. It can be defined as a new style of computing in which dynamically scalable and often virtualized resources are provided as services over the Internet. It has become a significant technology trend in information technology (IT) processes and the IT marketplace. With the services offered by the cloud computing providers, we can use different devices, including PCs, laptops, Smartphone, and tablets to access programs, storage, and application-development platforms over the Internet. The major advantages of this technology include cost savings, high availability, and easy scalability. There are three types of cloud computing[20]: (1) public cloud, (2) private cloud, and (3) hybrid cloud as described below. In the public cloud or sometimes known as external cloud computing, resources are dynamically provisioned over the Internet via Web applications or Web services from an off- site third-party provider [16]. Public clouds are run by third parties, and different customers have their applications, storage systems, and networks together on the cloud’s servers. Private
  • 21.
    12 cloud or sometimesknown as internal cloud means cloud computing on private networks. They are used mainly for one client only, which provide full control over data, security, and the quality of service. Private clouds can also be built and managed by a company’s own IT organization or by a cloud provider. A hybrid cloud combines an environment from multiple public and private cloud models. Hybrid clouds help in determining the complexity to distribute applications across both a public and private cloud. The ability to virtualize and share resources among different applications for better server utilization is the main advantage of using cloud computing. We can share servers or do virtualization for the use of operation systems and applications which results in fewer servers. The major features of cloud computing include Scalability and on-demand services, User- centric interface, Guaranteed Quality of Service (QoS), Autonomous systems, and Pricing. Cloud Computing provides services and resources for users on demand, and the resources provided are scalable over many data centers [17]. The interface that cloud provides is totally independent and can be accessed by Web services and Internet browsers. They guarantee QoS for users in terms of hardware, performance, and memory. The cloud computing systems are autonomous systems and can be managed transparently to users. However, depending on the needs of the user, the data and software inside the cloud can be reconfigured and consolidated to a simple platform automatically. It is less expensive compared to the physical resources since users pay for services and capacity as they need them. Different customers use the cloud service differently, and the service provider can’t anticipate the usage of each individual. One customer might use it as a primary development platform for all of its applications, and another customer might use it two or three times a year depending upon the need of his business. Due to the different needs of the customers, the service needs to be available everyday (7 days a week, 24 hours a day), and it should be designed to be scalable for high periods of demand and for lighter ones. Scalability can be defined with different meanings; for example, it can mean that an application can be scaled when the requirement of the application changes and when additional users are added into the application. Cloud services for customers are much easier to get without going through a lengthy process such as the services provided by data centers at any software organization. The customers have to make a simple request for an amount of computing, storage, software, process, or other resources from the service provider. Of course, the data center gets similar
  • 22.
    13 requests from departmentsof the company and checks for other requests from other departments to give priority and make sure of the availability of existing resources because of the need to purchase new hardware. After new hardware is purchased, the data center staff has to configure the data center for the new application. Depending on company policies, these internal procurement processes can take a long time. There are a number of cloud computing providers on the market. Not surprisingly, some of the biggest names in cloud computing are also some of the biggest names in the computer world, such as Google, Microsoft, Yahoo!, Salesforce.com, IBM, and others. Basically, these providers offer similar services such as to compute your tasks on virtual machines, store your data on remote servers, and use web based applications. As there are many options, finding the best cloud computing vendor is not easy and you need to assess the capabilities of the providers very carefully before choosing one. Some users don’t need too much power serving to perform the computing jobs and they are looking for some free service provider. Google is an example of a cloud computing vendor that provides a lot of free services. You can use office applications and communications, store data on remote servers, retrieve data anytime and use email services that are all web based. Free cloud computing services would be very useful for personal use, for startup companies, and for small and medium sized organizations. On the other hand, if you are looking for large storage and more powerful virtual servers, then there are cloud computing vendors which charge for services. Such web services are offered by big companies such as Amazon, IBM, and Microsoft. Amazon is the most widely known cloud computing provider. They offer services on many different fronts, from storage to platform to databases. The Elastic Cloud Computing (EC2) of Amazon can provide big savings for companies. Amazon Elastic Compute Cloud is a web service to provide virtualization of an IT environment to eliminate the need for buying new hardware to serve your computing needs and is designed to make web scaling easier for developers. It is a big time and money saver, since by utilizing the services of the Amazon EC2 you can lease any number of computers and servers for $0.10 per hour in standard instances. Also, Amazon EC2 saves time because it takes only a few minutes to obtain and boot new server instances whenever you need to change the scale as per the requirement of the application. For the big projects that will need large computing capabilities to perform the tasks efficiently, renting or leasing the virtual computers on EC2 services would be the best
  • 23.
    14 choice for bigsavings in terms of both time and money. Software organizations or companies can surely get substantial cost saving benefits from EC2 cloud computing and better performance and faster results from the ability to run computing instances simultaneously. Amazon Web Service is collection of many web services that people can enjoy and get benefits from. The EC2 or Elastic Compute Cloud is one of the services which people use mostly to have their own virtual private server which they can control and manage themselves. The S3 or the Simple Storage Service is the web service which is very timely and provides people with safer and better storage for their personal files and documents. The MTurk or the Mechanical Turk is known to be one of the most complicated services that Amazon has ever provided. Amazon EC2 or Elastic Compute Cloud is not the only service but it is the most renowned of these kinds of web services [18]. Cloud Computing is becoming more famous among developers and software organizations because it is flexible, cheap, and reliable. Amazon EC2 is highly elastic, you can change your capacity needs within a few minutes to scale your application needs. It will save your time and speed up the development process because you will not have to wait for hours for the server to adjust to the capacity needs. If you are using EC2 service, it can provide data storage and other Amazon services for virtual computing. So you will have a good range of tools and servers ready to use in order to make application development easier. Amazon can run Microsoft Windows Server 2003 and is a way to deploy applications using the Web Platforms such as ASP.NET, Internet Information Server (IIS) and many more. EC2 also supports SQL Server Standard and SQL Server Express and makes those available to customers on an hourly basis. EC2 also allows you to run the windows-based applications on the Amazon cloud computing platform. This can be web-sites, data processing, or any other application which requires Windows software. EC2 and other computing services offer fail safe platforms to protect the applications of their customers. They have redundant data centers and can host the applications on many servers, and if a server breaks down, another server can come in and run the application. This process reduces the data loss and eliminates the downtimes of the server. Also, cloud computing is more secure as compared to the local servers because of the strong firewall technologies implemented by the web service vendors such as Amazon. These technologies protect the data from hacking and the attacks that come from the Internet.
  • 24.
    15 Amazon provides asimpleDB for database services to store data for its customers. It provides basic and important functions of the data indexing and querying. This database service works closely with Amazon EC2 and Amazon Simple Storage Service (Amazon S3). Amazon S3 storage service can be used by developers and online organizations to get web- based solutions for their needs of storing data. This provides the ability to store, process, and query the data sets in the cloud. The traditional relational database requires a considerable upfront expense and also the design is too complex to understand and requires a database administrator. Amazon S3 is very important and useful because it is very simple to use and requires no schema, automatically indexes data, and provides a simple API for storage and access. With S3, this process becomes easier to manage and eliminates the administrative load of data modeling, performance tuning, and index maintenance. Amazon’s data storage facilities are reliable and S3 outages are very rare and usually last for only few minutes. Amazon S3 utilizes a simple web service to store and retrieve any amount of data from anywhere on the web. This type of utility gives developers a chance to access the same data storage infrastructure that Amazon uses to run its applications. Moreover, on a S3 server your data would be protected because its security features are very strong and can be relied on to protect the integrity of your data. Lastly, Amazon S3 is cheaper and only costs for the space and bandwidth used. Amazon S3 can also be used as a content delivery network (CDN) [21]. A content delivery network or content distribution network (CDN) is a system of computers containing copies of data placed at various nodes of a network. This system is used to achieve improved performance, a wider scale of delivery and reduced cost of delivery. All these benefits are useful for the customers. Amazon S3 is one of the few Amazon web services that can be deployed to work with a group of connected computer systems in order to perform the delivery of content to all the people that are getting service from them. Very few companies in the market are using this content delivery network due to the lack of information about its usage and benefits. However, the number of companies using this technology will grow eventually because of the rise in the demand for the effective delivery of contents. There are numerous benefits of using cloud computing on Large Scale Web Applications. Software designed for the commercial development of large users usually belongs to the category of Large Scale Web Applications. These types of applications will
  • 25.
    16 need a largeamount of server space and disk space. They also require powerful CPU performance and fast memories. Large scale applications would also require purchasing a lot of new hardware and upgrading the software for their development. This could cost a lot and might be an obvious reason not to start with such applications. But with cloud computing, it becomes so easy to start large scale web applications without worrying about the new hardware. You can rent/lease thousands or any number of computers required for the development of applications. Also you can benefit from the huge data storage and fast servers, with minimal overhead cost, if you are developing your application through cloud computing. There are virtual infrastructures available on clouds which are used to host the applications on the web. So, a company doesn’t need to buy expensive servers to host their applications on the internet. Cloud computing makes large scale web application development successful and saves a lot of money which promotes the growth because of improved profitability. The Amazon EC2 provides scalability, but the management of EC2 requires a lot of supervision from personnel. It can be labor demanding in terms of making sure that high traffic levels are served and other changes are in tune accordingly. Website failures or application crashes could happen, so personnel are needed to control and look into these matters to run the operations smoothly. For easier EC2 management there are various extensions and applications offered such as ElasticFox. EC2 users are assisted by this extension which allows them to perform basic tasks of EC2 management. By setting up the ElasticFox, EC2 users can assign IP addresses, spawn instances and open up ports. There are also applications that can be used alternatively to allow automated EC2 management. Such applications provide service to EC2 users by scaling automatically their systems depending upon the usage. Sudden growth and spikes can be easily handled by the use of these applications. EC2 management becomes more efficient and the need for personnel to handle all these operations is minimized because problems and adjustments are handled gracefully. Due to the unstable IP environments, there is always a temporary loss of websites but with the more stable IP address environment these problems can be easily solved and websites can be revived again within a few seconds. These applications are very efficient and many EC2 users benefit from them. Depending upon the user’s requirement, it will always be an option to manage the EC2 manually or use the extensions and applications. If the scale of operation is big then it is always better to use EC2 management applications to get efficient
  • 26.
    17 results. EC2 usersalso want to have easier management administration. EC2 allows flexibility in order to add more users and capacity depending upon the usage, in short, scalability. These actions require good EC2 administration. In order to get increased capacity and cut down on costs, EC2 users need to balance loads very efficiently and smartly. Again with dynamic IPs assigned to run the websites and downtime problems, EC2 automation applications have been developed for load balancing. These EC2 applications will do everything for the users, and scaling capacity will be done automatically, therefore, diminishing the need for the administrator. With EC2 automation, most of the problems are solved, and it becomes a more powerful and more robust product to serve the needs of the customers. With these automated applications, the IP environment becomes more stable and downtime problems are solved as uptime is just a few seconds. Manual management and administration has been replaced by the efficient and reliable extensions and applications. It certainly becomes a most popular product nowadays. In our product, we have used Amazon EC2 for the creation of images, and we will briefly describe the importance of creating EC2 images. Amazon Machine Image or AMI is a special type of encrypted file that is stored in S3 [2] of Amazon and used to instantiate a virtual machine within the Amazon EC2. AMI contains all the essential information required in booting the instance of your application or software. The instance is nothing but a running system based on the AMI. Amazon EC2 provides different types of instances and the flexibility to choose among them based on the requirements; each instance provides different computing capacity and is charged on a per-hour basis. The instances with the identical AMI are also executed in the same manner, so any information that might be lost will be terminated. Hence, to avoid instances of termination and failures, creating EC2 images are very important. There is a large amount of EC2 images. There are very steep chances of finding the instances as there are lots of images available. Every EC2 image has a unique identity which is helpful in finding the image once booting is done. Amazon assigns an API called REST which provides each instance an identity and all the information about itself. You can create new AMI either by creating a new one from scratch or modifying and extending the ones which are already available and very close to what you are looking for. The steps which you need to keep in mind are the modification of the existing image based
  • 27.
    18 on the originalimage, preparing for bundling once the system has been modified successfully, and creating a snapshot which needs to be packaged into the AMI by using the ec2-bundle-vol utility. This utility encrypts and protects the image from any tampering or damage such that only you and Amazon EC2 can decrypt it. Bundling the image requires uploading the certificate. To access later you have to use your AWS [14] account ID as your username. The AMI is then uploaded directly to the S3 or Amazon storage. The uploaded image will be used in signing up for Amazon’s S3. This upload process can be lengthy, but you will get continuous feedback about the upload till the time the image is successfully uploaded. Registration of AMI is also necessary so that you can trace the image in order to run the instance later on. You can now finally run the instance of the modified AMI by specifying the image identifier you received at the time of image registration. You can monitor the status of the image until it’s ready and then connect the image to your browser to confirm the changes you have made to the original image.
  • 28.
    19 CHAPTER 3 METHODOLOGYAND DESIGN INTRODUCTION Themotivation of this study is to explore the idea of an on demand, scalable, automated framework that automates the setup of development environments for the software industry. Public and Private Virtualization solutions such as Amazon’s EC2 or Eucalyptus lay the foundation for such an environment to be on demand and scalable. Through the study of various research papers and articles, we have realized that there are very few such solutions in the market today. This chapter is dedicated to describing the methodology and design of our product and its various components in order to illustrate how this idea could be useful for the software industry in the future. My part was to engineer a push button and administration interface in the form of a java plugin into an issue tracking system. The push button interface will spawn an image by the click on a single button through an issue screen of the issue tracking system. The administration interface provides user access to the push button interface. One time mapping of projects to configuration and automation scripts, source repository location and package repository location are all part of the functionality that is provided by the administration interface. I have used JIRA as a bug tracking system for the implementation of the study. To understand completely the design and methodology of the product, I will explain the JIRA Plugin and what software is required to begin the plugin writing. JIRAPLUGIN OVERVIEW AND SOFTWARE REQUIREMENTS JIRA is an extendable issue tracking system built by Atlassian. It is used by software engineering firms to keep track of requirements and bugs. JIRA can be extended by developing java plugins. These plugins are simply jar files which contain all the codes, configuration files and resources that can be installed into the Atlassian product to add new features or functionality into it or to change the behavior of the existing system. There are two types of Atlassian plugins: First is “Version 1” plugin which is compatible with the old
  • 29.
    20 plugin framework, andwas used in JIRA prior to version 4.0; it is also compatible with the new plugin framework of JIRA version 4.0 or later. The installation directory of “Version 1” plugin is atlassian-jira/WEB-INF/lib/ and it must be installed and uninstalled manually. The second plugin is “Version 2” which is compatible with the new plugin framework which is based on the open social java framework applicable to JIRA version 4.0 and later. It is installed automatically using the JIRA administration interface. Also the version 2 plugin is installed into the JIRA home directory under plugins/installed-plugins/. ‘Version 2’ plugins are deployed into the internal OSGi container which provides a service-oriented, component- based environment for developers and offers standardized ways to manage the software lifecycle. These capabilities greatly increase the value of a wide range of computers and devices that use the Java platform[12]. They isolates the plugin from the core code and from the other plugins, it also allows dependency management. Also, Version 2 plugins have to be specifically declared by defining one of their attributes as plugin-version=”2” in plugin’s configuration file. To set up the environment for the development of plugins, a java development kit is required because Atlassian applications support JDK 6. We need to download and install JDK 6 and set the JAVA_HOME environment variable. We even need to install Atlassian plugin SDK; this SDK includes Maven with a well-configured settings.xml file and also some scripts which are useful to speed up and simplify the plugin development. After that, we have to add the SDK bin’s directory to the path environment variable. The next step is to decide on the code development tool because we need one that supports Maven 2. Atlassian Plugin development process depends heavenly on Maven, so we have to make sure our IDE supports that. After considering all the above mentioned criteria, we have decided to use Eclipse for the plugin development process. Writing your own plugin will require the developer to call or use JIRA code to accomplish certain tasks; for example, to retrieve the list of projects, to get a list of users and add new data to the issues or make changes in workflow. Understanding the JIRA architecture and its functionality is another important thing to be completed before beginning the plugin development. Customizing the JIRA sometimes requires modifications in the code. There are some default classes in JIRA, which conform to an interface like Manager classes or the Webwork action classes, so modifying the classes and then plugging modified
  • 30.
    21 classes into JIRAcan be made easy by writing your own implementation of JIRA interface. It is always a problem to integrate modified classes into an existing system. In this section, I will describe methods to use JIRA resources and functionality into our own implementation to address this problem. PICOCONTAINER AND DEPENDENCY INJECTION Software components depend on each other; for example, component X depends on component Y, so for X to successfully complete its task, the correct execution of Y is very important. Dependency Injection [13] (DI) is where components are given their dependencies through their constructors, methods, or directly into fields. Now there are some cases where components depend on each other rather than the normal way in source code where various references are made to other components. Component interdependency makes it harder to modify the components because no one can predict what will be the effects of the changes on depending components. Also, maintaining a larger project becomes a problem because fixing these dependency problems invites new problems in other parts of the software product. Lastly, it makes unit testing possible by keeping the instantiation of objects outside separate from other objects. That is why it is always recommended to use Dependency Injections in your applications. JIRA uses PicoContainer as it is a lightweight dependency injection framework. One of the important features of the PicoContainer is its ability to instantiate arbitrary objects and resolves their constructor dependencies. This is accomplished by using its API, which works very similar to a hash table. PicoContainer is very useful when there are larger numbers of classes and interfaces with complex dependencies between each other. JIRA uses PicoContainer [10] to manage object creation throughout the system or as a central object factory. There are different types of dependency injections that are supported by PicoContainer. One of the recommended methods of injection is through the constructor where the constructor of a class is used to pass dependencies into it. In Table 1 are some of the regular types and their descriptions. Understanding the concept of dependencies injection is very necessary in order to resolve the uninvited conflicts between the classes and interfaces while plugging the modified code into it. Using the PicoContainer in JIRA plugin development as a DI is very
  • 31.
    22 Table 1. Typesof Dependency Injection Regular Types Description Setter Multiple Setter methods on the class are used to pass dependencies in. Annotated Method Methods are marked with an annotation to indicate that they should be used for injecting dependencies Field Fields are injected into for dependencies Annotated Field Fields are marked with an annotation to indicate that they should be injected into Named Field Fields of a certain name should be injected into helpful in simplifying the code, as any PicoContainer instantiated object of class X can obtain an instance of another class Y by simply requesting one in its construction or by any other means mentioned in above table. Construction injection is one of the best DI implementation approaches in the PicoContainer. JIRAINTERACTION WITH DATABASE JIRA uses Entity-Engine module of OfBiz [6] (Open for Business Entity Engine) to communicate with the database. OfBiz is a set of tools used to model and manage entity specific data. An entity is defined by a set of fields and relations to other entities; it is a piece of data managed by certain rules defined in xml. The entity engine uses various patterns such as Business Tier patterns and Integration patterns which include: Business Delegates or normal Delegates, Value Objects (VO), Data Access Object (DAO), and Service Locator. The main purpose behind using an entity engine is to reduce the requirement of persistence code related to entity from as many layers as possible. This kind of framework will make a sort of abstraction that saves a lot of development effort. It does that by reducing the persistence related issues in the system while developing the project. Any change in one piece of code will not affect the rest of the code because of the layered architecture which separates the business logic from the presentation logic.
  • 32.
    The Entity Engineprovides all the utilities and tools necessary to securely access an unlimited number of databases regardless of their physical location, as shown in the Figure 2. OfBiz framework The Entity Engine provides OfBiz applications with all the utilities, tools, and an application programming interface (API) to easily read and write data to all the configured data sources in a consistent manner without the or for the database connection. As shown in the above figure, a remote database can also be connected to the Ofbiz engine. A remote database can be network connected or installed on a local server. Remote database requ In JIRA, the database schema is described in the entitymodel.xml file under WEB INF/classes/entitydefs directory. This entitymodel.xml file contains the XML definition of all the JIRA database tables, table columns, and their data types. the tables is also found in this file. JIRA database can be accessed through APIs or through jdbc connections. The Entity Engine provides all the utilities and tools necessary to securely access an unlimited number of databases regardless of their physical location, as shown in the ramework. ngine provides OfBiz applications with all the utilities, tools, and an application programming interface (API) to easily read and write data to all the configured data sources in a consistent manner without the limitation of the physical location of the data or for the database connection. As shown in the above figure, a remote database can also be connected to the Ofbiz engine. A remote database can be network connected or installed on a local server. Remote database requires a datasource entry in the entityengine.xml file. In JIRA, the database schema is described in the entitymodel.xml file under WEB INF/classes/entitydefs directory. This entitymodel.xml file contains the XML definition of all table columns, and their data types. Relationships the tables is also found in this file. JIRA database can be accessed through APIs or through 23 The Entity Engine provides all the utilities and tools necessary to securely access an unlimited number of databases regardless of their physical location, as shown in the Figure 2. ngine provides OfBiz applications with all the utilities, tools, and an application programming interface (API) to easily read and write data to all the configured physical location of the data or for the database connection. As shown in the above figure, a remote database can also be connected to the Ofbiz engine. A remote database can be network connected or installed on a ires a datasource entry in the entityengine.xml file. In JIRA, the database schema is described in the entitymodel.xml file under WEB- INF/classes/entitydefs directory. This entitymodel.xml file contains the XML definition of all entry between the tables is also found in this file. JIRA database can be accessed through APIs or through
  • 33.
    24 Here is anexample of one of the entries of the tables in the entitymodel.xml file. <entity entity-name="TrackbackPing" table-name="trackback_ping" package-name=""> <field name="id" type="numeric"/> <field name="issue" type="numeric"/> <field name="url" type="long-varchar"/> <field name="title" type="long-varchar"/> <field name="blogname" type="long-varchar"/> <field name="excerpt" type="long-varchar"/> <field name="created" type="date-time"/> <prim-key field="id"/> <relation type="one" title="Parent" rel-entity-name="Issue"> <key-map field-name="issue" rel-field-name="id"/> </relation> </entity> Each table is treated as a single entity and field names and their data types can be defined as showed above, along with those relationships between the entities. There are many ways to access the JIRA database while creating plugins. There are JIRA APIs which can be used to access the database. If APIs are used, then a lot of code deals with GenericValue objects. Each GenericValue object represents the record in the database; it is also an object of the OfBiz entity engine. We can always edit the entitymodel.xml and put that under the different directory such as the webapp/WEB-INF/classes/entitydefs/entitymodel.xml and create an entry of the table required in the plugin development. The functionality of the OfBiz framework can be divided into the following distinct layers: Presentation Layer OfBiz uses the concepts of screens to represent the pages. A page in OfBiz consists of components which can be header, footer, etc., and all these components have to be put into a single directory. These components can be Java Server Pages (JSPs), Forms and Widgets and are a part of framework OfBiz components, responsible for rendering the screen, accessing
  • 34.
    25 the database, caching,managing transactions and several low level tasks used by the application components. Business Layer The business layer or application layer defines services provided to the users. The service engine as shown in the figure above is responsible for the invocation and transaction. OfBiz uses several types of services such as SOAP, Java methods, and workflows. Data Layer The data layer is responsible for the database access, storage and providing a common interface to the application layer. Data is accessed in such a way that each entity is provided to the business layer as a set of generic values. Generic values are not type specific, so fields of the entity are accessed by column name. DEVELOPMENT PHASE We will now explain the whole development phase, starting with the skeleton of the plugin. First of all, we need to make a new directory in which the plugin would be created. To do that, we need to open a command window and navigate to the newly created folder. On the command prompt, we have to then run the command “atlas-create-APPLICATION- plugin,” where APPLICATION is the Atlassian application for which we are developing our plugin. In this case, we are working with the JIRA product from Atlassian, hence we have to use the command “atlas-create-jira-plugin.” It will prompt for the following values: groupID, artifactId, version and package. This process will involve creating a lot of directories and subdirectories, XML files and java files. While creating the plugin skeleton, Maven will download all the dependencies from the Internet. It will create a new directory structure as shown in Figure 3. The following describes the files and directories in the JIRA plugin directory:  The pom.xml will be created in the root folder of the plugin. POM [1] stands for Project Object Model definition file.  Directories for the source code files will be in the “src/main” directory, and the test source code files will be in “src/test” directory. All the java files will go into the directory “src/main.”  Directories for the templates, javascript, css, images, etc., will be under resources.
  • 35.
    26 Figure 3. Exampleof pom.xml.  There is a package under test called “it,” which is for integration testing. An atlassian-plugin.xml file in “src/main/resources” directory is the plugin descriptor file; this descriptor tells the application all the plugins and the modules within it and is located in the root of the plugin’s jar file. Here is the sample of atlassian- plugin.xml as an example below. <atlassian-plugin key="${project.groupId}.${project.artifactId}" name="${project.name}" plugins-version="2"> <plugin-info> <description>${project.description}</description> <version>${project.version}</version> <vendor name="${project.organization.name}" url="${project.organization.url}" /> </plugin-info> </atlassian-plugin>
  • 36.
    27 “Atlassian-plugin” element isthe root element for the plugin descriptor. The key must be present and is used to identify the plugin uniquely and its name is used to display the plugin in the menus. The plugin version is to notify that version of plugin which we are using in our application. As explained earlier in this chapter, if we are using ‘version 2’ plugins, then it has to be explicitly mentioned in the code as above. The plugin-info block inside the atlassian-plugin is used to provide more information about the plugin. Description block is used for the details about the plugin, which describes what the plugin has inside it. Version is displayed in the application’s plugin manager and “vendor” supplies the information about the developer of the plugin. The Project Object Model, pom.xml also plays a very important role by describing everything about the dependencies, deployment location, tests, author, and source control that Maven needs to know about the plugin. After successfully building the skeleton, we have to go to the command prompt and navigate to the folder that contains pom.xml and run the command “atlas-run.” This command will be slow if running for the first time, but after completion, we can access our JIRA plugin through the browser with the URL mentioned in the command prompt. VELOCITY OVERVIEW User interacts with JIRA using the web browser. It is a web application and uses OpenSymphony’s WebWork1 [8]. WebWork1 is a MVC framework similar to struts. When an HTTP request is made by a browser at the client-side for a URL, the request is passed to the JIRA web application which returns the HTML. These HTML pages can be static pages or dynamic pages generated at the time of request. JIRA uses JSP for the presentation layer. A lot of business logic in JIRA is implemented in many java files that are present in various places inside the hierarchy of the plugin. We have used a velocity template as a presentation layer while implementing the JIRA plugins. Velocity is a template language which is used to present dynamic information in an easy way to the users of a web site or an application. A collection of java based classes is used to create a separation between the model and view components of the Model-View- Controller (MVC) model. One of the attractive features of velocity is the clear separation between the view classes and the rest of the code with only a simple set of syntax. The output
  • 37.
    28 yield by thevelocity can be HTML or any other content, like SQL, XML, or source code. Velocity also supports servlets directly and has access to java object methods. It can easily transform XML to any other content with the help of Anakia which is an application that allows XML to be processed using velocity instead of XSL (Extensible Stylesheet Language). There are many more features of velocity: It uses the log4J system for logging. Velocity macros allow the building of macros for HTML and scripting elements which are very useful from the developer point of view. It even supports all the major application servers and servlets. The obvious reason of choosing velocity over the JSP in our code implementation is that JSP files are converted in java files, which are compiled to java class files and reside inside the classes folder of the JIRA structure. If there are any changes to be done in the JIRA we have modify or add new JSP files to that class folder. This would need a recompilation of JSP files which would lead to starting up JIRA again and again. Also, if there is any JIRA upgrade to the system, then all the changes are lost and we have to repeat the same steps again to make our code work with the new version of JIRA. On the other hand, velocity templates are passed to a velocity engine code for evaluation and there is no need of restarting JIRA to check the affect after changing anything to the velocity template. Once the plugin is ready, we make a jar file and deploy it to any system without any changes. But if there are some modified JSP files then we have to put those files into a particular directory with the jar file. With the velocity template as a presentation layer, we do not need to do that extra step. Velocity is also much lot easier to learn and implement due to its simple syntax, and it’s even faster than JSP. As we are using xml files to map the action files with the views, the velocity template is best suited to be used with xml. Moreover, using JSP is troublesome because everything is statically typed and working with very a big project with lots of custom tags, which causes the development deployment cycle to slow down considerably, and it also results in inefficiently resolving dependencies. Both JSP and velocity templates have their own advantages and disadvantages. So, if we look from the JIRA plugin development side, we need something which is portable and does not change if there are any upgrades in the JIRA system. With the advent of new technologies and newer versions coming out at a rapid speed, we need something stable for our customers, so the velocity template is the right choice for these kinds of plugins.
  • 38.
    Here is theprocedure actual implementation was started. The main goal is to make avoid any mistakes in the development simple to use and intuitive for the user. The use of a link in the operations issue screen should suffice our need. This push button is linked with the Admin Screen and can be seen in Figure 4. Once this button is clicked, the action will spawn the image, install all packages and send out an email to the user wi manual steps that are still required to complete the setup, ssh key and role of the image. Figure 4. Mock up diagram. MOCKUP DESCRIPTION procedure of the user interaction of the project that was designed before ctual implementation was started. The main goal is to make the design clear avoid any mistakes in the development stage. The push button interface should be extremely simple to use and intuitive for the user. The use of a link in the operations section of issue screen should suffice our need. This push button is linked with the Admin Screen and . Once this button is clicked, the action will spawn the image, install all packages and send out an email to the user with information on how to log manual steps that are still required to complete the setup, ssh key and the Properties of the Figure 4. Mock up diagram. 29 of the user interaction of the project that was designed before design clear in order to . The push button interface should be extremely section of the JIRA issue screen should suffice our need. This push button is linked with the Admin Screen and . Once this button is clicked, the action will spawn the image, install th information on how to log in, display the Properties of the
  • 39.
    30 FUNCTIONALITY OF THESYSTEM Figure 5 is a screen shot of the Admin screen which has a link called “Nimbo Admin Screen” under the global settings. On clicking that link, the Admin screen will be opened and the user can enter relevant information as explained above. This screen is an extension of the JIRA system which gives the user an option of editing and deleting the information. If an administrator of a bug tracking system wants to configure an application to be spawned using EC2, they would need to configure the bug tracking system by entering the relevant information for that project in the screen shown below. This can be done by defining a screen in the bug tracking system that allows the configuration of Nimbo. All the components of this project were created in java in the Eclipse IDE; however, to write a JIRA plugin, one should have a specific development environment as explained above. Figure 5. Screen shot of admin screen. We need to log in to the JIRA bug tracking system with the particular credentials of the user. Next, select the issue to work on, which could be a bug fix, some task to be completed or anything related to the particular project. In JIRA, issues are linked with projects. To fix the bug or the problem, the developer needs a system to work on; on issue screen there will be a Launch button which will spawn an image on demand to the Amazon EC2 server with all the application related software on it. This Launch button is interlinked
  • 40.
    31 with the Adminscreen under the administrator section on the JIRA web application. The Admin screen will have the information of all the projects configured in the JIRA system. So, each project can be mapped to a single role (mysql, LAMP, etc.) from that particular screen. The administrator needs the ability to define the type of version control mechanism used and repository location for the applications that will be used in the image and map it to a project/component. The role of an image is a script that is in a revision control mechanism. The administrator wants to put the url into the source control mechanism where these cookbooks are found. All this information will be stored in the table in the JIRA database. Now every issue related to that particular project which has the “NIMBO” functionality enabled will have the “LAUNCH NIMBO” button as shown in Figure 6. Figure 6. Screen shot of issue screen with launch button. An engineer is looking through a bug record in an issue tracking system. He/she decides to work on this issue but needs to set up a “production like” system. The engineer does that by clicking the “Launch Nimbo” button on the issue screen. Once that is done, the issue key and username of the user that clicked the launch button is sent to a server side Java
  • 41.
    32 program which createsthe image on Amazon EC2. To create the image on EC2, we have used the Amazon Web Services (AWS) Software Development Kit (SDK), a set of developer tools which can be integrated into Java based programs which can then directly talk to Amazon Web Services Frameworks on their servers. The tools in the SDK have the capability of controlling almost every aspect of the image creation process, some of which are the zone which would be used to create the image, the image ID that is to be used for image creation and the keyname of the key that would be used to connect to the machine. The image ID is the ID of the bare bones skeleton which should be configured exactly like the machine that will be created by the AWS SDK tools later on by the user. Once the machine is an exact copy of the image required, there is a set of tools from Amazon called Amazon Machine Identifier (AMI) tools which create the files that will be used by Amazon to create an image. The files are then bundled together in an Amazon Bundle, and the bundle is then uploaded to Amazon S3 servers with a particular bundle or bucket name. Once the files are uploaded to a bucket, these set of files can then be registered by the Amazon API tools and are issued an image ID which can be either a public or private ID. If the user chooses to create a private Image ID, the user is restricting the image creation from the image ID only to the users which use the same credentials that were used to make the image in the first step. If a public image ID is used by users, the user can contribute this image to the vast amount of an already public repository of images so that other users don’t have to worry about creating images, and can use this image ID directly for creating a machine that can be used for their purposes. Eclipse is one of the most popular Integrated Development Environments (IDE) for creating Java based programs and is used extensively by Java programmers. As a result, Amazon Provides the AWS toolkit for Eclipse, which gives users access to AWS SDK for Java, AWS Elastic Benastalk Deployment tools, Amazon SimpleDB Management tools, and Amazon EC2 Management Tools. AWS SDK for Java is a set of libraries which can be integrated into the existing Java projects of different users, and the users can use these libraries to gain access to different classes which can control all the details of the Image creation process which would be explained later in the chapter. AWS Beanstalk Development tools are another set of libraries which can deploy Java based web applications to be deployed to the rigorously tested and stable Beanstalk Development framework
  • 42.
    33 provided by Amazon.Once a web based application is deployed to the AWS Elastic Beanstalk environment, this framework handles all the deployment details so that the programmers of the application can optimize the application to increase user satisfaction and deduce ways to prevent service attacks on the application. These deployment details include: load balancing of application; auto-scaling which can control different resources available to the application like reducing or increasing the amount of bandwidth available to the application depending upon the amount of requests coming in from different users; capacity provisioning; and active application health monitoring which gives users access to look at different metrics like number of user requests, disk space used, and bandwidth allocated to the application at different times. When a user or company signs up to use Amazon EC2 services, each user assigned is given some AWS Credentials. The two important properties in these credentials are the access key and the secret key associated with that account. For using the AWS SDK and all its associated libraries, it is necessary to define the above-mentioned credentials in a file called “AwsCredentials.properties” and then include this file in the root of the Java project. We use the “AwsCredentials” class to instantiate an object (let’s call this object “credentials”) that is given the above-mentioned file “AwsCredentials.properties.” as an input. Now, when we call our Java project, the credentials in this object is authenticated by the Amazon framework, and after the successful authentication, this object now becomes a direct connection to the Amazon framework. We use another class called “AmazonEC2” and instantiate its object, which in our case is “EC2.” This “EC2” object is then made the Amazon EC2 client object by using the “AmazonEC2Client” function of the above mentioned “AmazonEC2” class. Now the “EC2” object can be used to control all the subsequent image creation activities. In our project, we have made a file called “ImageCreation.properties” which is used to store the Image ID of the image that has to be created on Amazon EC2, Keyname of the key that would be used to connect to that machine by using the ssh client, and the interval of time that the server side Java program should wait after issuing the command to create the image. After which this file should call another server side program that connects to this new machine and then installs the software defined in the role associated with the project by the administrator of the company. Once we read all the above-mentioned properties from the “ImageCreation.properties,” we use the
  • 43.
    34 “RunInstancesRequest” class toinstantiate an object that associates the Image ID and the key name read above with the “EC2” object. We then call the “runInstances” function of the “AmazonEC2” class to create a new machine on the Amazon EC2 infrastructure using the above mentioned image ID and key name. The “runinstanceResult” of the “RunInstancesRequest” class then gives us the result of the image creation process. Once the image creation request comes into the Amazon EC2 framework, it assigns an instance ID to this request and then begins with the image creation process. AWS SDK gives us another class called “Reservation” which queries the Amazon EC2 framework during this instance ID assignment process, and once an instance ID is assigned to the incoming request, it provides access to the value of this instance ID. Once the image creation request is given to the Amazon EC2 framework, we have to query the framework about the state of the image creation process, so that we can extract the details like hostname, instance ID, and availability zone, which are details associated with the newly created image. The AWS SDK provides the user with a class “DescribeInstancesRequest” which has to instantiate an object and then run the “setInstanceIds” function which associates the instance ID of the newly created machine with this object. The “EC2” Amazon client object referred to above then calls the “describeInstances” function of the Amazon EC2 class to find out all the above mentioned machine details like hostname, zone, etc., which are used in the email that is sent to the user who originally clicked the “Launch Nimbo” button. Once all the above mentioned actions are done, this Java program then calls another server side Java wrapper program that talks to the JIRA server to fetch the Nimbo related attributes of the project and then uses all these attributes to install the software in the newly created machine, and send an email to the user who clicked the button to create the image. All the above mentioned actions of this server side Java program are logged in a file called “NimboJiraPluginLog.txt.” This file gives us the output of every step in the above mentioned program; for example, it gives out the image ID and the keyname read from the “ImageCreation.properties” file. If that file does not exist, it gives us the output of the failure to read the file. It then gives us details of the user that clicked the “Launch Nimbo” button by giving out the username and the issue key in the log file. Once we enter the AWS SDK portion of the image creation, it displays the big banner of entering that stage. In case we
  • 44.
    35 cannot initialize theSDK, it even gives out the error report for that. Every function associated with the image creation mentioned in the above procedure also logs all its output in the same file. The final output then mentions the hostname and other attributes associated with the newly created machine, and then the last line to call is the other server side wrapper program to install the software. All these log messages will be displayed in Chapter 4, and every line will be explained in detail in that chapter. An image has been started via the click of a button on a web interface. Once the image is fully configured and running, the engineer will receive an email with the IP address and/or name of the machine, along with the location of the ssh key link to download the key of the file server. A clear set of steps will be included in the email communication on how to log in with which utilities. The notification email needs to list the additional manual steps required to fully configure the development and or test environment. The engineer will receive an email after the start up of the image, about the name of the image, Operating system, Version, Architecture, and a list of dependent installed packages and applications. In the next chapter we will show the results of the implementation and their importance in the context of the users and developers.
  • 45.
    36 CHAPTER 4 RESULTS The mainobjective of this project was to give a client company using the Nimbo product the independence to create different kinds of machines automatically without the involvement of any IT department personnel. Once the machine is created, all the software required on that machine would also be installed automatically. Users would get a really powerful machine with a huge amount of resources within a short span of time. Also, a large amount of such machines could be created at the same time, giving that company access to resources beyond their current hardware and software budget resource. A team of developers working on different issues can create “production like” machines that would be used to reproduce the issues that they are trying to solve and once that is done, find the solution by changing either the configuration or their codebase to find the resolution to the problem. This project starts with a user (normally a developer) logging into a JIRA bug tracking system to work on a particular issue. The administrator of the JIRA system ensures that all the user properties like username, email address, First Name, and Last Name are all set before he or she is issued a username and password. The administrator also makes sure the project on which this user is working is mapped to a particular role and all the relevant Nimbo related attributes mentioned above all correctly filled. Unless all the Nimbo related attributes are populated for a project, the project is not “Nimbo enabled” and the developers of the project don’t get a “Launch Nimbo” button on the issue screen. In addition to the user working on projects, we have made another screen in which the administrator logs in to set all the Nimbo related attributes. The screen is called “Admin Nimbo Screen” and can be found under the “Administration” tab of the JIRA bug tracking system. This screen is only visible to users with the “JIRAadministrators” group. Once the administrator logs in, he or she can set these attributes according to the requirements of the project. The administrator would have to talk to the Nimbo programmer to decide the kind of image that needs to be created on the Amazon EC2, so that the Nimbo programmer can make a bare bones skeleton image and give the administrator the image ID for that particular role.
  • 46.
    37 In addition, theadministrator needs to set the APT repository and the cookbook repository location, so that the cookbooks for that role can be checked out once the image is created on the Amazon EC2. The users also have an option to install their custom software by giving the location of the apt repo. The administrator can provide the installers with their custom software by making the apt specific installers and then giving the location in this field. Figure 7 is a screenshot of the Nimbo Admin Screen: Figure 7. Screen shot of add admin screen. All the projects that do not have the Nimbo functionality enabled are present in the dropdown of the Projects category. We have the validation enabled in the above form in which the different fields are checked to make sure that none of them have a blank value. In addition, we check the locations for the apt and cookbook repos to make sure that they have a valid web address value. Now we go through each of the Nimbo related attributes and make sure that we go through all the validations. If the Role is given no value, we get the following validation error in Figure 8 which tells us that the role cannot be blank:
  • 47.
    38 Figure 8. Screenshot of role validation. We even check the attributes to make sure that the apt repo should have a valid web address. We do that by making sure that the apt repo value should begin with an http or an https value. If we do not do that, we get the following error in Figure 9: Figure 9. Screen shot of APT repo validation. We do the same validation for the value of the cookbook repo and make sure it also has a value with a valid Internet address beginning with an http or https. If we do not do that, we get the following validation error in Figure 10: Figure 10. Screen shot of cookbook repo validations. In addition to the initial entering of values by the administrator, we can edit the roles by using the “Edit” button on the right side of the role. Once we click that button, we go to the “Nimbo Edit” screen, which is shown in Figure 11 below:
  • 48.
    39 Figure 11. Screenshot of edit admin screen. All the above mentioned validations for the role, apt repo and cookbook repo designated in the “Admin Nimbo Screen” also apply to the edit screen. We check for the blank values and also valid Internet addresses for those attributes. For deleting the roles, we can use the “Delete” option in front of the roles which takes the user to the “Nimbo Delete” screen. Figure 12 is the screenshot for that screen: Figure 12. Screen shot of delete admin screen. For the image creation Server side wrapper Java program, we have used a log file to list the steps of the entering and exiting stage of a function. In addition, we have used
  • 49.
    40 exceptions to catchall the possible errors that could occur during program execution, all of which are described in this section. Below is the output of the log file after the user clicks the “Launch Nimbo” button: DEBUG - Start NimboIssue DEBUG - End NimboIssue DEBUG - NimboIssue Start doExecute DEBUG - Start Create Image With parameter DEBUG - issukey : NMB-2 DEBUG - username : shilpa DEBUG - NimboIssue End doExecute DEBUG - Run the image creation DEBUG - ---------[java, -jar, c:NIMBOImageCreation.jar, NMB-2, shilpa] DEBUG - ---------Properties are DEBUG - ---------imageid : ami-c0f906a9 DEBUG - ---------keyname : shilpa-ec2-keypair DEBUG - ---------interval : 120000 DEBUG - ---------issukey : NMB-2 DEBUG - ---------username : shilpa DEBUG - ---------=========================================== DEBUG - ---------Welcome to the AWS Java SDK! DEBUG - ---------=========================================== DEBUG - ---------May 6, 2011 6:09:12 PM com.amazonaws.http.HttpClient execute DEBUG - ---------INFO: Sending Request: POST https://ec2.amazonaws.com / Parameters: (ImageId: ami-c0f906a9, Action: RunInstances, MinCount: 1, SignatureMethod: HmacSHA256, AWSAccessKeyId: AKIAI3NLGY5ZWOMITILA, SignatureVersion: 2, Version: 2011-02-28, Signature: fKF+S/0QqPamdWQDnE7ymkGgRAKSA+16ZABsZ7Vnx5M=, Timestamp: 2011-05- 07T01:09:12.107Z, KeyName: shilpa-ec2-keypair, MaxCount: 1,) DEBUG - ---------May 6, 2011 6:09:13 PM com.amazonaws.http.HttpClient handleResponse DEBUG - ---------INFO: Received successful response: 200, AWS Request ID: 45de916c- b7f4-4379-9309-02cc86cf64cb
  • 50.
    41 DEBUG - ---------runinstanceResult: {Reservation: {ReservationId: r-4bfc2f27, OwnerId: 622506625911, RequesterId: null, Groups: [{GroupName: default, GroupId: sg-a011cac9, }], GroupNames: [default], Instances: [{InstanceId: i-6e6e6101, ImageId: ami-c0f906a9, State: {Code: 0, Name: pending, }, PrivateDnsName: , PublicDnsName: , StateTransitionReason: , KeyName: shilpa-ec2-keypair, AmiLaunchIndex: 0, ProductCodes: null, InstanceType: m1.small, LaunchTime: Fri May 06 18:08:35 PDT 2011, Placement: {AvailabilityZone: us- east-1a, GroupName: , Tenancy: default, }, KernelId: aki-6603f70f, RamdiskId: null, Platform: null, Monitoring: {State: disabled, }, SubnetId: null, VpcId: null, PrivateIpAddress: null, PublicIpAddress: null, StateReason: {Code: pending, Message: pending, }, Architecture: null, RootDeviceType: instance-store, RootDeviceName: null, BlockDeviceMappings: null, VirtualizationType: null, InstanceLifecycle: null, SpotInstanceRequestId: null, License: null, ClientToken: , Tags: null, SecurityGroups: [{GroupName: default, GroupId: sg-a011cac9, }], SourceDestCheck: null, }], }, } DEBUG - ---------New Instance ID: i-6e6e6101 DEBUG - ---------May 6, 2011 6:11:13 PM com.amazonaws.http.HttpClient execute DEBUG - ---------INFO: Sending Request: POST https://ec2.amazonaws.com / Parameters: (Action: DescribeInstances, SignatureMethod: HmacSHA256, InstanceId.1: i-6e6e6101, AWSAccessKeyId: AKIAI3NLGY5ZWOMITILA, SignatureVersion: 2, Version: 2011-02- 28, Signature: 92bzskBuyEUYSQbOWxqqxbL0ymcNU9HjYu4HVNu7BLs=, Timestamp: 2011-05-07T01:11:13.793Z, InstanceId.2: i-40fdd02f, ) DEBUG - ---------May 6, 2011 6:11:14 PM com.amazonaws.http.HttpClient handleResponse DEBUG - ---------INFO: Received successful response: 200, AWS Request ID: 241144fa- 1eba-4c95-93ec-3df44e24b8c8 DEBUG - ---------reservation : {ReservationId: r-4bfc2f27, OwnerId: 622506625911, RequesterId: null, Groups: [{GroupName: default, GroupId: sg-a011cac9, }], GroupNames: [default], Instances: [{InstanceId: i-6e6e6101, ImageId: ami-c0f906a9, State: {Code: 16, Name: running, }, PrivateDnsName: ip-10-83-11-163.ec2.internal, PublicDnsName: ec2-50- 19-66-249.compute-1.amazonaws.com, StateTransitionReason: , KeyName: shilpa-ec2- keypair, AmiLaunchIndex: 0, ProductCodes: null, InstanceType: m1.small, LaunchTime: Fri May 06 18:08:35 PDT 2011, Placement: {AvailabilityZone: us-east-1a, GroupName: , Tenancy: default, }, KernelId: aki-6603f70f, RamdiskId: null, Platform: null, Monitoring:
  • 51.
    42 {State: disabled, },SubnetId: null, VpcId: null, PrivateIpAddress: 10.83.11.163, PublicIpAddress: 50.19.66.249, StateReason: null, Architecture: i386, RootDeviceType: instance-store, RootDeviceName: null, BlockDeviceMappings: null, VirtualizationType: paravirtual, InstanceLifecycle: null, SpotInstanceRequestId: null, License: null, ClientToken: , Tags: null, SecurityGroups: [{GroupName: default, GroupId: sg-a011cac9, }], SourceDestCheck: null, }], } DEBUG - ---------Host name : ec2-50-19-66-249.compute-1.amazonaws.com DEBUG - ---------Instance Id : i-6e6e6101 DEBUG - ---------Start sending the issue key and username to server DEBUG - ---------issukey : NMB-2 DEBUG - ---------username : shilpa DEBUG - ---------hostname : ec2-50-19-66-249.compute-1.amazonaws.com DEBUG - ---------instanceId : i-6e6e6101 DEBUG - ----wait over--- 0 Once the user logs in and assigns issues to him or her, the user gets a “Launch Nimbo” button on the top left-hand corner of the issue screen. When that button is clicked, the form calls the program NimboIssue.java, which gives us the output of executing the “doExecute” function. This function calls the java program “CreateAmazon.java” which is called with the value of the username of the user who clicked the “Launch Nimbo” button and with the issue key of that issue screen on which the user is at that moment. In addition, all the other properties like the image ID, keyname of the image, and the delay interval are picked up from the ImageCreation.properties file. The delay interval controls the amount of time needed to wait after issuing the command to make the image and get the hostname of the newly generated machine. We can control these parameters and make them different for different client requirements. The AWS SDK now creates the image and gives all the relevant output until we get the instance ID associated with that image. All the functions visited in the ImageCreation.java are explained in the above chapter. This is the main program which creates the image on the Amazon EC2 using the AWS SDK and then gives control to other programs which first retrieve all the parameters from the JIRA bug tracking system, install software on the newly made machine and then finally send an email to the user that clicked the “Launch Nimbo” button to make that image.
  • 52.
    43 CHAPTER 5 CONCLUSION ANDFUTURE WORK By using Amazon EC2 and expanding the bug tracking system, it is conceivable to automate the process of creating development and testing environments on demand by a simple click of a button. Initially, we were not sure if this idea of automating the systems on demand by the click of button would be possible. We spent a lot of time in discussing and writing the design and requirements for the project. To come up with the tools and design is the most time consuming process for any project and so it was in this project. After studying various articles and research papers, we agreed on the technology needed to implement this project. We chose JIRA as a bug tracking system and Amazon EC2 as our platform in order to achieve a successful project. This project can be far reaching and very useful for any software organization which has a team that wants to reduce the environment set-up time for each developer and understands that the organization will save a significant amount of time. Different people will have different requirements depending on the project they are working on, so their need for a system as quickly as possible with all the software they need to start with the project is different. Keeping this in mind there will be many use cases to implement, to make this project successful for each individual. To start with, we have to focus on implementing only one development/test environment for now. On the JIRA front, we expanded the system and wrote a plugin which will store project specific information into the database which will be helpful in identifying the software requirements to install in the system. Each project is role specific which means one project will have only one specific role and cannot be enrolled in multiple roles. Role is the name of the environment which needs to be installed in the system on the user’s demand. We have the URL information stored for cookbook repo’s and APT repo’s. This information will be useful to check out the recipes from the cookbook and packages that need to be installed from the APT repo and can be identified by the name of the role mapped with the project. All these packages and software will then install in the system by a click on the button from the project issue screen. The user can be a developer who wants to add new features in the
  • 53.
    44 system or atester who wants to fix the bugs. Our aim is to reduce the environment set-up time so that any developer or tester can come in and by just a click of the button spawns an image with the entire project specific software environment on it. Sometimes it takes a week or two to set up an accurate development/testing environment for only a couple of hour’s work, which is sometimes very frustrating and disappointing to the developer and team. Also, having physical machines available immediately is also a problem and waiting for resources to be available is a waste of time for the company and the team. The implementation of automating the systems will help to overcome these kinds of problems which software companies have. We have designed our system with the limitation of having just one operating system, Ubuntu, as a base image for now and we will make our product available for the companies who are using Ubuntu to make their products. Some of the advantages of using Ubuntu are that it is a free operating system and users can download software and applications from huge repositories containing thousands of free software and applications. There are also many commercial alternatives to the free software available, such as OpenOffice and Firefox. Ubuntu is very easy to use and even a non-technical person is very comfortable in using Ubuntu and installing applications and software. One does not have to accept the agreements and simply clicks 10 times to install an application as most of the applications are free, and it is not necessary to worry about the license. Ubuntu is very secure as there are very few viruses on Linux and one need not worry about losing data. Changing the settings are also very simple as you need to be logged in as an administrator and change your settings as you want to for a particular user, you just have to enter administrator password and you are done. There is a lot of help available on the Internet in the form of forums and blogs; if you are looking for any kind of support whether technical or nontechnical, related to Ubuntu. Users don’t need to call customer service and waste their time waiting for a technician to help them. Everything is open and available for free. We can enhance our system by giving the flexibility to the user to choose any operating system he or she wants to work with based upon the requirements of the project. We are thinking of supporting Red Hat Enterprise and Centos Linux operating systems. Once we are comfortable with Linux based systems, we will study and understand the licensing issues with installing a Windows based system and will try to implement that into our project.
  • 54.
    45 For now theuser can have three development systems ready to work on demand with one operating system. We have given the user the flexibility to have three development environments, such as Web Server (apache) installed on the base AMI, at present Ubuntu; PHP Server (PHP and apache); and Database Server (mysql). If a user is working on a project related only to mysql, then he or she can log into the JIRA system and check the issue assigned to him or her and click on the “LAUNCH” button (which is an expansion of the JIRA system) and an image will be spawned on the Amazon EC2 with the base AMI and mysql installed on it. The user will get an email with the detailed information of username, password, key, image ID and other information required to log into the system. The user can then log into the system and directly work on the assigned task as the system will be ready to work on demand with all the project specific software installed on it. By customizing the JIRA, the administrator can store all the information by Web UI interface. We have a Web interface, which has access to only the administrator of the JIRA who is responsible for storing all the information of the project and whose role is required for the project with all the repo information. In big companies, there are hundreds or thousands of projects going on with different teams and departments and all of them are working with different technologies. They require different platforms to work on the projects. Most of the companies today are using JIRA as a bug tracking or as a project management system and also expanding JIRA by writing plugins, and the capability to customize JIRA is an important advantage. We can get a list of all the projects of an organization from JIRA, and with our plugin the JIRA administrator can map the projects with the role (PHP Server [php and apache], Database Server [mysql], and Web Server [apache]) and give the information about the repo’s from where the packages need to install. By this process, we can map different projects with different roles and the developer/tester can get a system on demand by a click of the button. In future, we can provide more roles depending upon the customers’ requirements and improve our system to be more flexible and user friendly. Currently, on the Admin screen, we have provided a combo box with a list of projects and a text box to enter the roles associated with that. We can improve this system as we add more roles into our project and convert the text box for entering the roles into the combo box and map them in such a way that all the configured roles in the cookbook on the server can be fetched directly and we need not manually enter
  • 55.
    46 the name ofthe role any more. Also, based on the selected role, the URL of the cookbook repo can come automatically. Doing this extension of our system will give more accuracy and reduce the chances of making mistakes while manually entering the name of roles associated with the project and cookbook repo. For this version of the project, we have made our system to be simple and work with limited tools and software because this will enable our system to function well without having to add the overhead of other tools. Also, by not making its design complex and complicated, we can test our product on this level more efficiently. The scope of this product is project based. But in JIRA we can subcategorize our projects and in future we can even add more fields in our Admin screen to select the project category, and, based on the project category, we can assign roles and other information. Then, our product will not be like one project mapped to one role. There could be a requirement where one big project is divided into two modules each of them using different technology and platforms. We could then handle these types of uses by enhancing our UI to make it flexible enough to map projects with multiple roles. This will increase the scope of our product usage and the visibility of our project as a product to be used commercially. We could even add custom software and applications if customers want to install their own software and packages on the system. As we are only supporting standard software and packages at this time, we could even extend and modify our system to install custom software as per the requirement of the customer. In the JIRA Admin screen we have given the provision of adding APT repo information to be stored in the database. With the use of that field, we can modify our system to install custom packages, and we would put the packages under APT repo and pull them out from a URL stored into the JIRA database and our program will install them. We can support any custom software and packages as long as they can be accessible through APT. The Advanced Packaging Tool (APT) simplifies the process of installing the software packages, and managing and retrieving the software by automating them. Different customers will have different needs for the development environment of their projects, and we have implemented our product to be simple and open in design so that we can easily extend the system without making much change in the design and implementation part of our present system. There is always room for improvement in any project, and as more and more users and customers use your product, you get to know the
  • 56.
    47 bugs and limitationsin the product. We are open to support and fix any bugs, add new features, and improve present functionality for the future. We started the requirements and design of the project to explore the idea of an on demand, scalable, automated framework that automates the set up of development environments for the software industry. We studied a lot of research papers, which proved to be helpful in deciding the technologies and tools to be used in our project. We have used the JIRA bug tracking system with the Amazon EC2 as the platform to build our product. We could look for other bug tracking systems other than JIRA to implement the concept of spawning an image on a click of the button. We are calling the Java program when the user clicks on a button for image creation, so we can implement our product just by extending any other bug tracking system, which has all the functionality we require completing the process as in JIRA. That will require some work at the Web UI interface front with no changes in the server program and client program. With the advent of new technologies and products, we will keep our project updated and will incorporate future modifications and improvements, whether they are suggestions from the customer or any further changes which can make our system more efficient and powerful. We will continue to work on this project to make a successful product in the software market that will save a lot of time for developers and testers and automate the process of installing the software and applications to set up the development environment.
  • 57.
    48 REFERENCES [1] APACHE SOFTWAREFOUNDATION, Introduction to the POM. Apache Maven Project, http://maven.apache.org/ guides/introduction/introduction-to-the-pom.html, accessed June 2011. [2] J. BROBERG , R. BUYYA AND Z. TARI, MetaCDN: Harnessing ‘storage clouds’ for high performance content delivery, J. Net. Comp. App., 32 (2009), pp. 1012-1022. [3] Q. W. FLEMING AND J. M. KOPPELMAN, Earned Value Project Management, Project Management Institute, Inc., Pennsylvania, 2000. [4] N. HANAKAWA AND K. OKURA, A project management support tool using communication for agile software development, Proceedings of the 11th Asia-Pacific Software Engineering Conference, Japan, 2004. [5] N. JALBERT, AND W. WEIMER, Automated duplicate detection for bug tracking systems, Proceedings of the Dependable Systems and Networks Conference, Alaska, 2008. [6] D. E. JONES, The open for business project: Entity engine guide. Ofbiz, http://ofbiz. apache.org/ docs/entity.html, accessed June 2011. [7] W. LU, J. JACKSON AND R. BARGA, AzureBlast: A case study of developing science applications on the cloud, Proceedings of the 1st Workshop on Scientific Cloud Computing, Chicago, 2010. [8] N.A., OpenSymphony RIP 2000-2011. http://www.opensymphony.com/, accessed June 2011. [9] J. NOLEN, JIRA architectural overview. JIRA 4.4 Early Access Program, http://confluence.atlassian.com/display/JIRA/JIRA +Architectural +Overview, accessed June 2011. [10] J. NOLEN, PicoContainer and JIRA. JIRA 4.4 Early Access Program, http://confluence .atlassian.com/display/JIRA/ PicoContainer+and+JIRA, accessed June 2011. [11] D. NURMI, R. WOLSKI, C. GRZEGORCZYK, G. OBERTELLI, S. SOMAN, L. YOUSEFF AND D. ZAGORODNOV, The eucalyptus open-source cloud-computing system, Proceedings of the International Symposium on Cluster Computing and the Grid, China, 2009. [12] OSGI ALLIANCE, OSGI main home page. http://www.osgi.org/Main/Home Page, accessed June 2011. [13] PICOCONTAINER, Dependency injection. PicoContainer, http://picocontainer.org /injection.html, accessed June 2011. [14] S. RICHTER AND C. THIELE, Benchmarking the cloud - A comparison of lokad on azure and elastic MapReduce. Unpublished report, 2010. [15] N. SERRANO AND I. CIORDIA, Bugzilla, ITtacker, and other Bug Trackers, IEEE Software, 22 (2005), pp.11-13.
  • 58.
    49 [16] S. TILLEY,J. GERDES, T. HAMILTON, S. HUANG, H. MULLER AND K. WONG, Adoption challenges in migrating to web services, Proceedings of the Fourth International Workshop on Web Site Evolution, Washington D.C., 2002. [17] M. A. VOUK, Cloud computing–Issues, research and implementations, J. Comp. Info. Tech – CIT, 16 (2008), pp. 235-246. [18] G. WANG AND T.S.E. NG, The impact of virtualization on network performance of Amazon EC2 data center, INFOCOM Proceedings, San Diego, 2010. [19] C. WEISS, R. PREMRAJ, T. ZIMMERMANN AND A. ZELLER, How long will it take to fix this bug? Proceedings of the Fourth International Workshop on Mining Software Repositories, Minnesota, 2007. [20] WIKIPEDIA, Cloud computing. Wikipedia, http://en.wikipedia.org/wiki/Cloud _computing, accessed June 2011. [21] WIKIPEDIA, Content delivery network. Wikipedia, http://en.wikipedia.org/wiki/Content _delivery_ network, accessed June 2011. [22] WIKIPEDIA, OpenSymphony. Wikipedia, http://en.wikipedia.org/wiki/OpenSymphony, accessed June 2011.
  • 59.
  • 60.
    51 Acronyms GUI Graphical UserInterface AWS Amazon Web Services EC2 Elastic Cloud Computing S3 Simple Storage Service IDE Integrated Development Environment AMI Amazon Machine Identifier SaaS Software as a Service IaaS Infrastructure as a Service