The document outlines Arcadia Inc.'s software development process for e-assessment projects. It details the phases of design, implementation, testing and deployment. It also describes Arcadia's emphasis on quality practices like security testing, automation and infrastructure. For testing in Scrum, it explains the regular sprint process and regression testing before release. Key roles in testing include the QA manager, release manager and the team.
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Quality Practices in e-Assessment Development
1. London, March 16th, 2016
As true as steel to your desire
You come with just an idea -
we make great software for you!
Trust
Teamwork
Transparency
Quality practices
for e-Assessment development
We’ve been building many different software systems including e-Assessment solutions.
We all understand clearly – quality is very important for e-assessment products.
Imagine a wrong mark is assigned to a candidate because of a bug in the product.
Imagine a national exam and many candidates cannot take exams because the product is down because of massive load.
Imagine a bad guy who steals questions before the exam.
We cannot afford any those situations.
No one from our clients would accept those situations.
Fortunately, there are best practices and procedures to avoid the issues and assure the quality. Well, at least, to minimize the probability to an acceptable level. And we will share them with you.
A typical e-assessment development project consist of several phases: Design, Implementation, Testing and Deployment - we’ll talk what should be done to manage quality on each phase.
Identify the capacities the system should handle: how many candidates can sit exams simultaneously, how many exam scripts can the system handle, how many items a typical exam can have.
Identify the locations where users and candidates will be – this helps to plan how the system will be deployed.
Identify the sensitive data stored and processed by the system: questions and answers, candidates personal data. Plan to protect this data.
Plan the system architecture accordingly.
One of key things is to agree what is minimum quality standards for every single piece of functionality. It is often named Definition of Done.
The team agrees a short list of testable statements and then every piece of work done by the team must pass the DoD.
The list varies from project to project; the following is a sample of it:
Developer tests own code.
Other developer reviews code written by the developer.
Developers test the product for security issues – it is very efficient to do whitebox mode. An experienced developer can see possible vulnerabilities just looking into the code.
Developers test their work for the main vulnerabilities (OWASP Top 10). This is mainly manual work to find ways the application can be attacked and try to exploit the vulnerabilities.
Using of automated scanners (e.g. Burp) also helps.
Even though we assured quality for a single piece of functionality it is not enough. We have to maintain the quality over time.
Automation is key here.
Team
A typical e-assessment project involves a number of developers and testers – from 5 to 15 in our experience. The work is arranged into iterations called sprints. A typical sprint is 2-3 weeks.
Every developer works on some piece of functionality and integrate their work into a common source code repository.
There is a process called Continuous Integration, which builds the product after each commit to the source code repository. It not only builds, but also runs automated tests, so every team member can clearly see the product health – if there is no build errors and if the tests pass.
Every night or more often, the product is automatically deployed to Alpha testing environment. The team uses this environment to test the product. Alpha is not stable – it is just what we have in source control repository at the moment. In some cases it can even not start.
After sprint is competed, the product is deployed to stable Beta environment. Sometimes it is called UAT environment. Here testers run extensive manual and automated tests and regression test the product. After successful regression testing the product tis ready to go Live.
Unit tests
CI
Automated deployment
Scrum->no documentation, but->test cases, Test plan