This document discusses proposed improvements to development and test infrastructure at a Q1 2016 hackfest. It outlines 6 improvements: 1) including more unit testing in verify jobs; 2) using clearer multi-scenario naming conventions; 3) allowing local testing of code changes; 4) updating dashboards automatically; 5) using a node label parameter plugin for lab resource pooling; and 6) implementing an ELK logging system.

1. Development and Test Infrastructure
Q1 2016 hackfest, San Francisco
test infra team

2. Improvement 1
Jenkins-ci, what should be included in verify job
The definition
little thing is done in verify job(e.g., https://build.opnfv.org/ci/job/functest-verify-master/720/console)
Unit test is needed in each code development project written by themselves according to their own needs, a developer has
to pas through the unittest before he/she puts up a patch

3. Improvement 2
Multi-scenario names
What does this mean?
[installer]-[SDN controller]-[?]-[HA or not]-[pod]-[branch]
Question: Does this satisfy our requirement?
Does this confuse somebody?
Do we have reservation for future scenario expand?

4. Improvement 3
A tip to develop code more efficiently without too much reviews for one test case
Problem description
The test cases running progress will not included in the unittest module(i.e., verify job), since it will last long
if a developer puts up an patch to make some changes, it’s hard for himself/herself and the reviewers to
check if it works well, so the review, merge process is largely depends on the reviewer’s personal knowledge
and experience.
solutions
Local test with only checking the functions which are changed.
Local test by setting up his/her own installer env and run the test case, which is hard for general developers
and need some negotiation with the reviewers, besides, the installers’ code is always changing.
Integration with the community installer and run the test case once a patch set put up related to its code

5. Improvement 3
A tip to develop code more efficiently without too much reviews for one test case
If the test case is a little complicated, u can use the following method
Define a patchset parameter
for debugging
(in the test case Jenkins job
and code repo)
update the patchset
parameter in the Jenkins job
git fetch the updated code
repo
run the Jenkins job and check
the test results
Add reviewers and wait to
merge
Amend the patch and put up
a new patch set
success
fail

6. Improvement 4
dashboard
MongoDB is used nowadays for most of the projects.
update the dashboard automatically once the graph function changes, or
maybe re-draw the dashboard everyday or few hours , this seems not work
well.
The dashboard may not satisfy everyone’s requirement, such as metric test
results, 3D graphs.
The graphs may not seems cool enough especially compared to some
expert graph and dashboard builder

7. Improvement 5
lab resource pool
NodeLabel Parameter Plugin already installed
Just to configure the job to the node with correct label parameter
Not widely used yet
For virtual env, the VM management tool Nodepool can be considered

8. Improvement 6
Log system
ELK is recommended, it can be easily set up and used.

9. Thanks