This document discusses testing a secure framework for downloading and testing secure world APIs. It covers using a test suite from STMicroelectronics to test GlobalPlatform APIs, functional testing versus other types of testing, and integrating tests with LAVA and build/baseline systems. An agenda is proposed to discuss topics like the GlobalPlatform test suite fees, performing security and stress testing, and whether the project will be standalone.

1. Thur-6-Mar, 5:05pm, Joakim Bech
LCA14-418: Testing a secure
framework

2. • Background
• GlobalPlatform and XML-files
• Functional testing vs other testing
• LAVA, Builds & Baselines
• Open discussion: Pros and cons
Agenda

3. • There is no test framework for download that test secure world APIs
• Test suite coming from ST-Ericsson which is now owned by STMicroelectronics
• Testing secure world GlobalPlatform APIs
Background

4. • GlobalPlatform have a list of Qualified Test Tools covering much more than just
secure APIs
• Comprehensive functional test of a TEE framework
• Initial TEE Configuration v1.0
• Trusted Core Framework API
• Trusted Storage API
• Cryptographic operations API
• Time API
• Arithmetical API
• Compliance Test Suite (including all minor updates for 2 years) - $6 200 USD!
• A set of XML-files that specifies how functions should be called and what
result to expect
• Linaro don’t want to compete with existing test firms (Galitt, FIME)
• Using a test suite from Linaro != GlobalPlatform qualified
GlobalPlatform Test Suite

5. • Using GoogleTest framework
• Not to reinvent the wheel
• Able to launch a single test, several times, in random order,...
• Used to test infrastructure cost and corner-cases
• Example of some of the tests:
• Performance: cost of going to the secure environment
• Tests for memory leakage detection / garbage collector
• Static TA testing (built-in extensions)
• ...
Extended Functional Testing

6. • The GlobalPlatform based test suite has good coverage on functional testing, but
lacks doing security- and stress-testing for example
• Not in scope initially
• Timing attacks: For how long will function X run?
• Power analysis: Variations in power consumption could leak information
• Fuzz testing: like Trinity for example (Linux kernel syscall testing)
• Stress tests: Is everything still stable as the load increases?
• Multi-TAs / Multi-Threaded
• Involving HW
• SMP
Only functional testing?

7. LAVA and Builds and Baseline integration
Build CI job Test suites
jenkins
lava
lava job
test results

8. Do we need ...
• … specialized hardware?
• … specialized network setup?
• … tests involving more than one hardware device in coordination (multinode
jobs)?
• … ?
LAVA - what are the requirements?

9. • Example topics that could be discussed
• Does it sound like the current plan seems sane?
• Will the fee to get GlobalPlatform XML-files be a problem?
• What about side channel attacks?
• How about other TEEs? How to test those?
• Performance testing?
• Will it be a standalone project?
• ...
Open discussion

10. More about Linaro Connect: http://connect.linaro.org
More about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/
Linaro members: www.linaro.org/members