Thur-6-Mar, 5:05pm, Joakim Bech
LCA14-418: Testing a secure
framework
• Background
• GlobalPlatform and XML-files
• Functional testing vs other testing
• LAVA, Builds & Baselines
• Open discussion: Pros and cons
Agenda
• There is no test framework for download that test secure world APIs
• Test suite coming from ST-Ericsson which is now owned by STMicroelectronics
• Testing secure world GlobalPlatform APIs
Background
• GlobalPlatform have a list of Qualified Test Tools covering much more than just
secure APIs
• Comprehensive functional test of a TEE framework
• Initial TEE Configuration v1.0
• Trusted Core Framework API
• Trusted Storage API
• Cryptographic operations API
• Time API
• Arithmetical API
• Compliance Test Suite (including all minor updates for 2 years) - $6 200 USD!
• A set of XML-files that specifies how functions should be called and what
result to expect
• Linaro don’t want to compete with existing test firms (Galitt, FIME)
• Using a test suite from Linaro != GlobalPlatform qualified
GlobalPlatform Test Suite
• Using GoogleTest framework
• Not to reinvent the wheel
• Able to launch a single test, several times, in random order,...
• Used to test infrastructure cost and corner-cases
• Example of some of the tests:
• Performance: cost of going to the secure environment
• Tests for memory leakage detection / garbage collector
• Static TA testing (built-in extensions)
• ...
Extended Functional Testing
• The GlobalPlatform based test suite has good coverage on functional testing, but
lacks doing security- and stress-testing for example
• Not in scope initially
• Timing attacks: For how long will function X run?
• Power analysis: Variations in power consumption could leak information
• Fuzz testing: like Trinity for example (Linux kernel syscall testing)
• Stress tests: Is everything still stable as the load increases?
• Multi-TAs / Multi-Threaded
• Involving HW
• SMP
Only functional testing?
LAVA and Builds and Baseline integration
Build CI job Test suites
jenkins
lava
lava job
test results
Do we need ...
• … specialized hardware?
• … specialized network setup?
• … tests involving more than one hardware device in coordination (multinode
jobs)?
• … ?
LAVA - what are the requirements?
• Example topics that could be discussed
• Does it sound like the current plan seems sane?
• Will the fee to get GlobalPlatform XML-files be a problem?
• What about side channel attacks?
• How about other TEEs? How to test those?
• Performance testing?
• Will it be a standalone project?
• ...
Open discussion
More about Linaro Connect: http://connect.linaro.org
More about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/
Linaro members: www.linaro.org/members

LCA14: LCA14-418: Testing a secure framework

  • 1.
    Thur-6-Mar, 5:05pm, JoakimBech LCA14-418: Testing a secure framework
  • 2.
    • Background • GlobalPlatformand XML-files • Functional testing vs other testing • LAVA, Builds & Baselines • Open discussion: Pros and cons Agenda
  • 3.
    • There isno test framework for download that test secure world APIs • Test suite coming from ST-Ericsson which is now owned by STMicroelectronics • Testing secure world GlobalPlatform APIs Background
  • 4.
    • GlobalPlatform havea list of Qualified Test Tools covering much more than just secure APIs • Comprehensive functional test of a TEE framework • Initial TEE Configuration v1.0 • Trusted Core Framework API • Trusted Storage API • Cryptographic operations API • Time API • Arithmetical API • Compliance Test Suite (including all minor updates for 2 years) - $6 200 USD! • A set of XML-files that specifies how functions should be called and what result to expect • Linaro don’t want to compete with existing test firms (Galitt, FIME) • Using a test suite from Linaro != GlobalPlatform qualified GlobalPlatform Test Suite
  • 5.
    • Using GoogleTestframework • Not to reinvent the wheel • Able to launch a single test, several times, in random order,... • Used to test infrastructure cost and corner-cases • Example of some of the tests: • Performance: cost of going to the secure environment • Tests for memory leakage detection / garbage collector • Static TA testing (built-in extensions) • ... Extended Functional Testing
  • 6.
    • The GlobalPlatformbased test suite has good coverage on functional testing, but lacks doing security- and stress-testing for example • Not in scope initially • Timing attacks: For how long will function X run? • Power analysis: Variations in power consumption could leak information • Fuzz testing: like Trinity for example (Linux kernel syscall testing) • Stress tests: Is everything still stable as the load increases? • Multi-TAs / Multi-Threaded • Involving HW • SMP Only functional testing?
  • 7.
    LAVA and Buildsand Baseline integration Build CI job Test suites jenkins lava lava job test results
  • 8.
    Do we need... • … specialized hardware? • … specialized network setup? • … tests involving more than one hardware device in coordination (multinode jobs)? • … ? LAVA - what are the requirements?
  • 9.
    • Example topicsthat could be discussed • Does it sound like the current plan seems sane? • Will the fee to get GlobalPlatform XML-files be a problem? • What about side channel attacks? • How about other TEEs? How to test those? • Performance testing? • Will it be a standalone project? • ... Open discussion
  • 10.
    More about LinaroConnect: http://connect.linaro.org More about Linaro: http://www.linaro.org/about/ More about Linaro engineering: http://www.linaro.org/engineering/ Linaro members: www.linaro.org/members