The document discusses AdaCore's tools for dynamic software verification using GNAT, including GNATemulator for target emulation, GNATcoverage for coverage analysis, GNATtest for unit test generation, and GNATdashboard for monitoring the verification process. It provides examples of analyzing coverage at the source code and object code levels, generating unit tests, and integrating results with partners' tools. AdaCore aims to help customers address verification challenges through collaborative, open, and dependable approaches.

2. Dynamic Software
Verification with GNAT
Martyn Pike
October 1st 2015

3. What will you learn today ?
• AdaCore are Inventive
– GNAT Technologies go beyond the Implementation Phase
• AdaCore are Collaborative
– Partners and Third-party Technology for Dynamic Verification
• AdaCore are Open
– Work with Open Source, Open APIs and Open Standards
• AdaCore are Dependable
– Committed to helping our customers address their challenges

4. Dynamic Verification Challenges
• Run Cross Compiled Code on Target Hardware
– Solution : Target Architecture Emulation
• Achieve a Planned Level of Code Coverage
– Solution : Coverage Analysis Tool
• Managing the Verification Environment
– Solution : Unit Test Harness Generator
• Monitoring the Verification Process
– Solution : Monitoring Technical Debt

5. GNAT for Dynamic Verification
TARGET EMULATION COVERAGE ANALYSIS
UNIT TEST GENERATION
GNATemulator GNATcoverage
GNATtest MANAGEMENTGNATdashboard

6. with Ada.Text_IO;
with Simple;
procedure Main is
Even_Count : Integer := 0;
Odd_Count : Integer := 0;
begin
Simple.SubP(
Loop_Count => 21,
Even_Count => Even_Count,
Odd_Count => Odd_Count
);
Ada.Text_IO.Put_Line("Evens : " & Even_Count'Img);
Ada.Text_IO.Put_Line("Odds : " & Odd_Count'Img);
end Main;
package Simple is
procedure SubP(
Loop_Count : in Integer;
Even_Count : out Integer;
Odd_Count : out Integer
) with
Pre => (Loop_Count > 0),
Post => ((Even_Count >= 0) and (Odd_Count >= 0));
end Simple;
package body Simple is
procedure SubP(
Loop_Count : in Integer;
Even_Count : out Integer;
Odd_Count : out Integer
) is
begin
Even_Count := 0;
Odd_Count := 0;
For_Loop:
for I in 1..Loop_Count loop
if I > 20 then
if I mod 2 = 0 then
-- Unreachable when Loop_Count = 21 !
Even_Count := Integer'Succ(Even_Count);
end if;
elsif I mod 2 = 0 then
Even_Count := Integer'Succ(Even_Count);
else
Odd_Count := Integer'Succ(Odd_Count);
end if;
end loop For_Loop;
end SubP;
end Simple;

7. GNATemulator
• QEMU
– Open Source Processor Emulator
• I/O Connections
– Ada.Text_IO
– GNATbus
– GDB Connection
• No Instrumentation
• Actual Target Object Code
EXE
PowerPC -> x86 Translation
Host Platform

8. GNATemulator Platform Support
• VxWorks 6
• VxWorks 653
• VxWorks 7 (coming soon)
• PowerPC ELF Bareboard
• LEON ELF Bareboard
• ARM ELF Bareboard (coming soon)

9. GNATemulator Demonstration

10. • Triage by Verification Engineer
• Reproduce Defects
GNATemulator Debugging
EXE
GNATemulator
GDB
SERVER
TCP

11. GNATemulator Debugging

12. GNATcoverage
• Coverage Analysis
• Multiple Modes of Operation
– Run with Capture of Execution Trace Data
– Conversion of IEEE-ISTO 5001-2003 (Nexus) Trace Data
– Coverage Analysis of Execution Trace Data
• Source Code Level
• Object Code Level

13. GNATcoverage Platform Support
• Intel 32-bit and 64-bit
• PowerPC ELF Bareboard
• LEON 2 & LEON 3 ELF Bareboard

14. Build Considerations
• Compiler Command Line Switches
– “-g”
– “-fpreserve-control-flow”
• Control Optimizers for precise SLOC info
– “-fdump-scos”
• Source Coverage Obligation in ALI files
• Support for Optimizations (upto -O1)
• Inlining Allowed (-gnatn)
• No External Libraries Needed

15. GNATcoverage Execution

16. GNATcoverage Analysis Levels
• Source Level
– gnatcov coverage --level=stmt
– gnatcov coverage --level=stmt+decision
– gnatcov coverage --level=stmt+mcdc
• Object Level
– gnatcov coverage --level=insn
– gnatcov coverage --level=branch

17. GNATcoverage Analysis Formats
• xcov
– Annotated Sources in Text Format
• report
– Textual Summary
• HTML
– Colours, Sortable Columns and Per-project indexes

18. GNATcoverage – Source Statement
• gnatcov coverage --level=stmt --annotate=dhtml

19. GNATcoverage – Source Statement

20. GNATcoverage – Source Decision
• gnatcov coverage --level=stmt+decision --annotate=dhtml

21. GNATcoverage – Source Decision

22. H:tech_day_eucodesrcsimple.adb:
80% of 10 lines covered
Coverage level: stmt+decision
1 .: package body Simple is
2 .:
3 .: procedure SubP(
4 .: Loop_Count : in Integer;
5 .: Even_Count : out Integer;
6 .: Odd_Count : out Integer
7 .: ) is
8 .: begin
9 .:
10 +: Even_Count := 0;
11 +: Odd_Count := 0;
12 .:
13 +: For_Loop:
14 +: for I in 1..Loop_Count loop
15 .:
16 +: if I > 20 then
17 .:
18 !: if I mod 2 = 0 then
19 .:
20 .: -- Unreachable when Loop_Count = 21 !
21 -: Even_Count := Integer'Succ(Even_Count);
22 .:
23 .: end if;
24 .:
25 +: elsif I mod 2 = 0 then
26 .:
27 +: Even_Count := Integer'Succ(Even_Count);
28 .:
29 .: else
30 .:
31 +: Odd_Count := Integer'Succ(Odd_Count);
32 .:
33 .: end if;
34 .:
35 .: end loop For_Loop;
36 .:
37 .: end SubP;
38 .:
39 .: end Simple;

23. ** COVERAGE REPORT **
===========================
== 1. ASSESSMENT CONTEXT ==
===========================
Date and time of execution: 2015-09-15 14:36:12.00
Tool version: XCOV 1.3.1 (20150118)
Command line:
H:Tech_Day_EUtoolsgnatcovbingnatcov.exe coverage --level=stmt --
annotate=report --target=powerpc-elf -Pex3.gpr exeex3ex3.trace
Coverage level: stmt
Trace files:
exeex3ex3.trace
program: exeex3ex3
date : 2015-09-15 12:35:06
tag :
============================
== 2. COVERAGE VIOLATIONS ==
============================
2.1. STMT COVERAGE
------------------
simple.adb:21:16: statement not executed
1 violation.
=========================
== 3. ANALYSIS SUMMARY ==
=========================
1 STMT violation.
** END OF REPORT **

24. GNATcoverage – Object Instruction
• gnatcov coverage --level=insn --annotate=dhtml

25. GNATcoverage – Object Instruction

26. GNATcoverage – Object Instruction

27. GNATcoverage – Object Branch
• gnatcov coverage --level=branch --annotate=dhtml

28. GNATcoverage – Object Branch

29. GNATtest
• Unit Test Harness Generation
• Included in GNAT Pro
• Based on the Open Source AUnit Framework
• Available for Native and Cross Compiled Code
• Increase Productivity Through Code Generation

30. GNATtest Work Flow
HARNESS
TEST CASES
UUT
package Simple is
procedure SubP(
Loop_Count : in Integer;
Even_Count : out Integer;
Odd_Count : out Integer
) with
Pre => (Loop_Count > 0),
Post => ((Even_Count >= 0) and (Odd_Count >= 0));
end Simple;

31. GNATtest Work Flow
• gnattest -v --RTS=ravenscar-full-prep -Pex4.gpr
• Generates Test Case Skeletons and a Harness
• Very Flexible and allows for Configuration Control
• Aware of User Defined Test Case Code
• Clearly identifies what will NOT survive a Generation

32. GNATtest –Test Case Generation
package Simple.Test_Data is
-- begin read only
type Test is new AUnit.Test_Fixtures.Test_Fixture
-- end read only
with null record;
procedure Set_Up (Gnattest_T : in out Test);
procedure Tear_Down (Gnattest_T : in out Test);
end Simple.Test_Data;
-- This package has been generated automatically by GNATtest.
-- Do not edit any part of it, see GNATtest documentation for more details.
-- begin read only
with Gnattest_Generated;
package Simple.Test_Data.Tests is
type Test is new GNATtest_Generated.GNATtest_Standard.Simple.Test_Data.Test
with null record;
procedure Test_SubP_75ecda (Gnattest_T : in out Test);
-- simple.ads:3:4:SubP
end Simple.Test_Data.Tests;
-- end read only

33. GNATtest –Test Case Generation
-- This package is intended to set up and tear down the test environment.
-- Once created by GNATtest, this package will never be overwritten
-- automatically. Contents of this package can be modified in any way
-- except for sections surrounded by a 'read only' marker.
package body Simple.Test_Data is
X : constant Integer := 20;
procedure Set_Up (Gnattest_T : in out Test) is
pragma Unreferenced (Gnattest_T);
begin
null;
end Set_Up;
procedure Tear_Down (Gnattest_T : in out Test) is
pragma Unreferenced (Gnattest_T);
begin
null;
end Tear_Down;
end Simple.Test_Data;

34. GNATtest –Test Case Generation
-- This package has been generated automatically by GNATtest.
-- You are allowed to add your code to the bodies of test routines.
-- Such changes will be kept during further regeneration of this file.
-- All code placed outside of test routine bodies will be lost. The
-- code intended to set up and tear down the test environment should be
-- placed into Simple.Test_Data.
with AUnit.Assertions; use AUnit.Assertions;
package body Simple.Test_Data.Tests is
-- begin read only
procedure Test_SubP (Gnattest_T : in out Test);
procedure Test_SubP_75ecda (Gnattest_T : in out Test) renames Test_SubP;
-- id:2.2/75ecda11d3241da6/SubP/1/0/
procedure Test_SubP (Gnattest_T : in out Test) is
-- simple.ads:3:4:SubP
-- end read only
pragma Unreferenced (Gnattest_T);
begin
AUnit.Assertions.Assert
(Gnattest_Generated.Default_Assert_Value,
"Test not implemented.");
-- begin read only
end Test_SubP;
-- end read only
end Simple.Test_Data.Tests;

35. GNATtest –Test Case Completion
Even_Count, Odd_Count : Integer;
begin
Simple.SubP(
Loop_Count => 21,
Even_Count => Even_Count,
Odd_Count => Odd_Count
);
Assert(((Even_Count = 10) and (Odd_Count = 10)),"Loop_Count => 21");
Simple.SubP(
Loop_Count => 22,
Even_Count => Even_Count,
Odd_Count => Odd_Count
);
Assert(((Even_Count = 11) and (Odd_Count = 10)),"Loop_Count => 22");

36. GNATtest – Harness Generation
• Hierarchy of GNAT project files
• Integrates Test Case Code with AUnit Framework
• Flexibility
– Build an individual executable per test case
– Build a single test case which runs all test cases

37. GNATtest – Harness Execution

38. GNATtest – Coverage Analysis

39. GNATtest – Coverage Analysis

40. GNAT Programming Studio (GPS)

41. GNATdashboard
• Monitoring Dynamic Verification
– GNAThub Data Aggregator
– Open Python API
– Roadmap [Q1 2016]
• Integration with GNATcoverage
• Support for GNAT Pro Cross Compilers

42. GNAThub Data Aggregator
project Hub5 is
for Object_Dir use "outex5decision";
package Dashboard is
for Plugins use ("GNATCoverage");
end Dashboard;
end Hub5;
• gnathub --exec my_script.py -P hub5.gpr
– Parses xcov Coverage Reports found in Object_Dir
– Builds SQLite snapshot database and executes “my_script.py”

43. Open Python API
• Generated SQLite Database can be accessed by Python script
• GNAThub API
• Implemented in Ada and exported to Python
• Integrate the GNAThub Data into customer QA environment
• Databases Delivered from External Teams or put under SCM

45. Partner Solutions
• GNATcoverage Integration
• On-Chip Analyser
• Freescale MPC5634M SoC
• IEEE-ISTO 5001-200 (Nexus)

46. Partner Solutions
• VectorCAST/Ada
– Ada 2012 support Q1’2016
• VectorCAST/Cover
– Joint Webinar on CodePeer 3.0 Integration
• Fully Integrated with GNATemulator supporting VxWorks

47. Partner Solutions
• Rapita Verification Suite (RVS)
– On-target timing analysis (RapiTime)
– RTOS-independent visualization of scheduling and event
tracing (RapiTask)
– On-target code coverage measurement (RapiCover)
• Tools are actually built using Ada, including some Ada 2012

48. • Integration with GNAThub Open Python API
Partner Solutions

49. What you have learnt today ?
• AdaCore are Inventive
– GNAT Technologies go beyond the Implementation Phase
• AdaCore are Collaborative
– Partners and Third-party Technology
• AdaCore are Open
– Open Source, Open APIs and Open Standards
• AdaCore are Dependable
– Committed to helping our customers address their challenges

50. Resources
• GNATcoverage
– http://www.adacore.com/gnatcoverage/
• GNATemulator
– http://www.adacore.com/gnatemulator
• GNATtest
– http://www.adacore.com/gnatpro/toolsuite/gnattest/
• GNATdashboard
– http://www.adacore.com/gnatpro/toolsuite/GNATdashboard/