Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

System Verilog Functional Coverage

14,877 views

Published on

Paper on my System Verilog Functional Coverage and SVA assertions work, from Boston SNUG 2005

  • Justin Sinclair has helped thousands of women get their Ex boyfriends back using his methods. Learn them at:  http://t.cn/R50e2MX
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Discover a WEIRD trick I use to make over $3500 per month taking paid surveys online. read more... ★★★ http://ishbv.com/surveys6/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Just got my check for $500, Sometimes people don't believe me when I tell them about how much you can make taking paid surveys online... So I took a video of myself actually getting paid $500 for paid surveys to finally set the record straight. I'm not going to leave this video up for long, so check it out now before I take it down! ♣♣♣ https://tinyurl.com/make2793amonth
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

System Verilog Functional Coverage

  1. 1. A Unique Functional Coverage Flow using SystemVerilog and NTB Richard Raimi ARM, Inc rraimi@arm.com Dennis Strouphauer Synopsys, Inc dstroup@synopsys.com ABSTRACTThere are two constructs in SystemVerilog that support functional coverage: the coveredproperty and the covergroup. At ARM, Inc. in Austin, Texas, we wanted to use SystemVerilogfunctional coverage on a next generation processor design; however, only the covered propertyand not the covergroup was going to be supported by Synopsys in the time frame of our project.Feeling that both were needed, we chose to write our functional coverage in SystemVerilog,using both the covered property and the covergroup, and translate the SystemVerilogcovergroups to NTB / Vera ‘coverage_groups’, a similar, but not identical, construct. In thisway, we were able to train our team in SystemVerilog, use the code we wrote in the present, andallow ourselves the option of reusing it in the future, when the covergroup would be fullysupported in VCS. In this paper, we describe this translation process, as well as the pros andcons of functional coverage in both NTB / Vera and SystemVerilog.
  2. 2. Table of Contents1.0 Introduction: Functional Coverage ....................................................................................32.0 Functional Coverage Constructs in SystemVerilog and Vera / NTB ................................32.1 NTB coverage_groups .......................................................................................................82.2 Bind Directives ..................................................................................................................93.0 SystemVerilog for Functional Coverage at ARM-Austin ...............................................103.1 Relating Design Components to Coverage Components.................................................113.2 Code Writing Standards...................................................................................................114.0 Translating SystemVerilog to NTB : The covermap Translator ....................................125.0 Results and Performance..................................................................................................15 Table of FiguresFigure 1 : SVA Sequence...............................................................................................................4Figure 2 : SVA Properties..............................................................................................................4Figure 3 : SVA Assert and Cover ..................................................................................................5Figure 4 : SVA Covergroup...........................................................................................................6Figure 5 : SVA Covergroup Advanced..........................................................................................7Figure 6 : NTB (Vera) coverage_group example ..........................................................................8Figure 7 : SVA Bind ....................................................................................................................10Figure 8 : Flow Diagram..............................................................................................................13SNUG Boston 2005 2 A Unique Functional Coverage Flow
  3. 3. 1.0 Introduction: Functional CoverageFunctional Coverage provides an indication of the thoroughness of the implementation of theverification plan and can help answer questions such as how much of the design specification hasbeen exercised? and what actually happened during a test? As chip designs grow larger andmore complex with thousands of possible states and transitions, a comprehensive verificationenvironment must be created that minimizes wasted effort. Functional coverage is used as aguide to direct the verification resources by identifying the tested and untested parts of thedesign. When properly used, functional coverage can serve as a formal specification of a testplan. In turn, the process of writing functional coverage can bring holes in a test plan to theattention of verification personnel.Functional coverage, because it is a measure of what is determined to be of interest and relevant,must be manually specified by members of the design and verification team. It cannot beextracted automatically from the design source code. This paper describes the use of theSystemVerilog language to create functional coverage models for an ARM processor.2.0 Functional Coverage Constructs in SystemVerilog and Vera / NTBThe constructs in SystemVerilog that are related to functional coverage are: 1. sequences and properties 2. covered properties and assertions 3. covergroupsThe sequence construct is a basic building block. A sequence is a description of temporalbehavior using logical and temporal operators of the SystemVerilog Assertion (SVA) language.SVA is a subset of SystemVerilog, useful for describing behaviors that unfold over multiple timesteps.In its simplest form a sequence can be just a Boolean expression, which is considered to be asequence of zero time steps. It is optional to define what constitutes a time step in a sequence.A time step may be thought of as a directive indicating when to sample variables appearing inthe sequence. The ability to write a sequence without defining the meaning of a time step allowsreuse of a sequence in many different contexts, each with different definitions of time steps.Labeling a sequence also facilitates reuse, as we will see below. A sequence has no defined truthvalue, but there is a concept of a sequence being matched, meaning that the behavior of a designunfolds as described in the sequence. Here, we also describe this as the sequence succeeding, orsuccessfully concluding. Figure 1 shows an example of a sequence describing a signal, req,going high, and one time step later, a signal, ack, also going high.SNUG Boston 2005 3 A Unique Functional Coverage Flow
  4. 4. sequence s_1; req ##1 ack endsequence Figure 1 : SVA SequenceA sequence can be reused, via its label, inside of a property. The body of a property is made upof one or more sequences operated on by temporal operators or logic operators. The sequencescan be written explicitly in the property, meaning, one can form expressions with variables andlogic and temporal operators, or one can reuse sequences that are defined elsewhere via theirlabel. (At ARM, we always do the latter.) A property will have a time step, or samplingdefinition. This is done using Verilog syntax, for example “@(posedge clk)”. If a sequence isreused by referencing its label inside of a property, the sequence will take on the time stepdefinition of the property. Figure 2 shows how a sequence can be reused inside a property: weshow sequence s_1 from Figure 1 being reused inside properties p_1 and p_2. Note thatevaluation of a property can be subject to constraints. Both properties in Figure 2 will only beevaluated if the signal, reset, is low. property p_1; @(posedge clk) disable iff (reset) s_1; endproperty property p_2; @(posedge clk) disable iff (reset) s_1 |-> done; endproperty Figure 2 : SVA PropertiesUnlike a sequence, a SystemVerilog property does have a Boolean truth value. At its simplest, ifthe body of a property is just a Boolean expression, the property is true on every time step inwhich the expression is true. In more complicated properties, the property is true, at a givensample point, when the mix of temporal and logical operators within it evaluates to true. Forexample, if s is the label of a sequence, and o is a variable representing a design signal, then theexpression s |-> oSNUG Boston 2005 4 A Unique Functional Coverage Flow
  5. 5. in the body of a property, where |-> is the implication operator, is true whenever: • s has not successfully concluded, or • s has successfully concluded, and at that same instant, signal o is highAs an aside, the fact that the implication operator evaluates to true when its left hand argument,known as the antecedent, evaluates to false, is often confusing to people. This is called “vacuoustruth”, and must be taken into account when setting up a functional coverage and assertion basedverification system. For example, property p_2 in Figure 2 will be vacuously true at everysample point at which sequence s_1 does not match. It would be desirable to have anindependent means of verifying that the property also was true a finite number of time whensequence s_1 did match. Counting how often property p_1 is true accomplishes this.Properties can be labeled, and, like sequences, they can be reused in a higher level context bytheir label. In particular, properties can be the target of the SystemVerilog cover and assertdirectives. When a cover directive is applied to a property, a count is generated of how often theproperty is true at its sample intervals. This is known as a covered property. When an assertdirective is applied to a property, an error is signaled if the property is ever false at a samplepoint. This is known as an assertion. Note that this allows reuse of sequences for bothfunctional coverage and assertions. Figure 3 shows cover and assert directives applied to theproperties of Figure 2, p_1 and p_2. Note that applying both these directives at the same timesolves the problem of vacuous truth for property p_2. If property p_2 is forever vacuously true,the cover directive, c_1, will have a zero count. If, on the other hand, property p_2, and,therefore, assertion a_2 is actually stressed, cover c_1 will have a positive, non-zero count, to theextent that that occurs. c_1: cover property (p_1); a_2: assert property (p_2) Figure 3 : SVA Assert and CoverSystemVerilog covergroups, in contrast to sequences and properties, are primarily targetedtowards combinational logic. They are not part of the SVA subset of SystemVerilog.SystemVerilog covergroups are closely related to the Vera ‘coverage_group’ construct thatpreceded it. A covergroup will have a sampling, or time step definition, using the same syntaxas we saw previously with properties (e.g., @(posedge clk)).The two main components of a covergroup are the coverpoint and the cross. A coverpoint isdefined relative to a signal of a particular width. The default behavior of a coverpoint is togenerate a counter for each possible permutation of the underlying signal’s values, up to somepreset limit. Thus, for example, a 3-bit wide signal would generate 8 counters, one for eachSNUG Boston 2005 5 A Unique Functional Coverage Flow
  6. 6. possible 0/1 permutation. These counters are referred to as bins. A cross generates counters forthe cross product of two or more coverpoints, meaning, counters for the occurrence of allpossible simultaneous hits to the bins of the underlying coverpoints. For example, if onecoverpoint is related to a 2-bit signal and another to a 3-bit signal, then the default behavior is tocreate 4 bins for the first coverpoint and 8 for the second. The cross of both coverpoints would,by default, create 32 distinct bins, one for each possible, simultaneous update to the bins of thecoverpoints. Figure 4 shows a covergroup like this, where coverpoints cp_1 and cp_2 aredefined from 2 and 3 bit signals, respectively, and a cross of these two coverpoints, cr_1,generates 32 bins. reg [1:0] two_bit_sig; reg [2:0] three_bit_sig; covergroup cg_1 @(posedge clk); cp_1 : coverpoint two_bit_sig; cp_2 : coverpoint three_bit_sig; cr_1 : cross cp_1,cp_2; endgroup Figure 4 : SVA CovergroupIf one does not constrain the generation of bins, a combinatorial explosion could quickly result.Fortunately, SystemVerilog has a great many ways of specifying constraints. One can put guardconditions on coverpoints and crosses, so that they are updated only when the guards hold. Onecan apply guards more finely, putting them directly on specific, user-defined bins. One canrestrict the range of values of a bin via user-defined bins, and one can label particular ranges asignorable or illegal. The illegal designation acts much like an assertion, in that it causes thesimulator to post an error if a value in the specified range is hit. In general, constrainingcovergroups involves some method of not allowing SystemVerilog to automatically generate thenumber of bins it would otherwise generate by default. This often involves creating binsexplicitly. The bins keyword is used to create explicit bins on either coverpoints or crosses, as isshown in Figure 5, below. Note that the square bracket operator, ‘[]’, will generate individualbins for each number in its range. Thus, the definition of bins c of coverpoint cp_1, Figure 5,will generate 3 bins, one each for the values 200, 201 and 202. In contrast, the definition of binsa of coverpoint cp_1 will generate only a single bin, for all of the 65 distinct values in its range.With bins a, it will be impossible to know how many times the individual values in the rangewere hit. If this information is needed, the ‘[]’ operator must be used.SNUG Boston 2005 6 A Unique Functional Coverage Flow
  7. 7. reg [10:0] v_a; reg [1:0] v_b; covergroup cg_2 @(posedge clk); cp_1: coverpoint v_a { bins a = { [0:63],65 }; bins b[] = { [127:150], [148:191] }; bins c[] = { 200, 201, 202 } iff (!reset) ; bins d = { [1000:2000] }; illegal_bins bad_vals = { [2001: 2047] }; } cp_2: coverpoint v_b; cr_1 : cross cp_1, cp_2 { bins b_cr = binsof(cp_1.b) && binsof(cp_2); } endgroup Figure 5 : SVA Covergroup AdvancedBins definitions for crosses are more complicated than those for coverpoints. This is because, ina cross, one is crossing all combinations of defined bins for the coverpoints, and one needs amechanism to choose these combinations. SystemVerilog has a binsof operator for this purpose.The argument of the binsof operator can be either a coverpoint, in which case all defined bins forthat coverpoint are meant, or a coverpoint bin, in which case only that bin is selected. One canuse a limited set of logic operators on binsof expressions, to further refine one’s selections. Toexplain this, consider the following cross bins definition shown in figure 5: • The cross is labeled cr_1 • The cross operates on coverpoints cp_1 and cp_2 • The cross has a single, user defined bin, b_cr • The first binsof operator selects a single bin, b, within the coverpoint cp_1. Other bins defined for cp_1 are ignored. • The second binsof operator selects all bins of coverpoint cp_2. This coverpoint will have 4 automatically generated bins, since it is 2 bits wide and the user had not defined any explicit bins for it. • The && operator will cause the cross bin, b_cr, to be updated only when the specific bin b in coverpoint cp_1 is hit, and a valid bin (any) within cp_2 is also hit.SNUG Boston 2005 7 A Unique Functional Coverage Flow
  8. 8. 2.1 NTB coverage_groupsIn figure 6, we show how one would write an NTB coverage_group that has the same semantics,but slightly different syntax, as the SystemVerilog covergroup of Figure 5. Note that the NTBsample is roughly equivalent to the SystemVerilog coverpoint. In addition, we show an NTBinterface block, which is the means we used to connect to design signals in NTB. The NTBm_state construct, used in the cp_1 sample point, has a similar role to the SystemVerilog []operator, in that it creates an individual bin for every value in its range. interface cg_if { input clk CLOCK hdl_node "`design_module.clk"; input reset INPUT_EDGE INPUT_SKEW hdl_node "`DESIGN_PATH.reset"; input [10:0] v_a INPUT_EDGE INPUT_SKEW hdl_node "`DESIGN_PATH.va"; input [1:0] v_b INPUT_EDGE INPUT_SKEW hdl_node "`DESIGN_PATH.vb"; } coverage_group cg_2 { sample_event = @(posedge cg_if.clk); sample cp_1 (ex_sysvlog_if.v_a) { state a (0:63,65); m_state b (127:150], [148:191]); m_state c (200:202) if (!cg_if.reset ); state d (1000:2000); bad_state bad_vals (2001:2047); } sample cp_2 (cg_if.v_b); cross cr_1 (cp_1,cp_2) { state b_cr ( (( (cp_1 >= 127) && (cp_1 <= 191) ) ) && (( (cp_2 >= 0) && (cp_2 <= 3) ) )); } } Figure 6 : NTB (Vera) coverage_group exampleSNUG Boston 2005 8 A Unique Functional Coverage Flow
  9. 9. 2.2 Bind DirectivesSampling of signals in SystemVerilog is made simple by the bind directive. Bind directivescause one module to be instantiated within another, without having to explicitly alter the code ofeither. This facilitates a strict separation between a design’s implementation code and itsverification code, whether the latter is code for functional coverage, for testbench modeling, orany other purpose.The first three fields of the bind directive are as follows: 1. The name of the module or instance that is the target of the binding 2. The name of the module that is being bound to the target module, and 3. The instance name of the module being boundAfter these three fields there follows a port list, with the same syntax as for a normal, Verilogmodule instantiation. Figure 7 illustrates this. There, we show a module with functionalcoverage code, cover_mod. It contains a single covergroup, cg_sig, which uses signals local tocover_mod. But, these signals are actually connected to those of a design module, design_mod,which is not shown. A bind directive (note that it is defined outside of module scope) makes thisconnection. The signal names in the port list are those local to the design module. They areconnected to the inputs of the coverage module, cover_mod, in order of appearance. Similar tothe rules for traditional Verilog instantiations, we also could have used dot operator connectionsin the port list. These are safer, being less error prone, but for brevity, we chose to not to showthem.When the bind directive is executed during compile time, the bound module is instantiateddirectly below the target module or instance. Since the target may be named by its module orinstance name, there are really two types of bindings. If module binding is used, an instance ofthe bound module is placed below each instance of the target module. If instance binding isused, an instance of the bound module is placed below only one, particular instance of the targetmodule. Usually, for functional coverage, there is no need for module binding, as coverage isgenerally written for modules that are rather high up in the hierarchy and have only one instance.Signals in the target instance are bound to inputs in the bound module through the bind port list.Thus, the bound module has access to any and all signals in the scope of the target instance,simply by putting them in the port list, which makes sampling of design signals for functionalcoverage rather easy. The hierarchical path to the coverage module instance also makes clearwhich parts of the design are being covered, because the design instance whose signals are beingsampled is placed immediately above the coverage module instance in the hierarchy. Thus, ifone adopts a naming convention that makes it easy to recognize coverage modules, one canknow which design modules they are sampling simply by reading the hierarchical path name ofthe coverage module.SNUG Boston 2005 9 A Unique Functional Coverage Flow
  10. 10. module cover_mod (clk, reset, enable, sig1, sig2, sig3); input clk; input reset; input enable; input [1:0] sig1; input [2;0] sig2; input [3:0] sig3; covergroup cg_sig @(posedge clk) cp_1 : coverpoint sig1 iff (!reset && enable); cp_2 : coverpoint sig2 iff (!reset && enable); cp_3 : coverpoint sig3 iff (!reset && enable); cr_1 : cross sig1,sig2,sig3; endgroup endmodule bind design_mod cover_mod u_cover_mod (d_clk, d_rst, d_en, d_sig1, d_sig2, d_sig3); Figure 7 : SVA Bind3.0 SystemVerilog for Functional Coverage at ARM-AustinAt ARM in Austin, we debated several choices for a functional coverage vehicle for our currentproject, the design of a next generation, high performance ARM core. In the end, we decided onSystemVerilog as implemented in Synopsys’ VCS. We were motivated by the fact thatSystemVerilog was an emerging standard that would eventually have wide vendor support, andthat it was being integrated at no extra cost into VCS. We also liked the fact that, via itssequence and property constructs, with the associated cover and assert directives, one couldreuse SystemVerilog functional coverage code for assertions, and visa-versa. In addition,SystemVerilog supported coverage of permutations of values of bit-vectors, through itscovergroup construct. This was extremely important to us, and, it turns out, the overwhelmingmajority of our coverage bins ended up coming from covergroups. None of the other, competingfunctional coverage vehicles had the same degree of support for all these aspects of functionalcoverage. SystemVerilog stood out in this regard.However, we had to deal with the problem that SystemVerilog support was being put into VCSincrementally, and not all coverage constructs we wanted to use were available yet. The mostimportant one that was missing was the covergroup. However, at the suggestion of Synopsys,we decided this could be overcome via a translator. Synopsys was building up support for NTB,the natively compiled version of Vera now being bundled with VCS, at the same time as it wasbuilding up support for SystemVerilog,. The NTB / Vera coverage_group construct was verySNUG Boston 2005 10 A Unique Functional Coverage Flow
  11. 11. similar to that of the SystemVerilog covergroup and was immediately available in VCS. Wedecided translating between the two was feasible. We created a translator and called itcovermap. With covermap, we were able to train our team in the use of SystemVerilog in thepresent, while building up a body of code that could be reused in the future, when SystemVerilogwould be fully supported. In the meantime, covermap would insure (hopefully) that ourfunctional coverage code executed with the expected functionality of SystemVerilog.3.1 Relating Design Components to Coverage ComponentsAs part of our functional coverage methodology, we mandated that: • Functional coverage code be written inside of standard, Verilog modules, as opposed to newer SystemVerilog constructs such as program blocks or classes. This allowed our hierarchy to be composed of similar structures, and we did not have a strong need for the other structures, as they were more targeted towards testbenches and we were using an all Verilog testbench. • Bind directives be used to create a one-to-one relationship between a coverage module and a design module. This facilitated our post-processing tools relating the sampled modules to the modules doing the sampling. • Only standard Verilog reg and wire variables be used in coverage modules. While other constructs were available in SystemVerilog and had some advantages, this kept the coverage modules as similar, in coding style, to the design modules as possible. • Local variables be allowed in coverage modules, but bracketed between formatted comments. This allowed the covermap translator to recognize them easily. The local variables were used to form Boolean expressions, or, quite often, complicated state machines from sampled, design variables. This, in turn, simplified the task of functional coverage. For example, bus signals were used to create state machines that kept track of transaction histories, and, in turn, functional coverage was written in terms of these histories.In our configuration management structure there is a directory for each distinct unit of ourdesign. We made a subdirectory under each such unit directory, for functional coverage files.We initiated a naming convention where the file names for functional coverage files reused thenames of the design modules they were covering, with an added “_sysvlog.v” suffix. We createda small Perl script that found all such files with these suffixes, whenever we compiled asimulation model with functional coverage, and put them in a file list for the VCS compiler.3.2 Code Writing StandardsFor covered properties, we mandated that all coverage code writers would use the sequenceconstruct to describe temporal behavior, and that all sequences would be labeled for reuse. Wethen created covered properties by labeling the property, and applying a cover directive to thatlabel. We created assertions from sequences in a similar manner, using the assert directive.Within assertions, a sequence would typically occur as the antecedent of an implication.Assertions and covered properties work synergistically, in that the covered property measureshow often the design has been stressed, while the assertion monitors for correct behaviorwhenever the stress is applied. As stated above, “vacuous truth” is a problem in a verificationsystem. Using our methodology, if an assertion never fired (which is desired) one could easilySNUG Boston 2005 11 A Unique Functional Coverage Flow
  12. 12. ascertain, from looking at hit total for the related covered property, whether that was because itwas never stressed, i.e., whether its left hand side, the antecedent, had never been true.For covergroups, we supported most of the constructs that are in the current SystemVerilog 3.1aLRM; except for transition coverage (no one felt a strong need for it).For all coverage constructs, we mandated a naming convention. For example, covered propertynames would begin with “c_”, covergroups with “cg_”, coverpoints with “cp_”, etc. Thisfacilitated easy recognition of an item’s type in our output, when a coverage bin was listed usingits full hierarchical path. In that case, with these conventions it was easy to see that it was, forexample, a bin within a cross within a covergroup, and to know which design module’s signalswere being sampled.We also utilized a priority labeling system, with each covered property and each coverage binallowed to have a separate priority. We allowed three levels: 1. L1. Meaning, important enough that it must be hit in top level simulations 2. L2. Meaning, desirable to hit at top level, but acceptable if hit only in unit level simulation (where it is easier to generate specific input sequences) 3. L3: Meaning, desirable to hit at any level, but not requiredUse of anything but L1 coverage was discouraged, and had to be justified. L3 coverage wasused only rarely, and had to be rigorously justified. Usually, it was used as a temporary holdingplace for coverage points that were new and not thoroughly tested.We implemented the priority labeling system through formatted comments containing a coverageitem’s name and its priority level. The covermap translator then applied that label to that itemand to all items below it in the coverage hierarchy, unless the label was contradicted by aspecific directive found for an item lower down. The highest element in the coverage hierarchyfor any Verilog module was the module itself, the next highest were individual coveredproperties and covergroups, and, within a covergroup, coverpoints and crosses were next highest,with the lowest level being the bins of coverpoints and crosses.4.0 Translating SystemVerilog to NTB : The covermap TranslatorThe main task of the covermap translator is to read in SystemVerilog files, find covergroupconstructs, and translate these to NTB coverage_group constructs. But, it performs many otheruseful tasks, as well, for example enforcing our priority labeling system described above. Thetranslator would signal an error if a coverage item did not have an associated priority label.Each file read in by covermap is expected to contain a single, Verilog module definition, and oneor more SystemVerilog bind directives. The Verilog coverage module may contain one or moreof the following: • SVA sequence and property definitions • SVA covered property and assertion declarations • SystemVerilog covergroup definitions and instantiationsSNUG Boston 2005 12 A Unique Functional Coverage Flow
  13. 13. • Local variable definitions, and Verilog code (including tasks and functions) utilizing theseWhen covermap translates ‘n’ SystemVerilog input files, it will produce 2n + 1 output files, ofthe following type: • ‘n’ files with all the original SVA covered properties and assertions, and with all Verilog code utilizing local variables, but with covergroups and their instantiations commented out • ‘n’ NTB files holding NTB coverage_groups derived from the original SystemVerilog covergroups • A “master” NTB file holding a program blockNTB begins execution in a program block, a construct somewhat similar to the ‘main’ functionin C. There must be at most only one such block in any body of NTB code. All the NTBcoverage_groups must be instantiated in this program block, even if they are defined in separatefiles. covermap facilitates this by generating include directives within the master NTB file forall the separate NTB files.The following diagram illustrates the functional coverage flow: Test Stimulus Original SV Files Verilog Design Files Files NTB Files Compile and Sim Scripts covermap translator SV Files NTB SVA DB Init DB Post processing File Viewer Programs Figure 8 : Flow DiagramSNUG Boston 2005 13 A Unique Functional Coverage Flow
  14. 14. In this flow, we start with hand coded SystemVerilog files. As covermap reads in these files, itstores information in a set of C++ classes that give it a picture of the coverage hierarchy. It thentraverses this hierarchy down to individual coverage bins and prints out a list of all these bins ina database initialization file (“DB Init File”). This file is used to initialize an SQL database usedby an HTML viewer for coverage results. Along with the hit counts for each bin, the databasestores a reference to the test names that hit it, and Perl scripts operate on the database to generatea minimized suite of tests that can hit all the coverage points. In addition to using this SQLdatabase and HTML viewer for functional coverage, we track pass / fail numbers on regressionsuites and track progress of the debug effort, with it, as well. One of the most important reasonsfor creating our own means of post processing functional coverage results was to integrate thoseresults with this type of information coming from our simulations, so that, within one viewingtool, we could track all pertinent aspects of our project’s progress.In the normal translation flow, the original SystemVerilog files are translated into 2 types offiles: (1) NTB files holding coverage_groups, and (2) SystemVerilog files holding all theoriginal code except that related to covergroups. When the Verilog from the design is compiledfor simulation, these files are compiled in as well.As covermap translates, it changes names of coverage items in order to transmit information ondesign hierarchy into NTB data. Since NTB coverage_groups are instantiated in the NTBprogram block, which is in the testbench, no information is readily available in output files as tothe design blocks being sampled by a coverage_group. covermap is able to recreate thehierarchical path that would have existed had the NTB coverage_group been a SystemVerilogcovergroup. It encodes this information in a new name for the NTB coverage_group. In turn,post processing scripts are able to parse this name, and show the user where the coverage itemsare in relation to the design hierarchy.As test stimulus is loaded and simulations are run, two different types of coverage databases areformed: SVA and NTB. The SVA database holds hit counts for covered properties, while theNTB database holds the same for the bins created by NTB coverage_groups. Special scripts andprograms then read these databases, and reformat the data to make it suitable for the HTMdatabase viewer (“DB Viewer”). This requires anticipating the names VCS / NTB will apply toautomatically generated coverage_group bins.When the coverage files are translated, the resulting set of NTB and SVA files are compiled withthe code for the design and testbench. During covermap’s translation, the user may obtain twotypes of optional output: • statistics on the distribution of properties and covergroups through the design hierarchy, as well as a breakdown within the covergroups on the quantities of bins in each member coverpoint or cross, and • an initialization file for loading our SQL coverage database, listing the hierarchical path to each coverage bin of all the covergroup coverpoints and crosses, as well as the hierarchical path to each covered property.Great care is taken that the names of bins coverpoint and cross bins correspond to what will begenerated after simulation. For instance, NTB has certain naming conventions for auto-generated bins, and certain formatting requirements for fields within names. We take care toSNUG Boston 2005 14 A Unique Functional Coverage Flow
  15. 15. produce names consistent with those in our SQL initialization file, because we will use stringmatching to compare names of bins hit in simulation to those in the initialization file.After simulations have concluded, functional coverage data exists in two separate sets of files,one for NTB coverage_groups and one for SVA covered properties. We then transform this datafor our SQL / HTML viewer. In our post process scripts, we label it as L1, L2 and L3 prioritylevel. The covermap translator enables this by prepending the priority level to the translatedname of the coverage item. We run scripts that use the Tcl SVA API (application programminginterface) to extract the covered properties that were hit, and the number of hits for each. We runa C program that utilizes Synopsys new UCAPI (Unified Coverage API) library, to read out thesame information for the NTB coverage_group members. The UCAPI is a relatively new libraryintended to be a single interface through which all the various types of coverage formats in useby Synopsys may be read.5.0 Results and PerformanceAt this point, we are still in the midst of our design project, and while we have written most ofour intended coverage code, we have not written all of it. At the unit level, our functionalcoverage is well above 90%, and it is approaching that for top level.One important issue is the cost of functional coverage, i.e., how much it reduces simulationperformance. Of course, this will be in proportion to the size of the functional coverage codebase relative to the design code. There is no universally accepted way to measure the size of abody of functional coverage code. We feel the most appropriate way is to count coverage bins,in other words, to determine how many separate events are being monitored. When we expandour code, we currently generate around 100,000 coverage bins; but, among these there can be awide variance in simulation time. A simple coverpoint bin, where the coverpoint hasn’t anycomplex sampling guards, would be very quick to simulate, since it can be implemented withefficient, integer operations. A covered property (which can be considered to implement a “bin”,if we understand that to mean a counter of how often the property was hit), on the other hand,could be very costly to simulate, since it may describe a very complex sequence of events, thetracking of which requires a large overhead. However, the vast majority of our coverage binscome from covergroups.Right now, we are experiencing slowdowns of 300% to 450% in top level simulations withfunctional coverage. In other words, if a simulation takes, say, 100 CPU seconds to simulate,using a simulation model compiled without functional coverage code, it will take between 300 to450 CPU seconds to simulate when our functional coverage code is compiled in. This is just thesimulation time, and does not include any post processing time for analyzing results. Whenconsidering these numbers, it should be borne in mind that the testbench for our full design (asopposed to our unit level testbenches) is an all Verilog testbench, using only a very few PLIcalls. For many users of functional coverage, they will be adding functional coverage to thealready existing overhead of a testbench in a specialized language, such as NTB / Vera. In thiscase, the effect of functional coverage may not seem as large. Ours is a rather unusual case,allowing one to see the overhead of functional coverage relative to a rather pure body oftraditional Verilog code (note that in our design and testbench we are not using SystemVerilogSNUG Boston 2005 15 A Unique Functional Coverage Flow
  16. 16. constructs, only in our functional coverage files, and much of our testbench is written insynthesizable Verilog).We have attempted to analyze these slowdowns, with help from Synopsys R&D teams. Onemajor aspect, which cannot be resolved quickly, is that much of the work VCS does to analyzefunctional coverage is still being done through function calls into libraries. While this is quitenormal, it obviously would be better if the function call overhead could be eliminated. VCS getsmuch of its speedup on regular Verilog code from transforming it into natively compiledconstructs. Synopsys plans to eventually do all functional coverage work in a native compile aswell, and this is where the largest performance gains are expected. Until then, performanceimprovements must come from incremental sources.One workaround we are investigating is compiling only those coverage items that have not beensufficiently hit. The idea is to create a feedback loop between the coverage results database andthe covermap translator. The translator would inspect a list of coverage items hit to date on aspecific version of the design, and not translate those that have been sufficiently hit.While the reduced performance we have experienced is a major problem, we do not consider thisa defect specific to SystemVerilog or VCS, we suspect that slowdowns like this are commonthroughout the industry. Since use of functional coverage is only now starting to spread, theproblem may not be widely recognized; but, it is certainly important. It is obviously desirable todo functional coverage; but, at the same time it is very costly. Design teams are facing thedifficult choice of having to split their resources between simulating the design, which is theirprimary means of finding bugs, and tracking whether their simulations are effective, throughfunctional coverage. Given the current performance problems of functional coverage, it istempting to skip it in favor doing more simulation. Synopsys must work hard to eliminate theneed for this tradeoff, by boosting the performance of functional coverage.SNUG Boston 2005 16 A Unique Functional Coverage Flow

×