6.26.10 Code Coverage and Execution Frequency

If you run extensive tests on your code, you often want to figure out if the tests exercise all parts of the code. This is called (test) coverage. The file coverage.fs contains tools for measuring the coverage as well as execution frequency.

Code coverage inserts counting code in every basic block (straight-line code sequence) loaded after coverage.fs. Each time that code is run, it increments the counter for that basic block. Later you can show the source file with the counts inserted in these basic blocks.

nocov[ ( ) gforth-1.0 “nocov-bracket”

(Immediate) Turn coverage off temporarily.

]nocov ( ) gforth-1.0 “bracket-nocov”

(Immediate) End of temporary turned off coverage.

coverage? ( – f  ) gforth-internal “coverage?”

Value: Coverage check on/off

cov+ ( ) gforth-experimental “cov+”

(Immediate) Place a coverage counter here.

?cov+ ( flag – flag  ) gforth-experimental “?cov+”

(Immediate) A coverage counter for a flag; in the coverage output you see three numbers behind ?cov: The first is the number of executions where the top-of-stack was non-zero; the second is the number of executions where it was zero; the third is the total number of executions.

.coverage ( ) gforth-experimental “.coverage”

Show code with execution frequencies.

annotate-cov ( ) gforth-experimental “annotate-cov”

For every file with coverage information, produce a .cov file that has the execution frequencies inserted. We recommend to use bw-cover first (with the default color-cover you get escape sequences in the files).

cov% ( ) gforth-experimental “cov-percent”

Print the percentage of basic blocks loaded after coverage.fs that are executed at least once.

.cover-raw ( ) gforth-experimental “.cover-raw”

Print raw execution counts.

By default, the counts are shown in colour (using ANSI escape sequences), but you can use bw-cover to show them in parenthesized form without escape sequences.

bw-cover ( ) gforth-1.0 “bw-cover”

Print execution counts in parentheses (source-code compatible).

color-cover ( ) gforth-1.0 “color-cover”

Print execution counts in colours (default).

You can save and reload the coverage counters in binary format, to aggregate coverage counters across several test runs.

save-cov ( ) gforth-experimental “save-cov”

Save coverage counters.

load-cov ( ) gforth-experimental “load-cov”

Load coverage counters.