Code Coverage in VSTS with xUnit, Coverlet and ReportGenerator

Today I was reading a post by Scott Hanselman about Code Coverage in .Net Core projects. Suddenly I started to go down a rabbit hole of posts about tools I never heard of, but that sounded quite interesting, such as Coverlet and ReportGenerator. Then I thought that they could possibly fit into our CI flows.

When I started .NET development I didn't do Unit Testing at all, I admit it. However, thanks to good folks such as Luis Fraile and Unai Zorrilla Castro I soon realized that they were so important that I could not live without them. So, I adopted Microsoft’s MsTest and started to write lots of [TestClass]es and [TestMethod]s. Then at some point I heard about xUnit and nothing was the same again. I forgot about [TestClass] attributes and started to write [Fact]s and [Theory]es.

I have been using them for a long time, but the problem I found when changing from MsTest to xUnit was that the information shown in Visual Studio Team Services’ build results was very poor, to be polite. And there was no code coverage data at all. That was bad indeed, but I am lazy and I have postponed the task of improving this until today.

a “cross-platform code coverage library for .NET Core, with support for line, branch and method coverage”.

To include Coverlet build targets in your project you must add the following line to your test project’s csproj file:

its Github home page, but basically you can generate a code coverage file in several common formats by invoking dotnet test with the CollectCoverage parameter.

dotnet test /p:CollectCoverage=true

This, by default, generates a code coverage file in JSON format. If you want to experiment with Visual Studio Code extension Coverage Gutters, as suggested by Hanselman, you can use a task such as this in your .vscode/tasks.json file:

}
}

dotnet test with these parameters:

dotnet test 
  /p:CollectCoverage=true 
  /p:CoverletOutputFormat=cobertura

coverage.cobertura.xml. This alone can be used to publish code coverage data to VSTS build results. However, we’ll go a step further and generate a full report that can be viewed and browsed with source code linking.

ReportGenerator

coverage.cobertura.xml file.

dotnet CLI tool into our project. Just add the following lines into your test project’s .csproj file:

</ItemGroup>

NOTE: this is an alpha version of the tool, so be prepared to face potential issues and changes without notification. If you need something more stable, you can download and install the ReportGenerator.exe in a folder of your build agents and set your PATH accordingly.

.csproj file should look something like this:

</Project>

Set up build definition

Now that we have the project correctly configured, it’s time to set up our build definition. Keep in mind that the following steps won’t work if you haven’t modified your project with the NuGet packages defined before.

Below you can see an example build definition. We are interested in the three highlighted tasks.

单元测试 & 测试覆盖率——Code Coverage in VSTS with xUnit, Coverlet and ReportGenerator
Coverlet
ReportGenerator
Set up build definition

The first task runs the tests using the xUnit test runner and generates the code coverage file. The second task takes the generated file and builds a detailed report linked to source code. The third task publishes the two previous results so that they can be shown in the build results panel. Let’s see each one in detail.

Test and collect coverage

.NET Core task with these options:

Command: test
Path to project(s): select just one test project. It should be possible to choose more than one with globs but I haven’t tried.
Arguments:

cobertura 
/p:CoverletOutput=$(Build.SourcesDirectory)TestResultsCoverage

TestResultsCoverage.

dotnet test and collect coverage