0.1.1-pre.267
)The purpose of this document is to give a general state of the alpha package releases of xUnit.net v3. I will work to update this document after each purposeful release, both by updating the general structure of the content as well as to provide a quick diff from the previous release. In particular, save yourself a lot of grief by checking out the "Known issues" section. :)
The packages from CI builds are available on feedz.io.
https://feedz.io/org/xunit/repository/xunit/search
https://f.feedz.io/xunit/xunit/nuget/index.json
This list highlights the major architectural changes in v3.
For unit test authors, we have bumped up the minimum runtime requirements to match
with our move to netstandard2.0
. Today, our supported runtimes include:
Also new for v3: Mono is officially supported on Linux and macOS for .NET Framework test projects. While it did often work with v1 and v2, we do officially test and verify with it now.
The full list of planned runtimes for v3 can be found in our roadmap.
With xUnit.net v1 and v2, unit test projects were class library projects;
that is, when compiled, they always generated .dll
files,
which relied upon an external runner to run.
This is a design that dates back to the beginnings of .NET Framework, long before .NET Core came into being. Even then it had some unfortunate downsides. Let's take a look at a couple.
One example is library dependency management: if the runner loads your test assembly into the same process as itself, and both pieces of code wish to use a library, there was a "first one wins" conflict, which meant your unit test always lost. In .NET Framework, the workaround was App Domains, which is not available with .NET Core (and sometimes test code didn't run properly with app domains). Additionally, the assembly dependency resolution system in .NET Core is exceptionally complex and not designed to consumed directly (especially as it pertained to un-managed dependencies, like Win32 DLLs), so it was a frequent problem running .NET Core tests in-line with a runner.
Our second example is that there are some things that can only be chosen on
a process basis, not an App Domain basis. One category of those things would be
where .NET APIs are just thin wrappers around Win32 functionality, where there's
no App Domain awareness; the most common one that bit people with testing is
Directory.SetCurrentDirectory
. If a runner had loaded multiple test
assemblies to run in parallel, and they each call that API, they are mutating a
piece of shared state, which can cause unpredictable failures. Similarly, with
.NET Framework, the "chosen" version of the .NET Framework (as well as 32- vs.
64-bit-ness) is a decision made by the process, which in the case of stand alone
runners means the runner chooses that rather than the unit test. This became
such a significant issue that we shipped at least a dozen versions of the console
runner at a time: the cross-product of 32- vs. 64-bit and .NET 4.5.2 vs. 4.6 vs.
4.6.1, etc.
The solution to all these problems is that unit tests should be run in their own process. It lets us leverage the existing assembly resolution logic without needing anything special, and offers a better level of isolation from one test project to another when running in parallel.
In v2, we separated two libraries: xunit.core.dll
and
xunit.execution.*.dll
. The purpose of this separation was two-fold:
to isolate the code used to write tests and the code used to run those tests;
to hide the fact that while core
targeted netstandard1.1
,
execution
was forced to ship framework-specific DLLs.
With v3, these two libraries have been collapsed into xunit.v3.core.dll
and the target is now netstandard2.0
. This will primarily benefit
extensibility authors who previously had to choose whether to extend core
and/or execution
, and more specifically, had to ship multi-targeted
libraries to match whichever runtimes they wanted to support.
Note that currently the xunit.v3.core
(and xunit.v3
) NuGet
package shows target frameworks of net472
and net6.0
because of the in-process runner requirement. Developers who extend xUnit.net will
instead use the xunit.v3.extensibility.core
NuGet package, which is
single-targeted against netstandard2.0
. Extensibility authors will no
longer need to ship multi-targeted NuGet packages.
This will typically only affect developers who wish to contribute PRs to xUnit.net.
Our CI process runs everything against Windows, Linux, and macOS; the final binaries
are built on Linux. We do still provide Visual Studio solution files for those who
wish to run Visual Studio, but we also provide Visual Studio Code build and debug
gestures. We ship both PowerShell and bash supported build
commands.
The CI builds are hosted on feedz.io.
In order to download packages from this feed, you need to configure it with a
nuget.config
file. Regardless of whether you're planning to upgrade
an existing project from v2 to v3, or start a new v3 project, you must take this
step first or else package restore of the v3 packages will fail.
In your solution folder, create a file named nuget.config
and add the following
contents:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="nuget" value="https://api.nuget.org/v3/index.json" />
<add key="xunit-ci" value="https://f.feedz.io/xunit/xunit/nuget/index.json" />
</packageSources>
</configuration>
Note: You may need to restart your IDE for it to pick up these changes.
If you already have a nuget.config
file, you can simply merge the
<add key="xunit-ci" ...>
line into it.
The following is a quick list of changes that are needed when moving a test project from v2 to v3. Your project may require additional changes. Note that it's generally expected that unit test projects should "just work" when migrating from v2 to v3; porting extensibility libraries from v2 to v3 is beyond the scope of this document at this time (in addition to the APIs being still very much under development).
Change the following package references:
v2 package | v3 package |
---|---|
xunit | xunit.v3 |
xunit.abstractions | Remove, no longer required |
xunit.analyzers | Unchanged (though there are no v3-specific analyzers yet) |
xunit.assert | xunit.v3.assert |
xunit.assert.source | xunit.v3.assert.source |
xunit.console | Not yet available |
xunit.core | xunit.v3.core |
xunit.extensibility.core xunit.extensibility.execution | xunit.v3.extensibility.core (*) |
xunit.runner.console | Not yet available |
xunit.runner.msbuild | Not yet available |
xunit.runner.reporters xunit.runner.utility | xunit.v3.runner.utility (*) |
xunit.runner.visualstudio | Unchanged (though v3 is not yet supported) |
Note: In some cases multiple libraries/packages were merged together into a single new library/package, as denoted in the table above with (*).
Update your project file (i.e., .csproj
) and change OutputType
from
Library
to Exe
. You may need to add OutputType
if it's
not present, since Library
is the default value:
<PropertyGroup>
<OutputType>Exe</OutputType>
</PropertyGroup>
There are new minimum target framework versions; make sure to update your target framework(s) if you're currently targeting something that's too old.
Since there is no project template yet for xUnit.net v3, you should
create a project using dotnet new console
from the .NET
SDK command line tool. We currently support C#, F#, and VB.NET,
targeting .NET Framework 4.7.2+ and/or .NET 6+.
After creation, edit your project file to make it look like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="xunit.v3" Version="0.1.1-pre.267" />
</ItemGroup>
</Project>
In current form, the only way to run unit test projects is directly, since they are executables. All test projects are linked with an "in-process runner" for their appropriate target framework, which
.NET Framework projects directly result in a .exe
file,
which can be directly run on Windows or run via Mono on Linux and
macOS. The executable has many of the same command line options that
you're accustomed to having access to with the stand-alone console
runner; simply run MyTests.exe -?
to get a help page full
of available options.
.NET projects can be run with dotnet run
(advanced
users can also use dotnet exec
). To run a project, use
a command like dotnet run --project src/MyTests/MyTests.csproj
.
This also supports command line options; to specify them with dotnet run,
you need to include a double-dash to separate dotnet run's options from
your program options. To see the help page, for example, run
dotnet run --project src/MyTests/MyTests.csproj -- -?
.
If your test project is multi-targeted, you must specify --framework
when using dotnet run
. Note that dotnet run
is also able
to run .NET Framework projects (assuming you have Mono installed), so you can use
dotnet run
for both types of project if you wish.
Since unit test projects are programs, that means they need a Main
method. However, you didn't write one, so where did it come from?
We inject one. Here are the three versions:
using Xunit.Runner.InProc.SystemConsole;
public class AutoGeneratedEntryPoint
{
public static int Main(string[] args)
{
return ConsoleRunner.Run(args).GetAwaiter().GetResult();
}
}
If you want to provide your own entry point (for example, because you want to run ASP.NET Core initialization code before running your tests), you can set the following property in your project file:
<PropertyGroup>
<XunitAutoGeneratedEntryPoint>false</XunitAutoGeneratedEntryPoint>
</PropertyGroup>
Once you've done this, you're responsible for defining the Main
method
for your application, and then calling ConsoleRunner.Run
to get
things started.
As always, the best place to keep track of the ongoing work is the roadmap.
xunit.v3.runner.utility
.xunit.v3.runner.utility
claims to support running v3 projects
in the NuGet package description, but this support is not yet enabled. This library
should still be able to run v1 and v2 projects, though.
xunit.runner.visualstudio
is not yet available with v3
support, so there is no support for running tests inside of Visual Studio's Test
Explorer, nor with the dotnet test
command line.
Program.cs
for C# and Module1.vb
for VB) and make
sure to remove any <StartupObject>
properties that might exist
in your project file, as the entry point is automatically provided by the NuGet
package.