Release Notes for v3 0.7.0-pre.15 [2024 November 30]

Today, we’re shipping one new prerelease:

xUnit.net Core Framework v3 0.7.0-pre.15

It’s been 3 weeks since the release of 0.6.0-pre.7.

As always, we’d like to thank all the users who contributed to the success of xUnit.net through usage, feedback, and code contributions. 🎉

Note: Anybody who clones the source for the core project will note that we are now sticking to .NET 8 SDK with global.json. Please make sure you have .NET 8 SDK installed to build the source.

If you are new to v3

We have created two new pieces of documentation that we will continue to evolve as we create newer prerelease versions of xUnit.net v3:

We recommend you read them in that order. The migration guide will help you understand what changes are necessary to get your v2 test project ready to upgrade to v3, and it includes a fairly comprehensive list of the changes (and hopefully relatively small bumps in the road) that you can expect. This latter part is provided primarily in resource form, so rather than reading the giant boring lists, you can do a search on the page when you encounter issues with specific types. Once you’ve got your project ported over to v3, the what’s new guide will be useful to understand what new features are available to you with v3.

Please note that you will need to use a version 3.x.y release of the Visual Studio adapter to run v3 tests, as the 2.x.y releases are only capable of running v1 and v2 tests. We will eventually stop shipping the 2.x.y builds, as the 3.x.y builds are also capable of running v1 and v2 tests. We’d love if users test the 3.x.y release of the Visual Studio adapter even if they cannot migrate to the v3 Core Framework.

Release Notes

These release notes are a list of changes from 0.6.0-pre.7 to 0.7.0-pre.15.

Core Framework

Fixtures can now be retrieved via TestContext.Current.GetFixture(). This makes it easier for developers who create base test classes to avoid using constructor injection (which forces derived classes to implement the constructors) and instead retrieve fixture instances on-demand. xunit/xunit#3056
We have updated the XML documentation for places where nameof() is recommended (this includes FactAttribute.SkipWhen, FactAttribute.SkipUnless, and the MemberData.memberName constructor parameter). xunit/xunit#3062
We have reverted BeforeAfterTestAttributes’s Before and After methods to return void rather than ValueTask, as some of the extensibility that used to be achievable here was no longer possible, due to the async machinery that would prevent values from propagating out of an async method (for example, setting thread-local values).
We have added Xunit.v3.DynamicSkipToken.Value to xunit.v3.core to provide a standardized way to access the token value ( $XunitDynamicSkip$ ) that should start exception messages to indicate when the exception indicates that the test should be skipped, rather than failed.
BUG: The XML report was inappropriately escaping CRLF characters in skip reasons and exception messages. These are now properly preserved in the output.
BUG: The xunit.v3.core NuGet package was constructed in such a way that F12 into the SourceLink’d source code was broken. Fixing this required removing our reference libraries, which were used to hide an attribute that’s not supported on .NET Framework (CollectionAttribute<T>). We have updated the XML documentation for this attribute to note that this is not supported on .NET Framework, even though it’s now available for consumption. Unfortunately, the failure is surfaced at runtime, not compile time (owing to the fact that .NET Framework cannot instantiate generic attributes).

Extensibility

We have removed default self-execution from IXunitTestCase because it complicated extensibility by forcing developers to create new test case objects (and replace the discoverers) in order to customize execution. This involved:
- Removing IXunitTestCase.Run
- Adding IXunitTestCase.PreInvoke and IXunitTestCase.PostInvoke to easily allow before and after operations
- Creating ISelfExecutingXunitTestCase, with the Run method moved here, so we could still allow developers to create their own self-executing test cases without having to override most of the execution pipeline.
The samples project includes samples that utilize these new extensibility points:
- The RetryTestCase in the RetryFactExample shows using ISelfExecutingXunitTestCase to customize the execution to allow the test case to be run multiple times until it hits a maximum count or until the test passes
- The NamespaceParallelization sample is able to override test case execution without using a custom test case
We have restructured the runner classes to introduce base classes with a customizable context as needed, as well as moving more functionality from the XunitXyzRunner classes into the base XyzRunner classes. We have also collapsed three classes (test case runner, test runner, and test invoker) into two, removing the old invoker classes. This includes:
- TestAssemblyRunner
  - Added helper/extensibility point FailTestCollection (fails the test cases for the given collection)
  - Implements OnTestAssemblyCleanupFailure (sending TestAssemblyCleanupFailure)
  - Implements OnTestAssemblyFinished (sending TestAssemblyFinished)
  - Implements OnTestAssemblyStarting (sending TestAssemblyStarting)
- XunitTestAssemblyRunnerBase
  - Contains most of the original implementation code from XunitTestAssemblyRunner, with a customizable context
- TestCollectionRunner
  - Added helper/extensibility point FailTestClass (fails the test cases for the given class)
  - Implements OnTestCollectionCleanupFailure (sending TestCollectionCleanupFailure)
  - Implements OnTestCollectionFinished (sending TestCollectionFinished)
  - Implements OnTestCollectionStarting (sending TestCollectionStarting)
- XunitTestCollectionRunnerBase
  - Contains most of the original implementation code from XunitTestCollectionRunner, with a customizable context
- TestClassRunner
  - Added helper/extensibility point FailTestMethod (fails the test cases for the given method)
  - Implements OnTestClassCleanupFailure (sending TestClassCleanupFailure)
  - Implements OnTestClassFinished (sending TestClassFinished)
  - Implements OnTestClassStarting (sending TestClassStarting)
- XunitTestClassRunnerBase
  - Contains most of the original implementation code from XunitTestClassRunner, with a customizable context
- TestMethodRunner
  - Added helper/extensibility point FailTestCase (fails the given test case)
  - Implements OnTestMethodCleanupFailure (sending TestMethodCleanupFailure)
  - Implements OnTestMethodFinished (sending TestMethodFinished)
  - Implements OnTestMethodStarting (sending TestMethodStarting)
- XunitTestMethodRunnerBase
  - Contains most of the original implementation code from XunitTestMethodRunner, with a customizable context
- TestCaseRunnerBase
  - Implements OnTestMethodCleanupFailure (sending TestMethodCleanupFailure)
  - Implements OnTestMethodFinished (sending TestMethodFinished)
  - Implements OnTestMethodStarting (sending TestMethodStarting)
- TestCaseRunner
  - Added helper/extensibility point FailTest (fails the given test)
- XunitTestCaseRunnerBase
  - Contains most of the original implementation code from XunitCaseMethodRunner, with a customizable context
- TestRunnerBase
  - Implements OnTestCleanupFailure (sending TestCleanupFailure)
  - Implements OnTestFailed (sending TestFailed)
  - Implements OnTestFinished (sending TestFinished)
  - Implements OnTestNotRun (sending TestNotRun)
  - Implements OnTestPassed (sending TestPassed)
  - Implements OnTestSkipped (sending TestSkipped)
  - Implements OnTestStarting (sending TestStarting)
  - Adds an overload of OnTestStarting to support explicit and timeout values
  - Adds UpdateTestContext to allow the test class instance and test state to be updated without destroying the existing context
- TestRunner
  - Implements DisposeTestClassInstance (supporting IDisposable and IAsyncDisposable)
  - Added helper/extensibility point InvokeTest (running the test method via reflection)
  - Implements IsTestClassDisposable (looking for IDisposable and IAsyncDisposable)
  - Implements OnTestClassConstructionFinished (sending TestClassConstructionFinished)
  - Implements OnTestClassConstructionStarting (sending TestClassConstructionStarting)
  - Implements OnTestClassDisposeFinished (sending TestClassDisposeFinished)
  - Implements OnTestClassDisposeStarting (sending TestClassDisposeStarting)
  - Now responsible for creating, initializing, and disposing TestOutputHelper (during OnTestStarting and OnTestFinished)
- XunitTestRunnerBase
  - Contains most of the original implementation code from XunitTestRunner, with a customizable context
We have updated XunitRunnerHelper to work against ITestCase instead of requiring IXunitTestCase, and added a new helper method to run any IXunitTestCase (to replace the old default implementation of XunitTestCase.Run).
We have updated TestCaseDiscoverer to allow discoverers to return any ITestCase implementation, rather than being stuck to a more derived implementation. This more easily allows the creation of composite test frameworks which are able to return and run test cases that implement various interfaces (for example, implementing most of the discovered test cases with a new custom type, while utilizing test cases from outside that class hierarchy, like ExecutionErrorTestCase).
BUG: It was discovered that test case objects (namely, anything which implements ITestCase) were not getting disposed when they should have. The contract for ITestCase does not include IDisposable or IAsyncDisposable by default, but the intention was that we would support it if you added it. This has been fixed.

Runner support for Core Framework v3

Supported by the xUnit.net team

Functional:

In-box Console and MSBuild runners
Command line (dotnet test and dotnet vstest) [tested with 17.12.7]
Visual Studio 2022 (Test Explorer) [tested with 17.12.2]
Visual Studio Code [tested with 1.95.3 and C# Dev Kit 1.13.9]

Support for dotnet test, Visual Studio, and Visual Studio Code comes from the Visual Studio runner (a 3.x.y build of xunit.runner.visualstudio).

Supported by third parties

Functional:

NCrunch [requires 5.11 or later]
Resharper/Rider [requires 2024.2 or later]

Status unknown:

CodeRush