Core Framework v2 2.4.2 2022 August 1

Today, we're shipping three new releases:

• xUnit.net Core Framework v2 2.4.2
• xUnit.net Analyzers 1.0.0 (release notes)

It's been nearly 4 years since the release of 2.4.1. This release includes a few targeted bug fixes for the core framework, and an updated assertion library with many of the new v3 assertions being back-ported to v2. This new assertion library includes several new assertions and overloads of existing assertions, improvements to output for failing tests, and nullable annotations.

This is also the long-overdue 1.0.0 release of the analyzers!

We did not intend to ship any more v2 releases; that said, we also did not intend for the v3 development process to take as long as it has. We felt it was necessary to address a couple specific high-friction issues (including two from the .NET Team at Microsoft); the new assertions are merely an extra reason to encourage the upgrade (and help tide teams over who might be waiting for v3 with some of the new features).

Important

You will notice that we have not changed any of the minimum requirements for this release, despite the fact that .NET 4.5.2 and .NET Core 1.0 are quite old at this point. We did not want to disrupt the workflow of those who may still be stuck on older (even out of support) versions of .NET Framework or .NET Core.

You should not consider this an endorsement that you use these older versions of .NET Framework or .NET Core; this was merely about making as few changes as possible to minimize compatibility issues, especially with third party extensibility libraries and test runners. Please upgrade to supported versions as soon as possible, as some of them are not even receiving security patches at this point in time.

This need to stick to the older target frameworks is the reason we are unable to ship the new assertions based on Span, Memory and ValueTask. If you wish you be able to consume these, they are part of the source package xunit.assert.source, which you could use (instead of xunit.assert), and then define the appropriate constants in the project (using <DefineConstants> in your project file) to enable more advanced features. For more information on the constants and available features, please see the Assertion library home page README.

As always, we'd like to thank all the users who contributed to the success of xUnit.net through usage, feedback, and code. 🎉

Release Notes

These release notes are a comprehensive list of changes from 2.4.1 to 2.4.2.

Core framework

• BUG: Fixed an issue where placing Skip on an [InlineData] attribute was being ignored by Visual Studio Test Explorer. xunit/visualstudio.xunit#266

• BUG: Fixed an issue where decimal values weren't usable in generic theories because the type coercion logic was not accounting for the possibility of multiple type-compatible op_Implicit or op_Explicit methods. xunit/xunit#2418

• BUG: Fixed an issue where type inference for generic test methods was not done correctly when one of the test data values was null (it would incorrectly use object to close all the generic types rather than continuing to inspect the subsequent non-null values for their correct types). xunit/xunit#2401

• BUG: Fixed a bug in DisposalTracker which was a race condition that could cause objects to fail to get disposed and/or cause objects to get disposed twice. xunit/xunit#1855

Assertion library

New assertions

• Added Assert.Distinct to verify that a collection does not contain any duplicate values. xunit/assert.xunit#35

• Added Assert.Equivalent to check for object equivalence. Object equivalence is intended to be a looser kind of assertion than Assert.Equal. This assertion allows an optional strict flag to influence its behavior. The rules for equivalence are as follows:

  • null values are only ever equivalent to other null values (regardless of strict).
  • For value types and strings, equivalence and equality are the same (regardless of strict). In particular, this means that types must match exactly (for example, int and long values are never equivalent, even if they have the same underlying value; or, in code form: Assert.Equivalent(12, 12L); will fail).
  • For collections, the items in the two collections are compared for equivalence regardless of their order in the collection (meaning [1, 2] is equivalent to [2, 1]) and regardless of the collection type (for example, List<int> can be compared against int[]). The strict parameter influences this comparison by requiring that all values in the expected list are in the actual list (meaning expected: [1, 2] is equivalent to actual: [2, 3, 1] when strict is false, but not equivalent when strict is true). Another way to think of this is that strict: false allows actual to be a strict superset of expected, whereas strict: true does not.
  • Reference types which point to the same instance are always equivalent (regardless of strict). Circular references are not allowed, and will be surfaced as test failures. For all other reference types, the public non-static fields and readable properties are compared for equivalence without regard to the types of the two values and without regard to which values came from fields vs. properties. Like collections, the strict flag indicates whether actual is allowed to be a strict superset of expected (strict: false) or an exact match (strict: true). Reference types inspected in this way determine the equivalence of individual field/property values by using these same sets of rules when comparing the field/property values. This includes reference type values (meaning the reference type equivalence is considered to be "deep", not "shallow").
  • KeyValuePair<,> is handled specially: the keys are checked for equivalence against each other, as are the values; this allows embedded collections inside of keys and/or values inside a KeyValuePair.

  It's somewhat challenging to describe these rules in plain English, so please see the comprehensive test suite linked below for codified versions of all these rules. xunit/xunit#1604 Assert.Equivalent test suite

• Added Assert.Fail which unilaterally fails the test with the given message. xunit/xunit#2105

• Added Assert.Multiple which allows the developer to run several independent assertions (expressed as Actions) and then collect up all the failures into a single failure. This behavior differs from the default behavior (which stops running assertions upon the first failure). xunit/xunit#1920

New overloads

• Added an overload of Assert.All which takes an Action<T, int> so the inspecting action get access to both the value and its index in the collection. xunit/xunit#2082

• Add overload of Assert.Equal for doubles which includes a MidpointRounding flag. xunit/xunit#2074

• Add overload of Assert.Equal for floats and doubles that allows specifying a tolerance (rather than a precision). xunit/xunit#1293 xunit/xunit#1984

Message improvement

• Improved the output of Assert.Collection with a pointer-style message (like string assertions) that make it easier to identify the failing part of the collection. xunit/xunit#994 xunit/xunit#1924

• Improved the output of Assert.Empty to match other expected/actual exceptions. xunit/xunit#1806

• Improved the output of Assert.Single to improve the output message when being used with a "match" expression (it used to say the collection was empty, but now correctly mentions the fact that you were matching specific values). xunit/assert.xunit#28

• Improved the output from exceptions which show expected and actual values which are multi-line values, so that the lines are properly indented. xunit/xunit commit 8b67420

Bugs fixed

• Updated the white space list Assert.Equal for strings with ignoreWhiteSpace: true. xunit/xunit#1931

• There are times during Assert.Equal with collections that can iterate multiple times (which is generally considered bad form with IEnumerable). The most egregious case of this was fixed, though there is still at least one more possibility during test failure to end up enumerating a second time. xunit/xunit#2402

Other improvements

• All appropriate assertions have been annotated for nullability:

  • Assert.NotNull tells the compiler that the object parameter will not null if the function doesn't throw. Assert.Null similarly tells the compiler that the object parameter may be null if the function doesn't throw.
  • Assert.Contains and Assert.DoesNotContain for dictionaries requires the dictionary key type to be non-nullable.
  • Assert.Single against non-generic IEnumerable indicates the return value may be null.
  • Many assertion parameters were updated to indicate when null parameter values are acceptable and when they aren't, as well as updating return signatures to indicate when returned values may or may not be null.
  • A few assertions (True, False, and Fail) are marked as never returning (aka, throwing) based on the input data.

• Marked XunitException's constructors which take inner exceptions as public. xunit/assert.xunit#33

• The event assertions (Assert.Raises, Assert.RaisesAny, and Assert.RaisesAsync) no longer express type requirements for EventArgs.

Runner Utility (and all runners)

• BUG: Updated the output when running tests inside TeamCity to improve the way TeamCity parsed test output (missing its flow ID and a flag to indicate the text was parsable). xunit/xunit#2397

• BUG: Fixed in issue with incorrect lock order when coloring console output. xunit/xunit#2228

A note about xunit.runner.visualstudio

The xunit.runner.visualstudio package has a separate lifecycle and separate minimum framework requirements from xUnit.net Core Framework. Use this table to determine which version you should use (pick the lowest version of the combination of target frameworks you're using):

	2.4.1	2.4.3	2.4.5
.NET Framework	4.5.2	4.5.2	4.6.2
.NET Core	1.0	2.1	3.1
UWP	10.0.10240	10.0.16299	10.0.16299

Visual Studio does not allow mixed versions of this package in your solution, so do not mix versions; doing so may cause problems running your projects that need the older version of the runner package. Choose the lowest version of the package that applies to all of the test projects in your solution.