Table of Contents

Core Framework v2 2.5.0 2023 July 6

Today, we're shipping three new releases:

  • xUnit.net Core Framework v2 2.5.0
  • xUnit.net Analyzers 1.2.0 (release notes)
  • xUnit.net Visual Studio adapter 2.5.0 (release notes)

It's been 11 months since the release of 2.4.2.

Aside from normal quality of life improvements, the primary feature of this release is a heavily updated assertion library, which includes several new assertions, several new overloads for existing assertions, and a pass to ensure that the messages from the assertions are now more concise and consistent. We wrote up an overview of the updates in the 2.5.0 assertion library messages. For the most part, developers should not notice much compile-time differences (aside from perhaps the occasional type ambiguity that may come from new overloads); there are also some breaking binary changes, most prominently with the gutting and reworking of the exception classes that are thrown by the individual assertions. Because of some of these breaking changes, we felt it was warranted to bump the version up to 2.5.0.

Important

This release removes all support for UWP from the core framework and VSTest adapter, based on Microsoft's deprecation of UWP in favor of WinUI 3 (and now Maui). Users who need to continue to work with UWP will need to stay with the v2 2.4.2 core framework and 2.4.5 Visual Studio runner.

As always, we'd like to thank all the users who contributed to the success of xUnit.net through usage, feedback, and code. 🎉

Note

We have moved away from Twitter for our occasional updates, and now make those available on our Mastodon account (@xunit@dotnet.social) and Bluesky account (@xunit.net).

Release Notes

These release notes are a comprehensive list of changes from 2.4.2 to 2.5.0.

Core Framework

  • Four new types are supported for serialization, which means they can be used in theory data without breaking your ability to run the individual test cases in Visual Studio Test Explorer. These are:

  • TimeSpan (supported on all frameworks)

  • BigInteger (supported on all frameworks except .NET Framework)

  • DateOnly (supported on .NET 6 and later)

  • TimeOnly (supported on .NET 6 and later)

xunit/xunit#2640

  • Added assembly-wide ExcludeFromCodeCoverageAttribute to all the xUnit.net assemblies. This should resolve issues with dotnet test ---collect "Code Coverage" (the new cross-platform code coverage system) incorrectly showing coverage for xUnit.net itself. xunit/xunit#2682

  • Updated support for TeamCity, to better support flow IDs, which should now correctly show hierarchy with test collections, as well as supported nested flows via TEAMCITY_PROCESS_FLOW_ID. xunit/xunit#2385

  • BUG: Fixed an issue where you could not run static test methods inherited from base classes. xunit/xunit#2704

Assertion Library

New assertions

  • Added a new assertion: Assert.IsNotAssignableFrom. It performs the opposite action of Assert.IsAssignableFrom. xunit/xunit#2727

New overloads

  • Assert.Contains behaved unexpectedly with dictionaries and sets. This is because these both have custom logic for their Contains function that is contrary to our normal container comparison logic. Their constructors allow you to pass key comparers; normally, comparers are only used when looking things up. This fixes this so it will have consistent behavior with the framework. xunit/xunit#2671

  • Added Assert.Equal(DateTime, DateTime) and Assert.Equal(DateTimeOffset, DateTimeOffset) overloads.

  • Added Assert.Equal(DateTime, DateTime, TimeSpan) and Assert.Equal(DateTimeOffset, DateTimeOffset, TimeSpan) overloads. They specify a precision (in time span) that the two times must be within one another. This precision works in both directions: expected earlier than actual, or expected later than actual. xunit/xunit#2588

  • Added Assert.Equal(T, T, Func<T, T, bool> comparer) and Assert.NotEqual(T, T, Func<T, T, bool> comparer) overloads. This allows the dev to write an inline comparison function rather than being forced to implement one of the equality interfaces (like IEqualityComparer<T>) in a separate class. xunit/xunit#2692

  • Assert.Equal() for strings has a new flag: ignoreAllWhiteSpace indicates that all whitespace should be ignored. The difference between this and ignoreWhiteSpaceDifferences is that the latter requires at least one piece of whitespace. Examples:
    Fails: Assert.Equal("a b", "ab", ignoreWhiteSpaceDifferences: true);
    Passes: Assert.Equal("a b", "ab", ignoreAllWhiteSpace: true); xunit/xunit#2440

  • Added Assert.NotNull and Assert.Null overloads explicitly for Nullable<T> struct values. Both include new messages, and the NotNull variant returns the unwrapped T value. This prevents boxing, and makes it trivial to use the non-null value. (We don't need this with the reference versions, since the type doesn't change, it just sheds its nullability, which we automatically do with the [NotNull] attribute.) xunit/xunit#2516

  • Added overloads of Assert.Contains and Assert.DoesNotContain for the concrete type ConcurrentDictionary. xunit/xunit#1857

Fixes

  • BUG: Assert.Equivalent with two equal values that are of different (but compatible) types previously failed. After discussion with the community, this was fixed to now pass. Example of code that now passes that previously failed: Assert.Equivalent(12, 12L); Poll on Mastodon

  • BUG: Fixed an issue where using the assertion library as source, combined with xunit.analyzers, causes compilation to report xUnit2007 and xUnit2015 warnings. These warnings aren't applicable to the assertion library itself, so they've been suppressed with pragmas. This means you should no longer need to ignore or disable those warnings when using xunit.assert.source (or the Git submodule). xunit/xunit#2472

  • BUG: Assert.Throws<ArgumentException>() threw an exception when you passed a null parameter name, despite being legal in ArgumentException. xunit/xunit#2396

  • BUG: Assert.Equivalent was not appropriately handling reference types inside of value types. xunit/xunit#2431

  • BUG: Assert.Equivalent was inadvertently comparing non-public properties. xunit/xunit#2621

  • BUG: Assert.Equivalent was incorrectly trying to compare indexer properties (which are more like functions than properties). Now, this will skip indexer properties. xunit/xunit#2698

  • BUG: The argument formatter did not correctly pretty-print rank-1 variable bound arrays properly. They are now show as type[*] correctly. xunit/xunit#2534

  • BUG: Resolved a compiler argument ambiguity problem by adding overloads Assert.Equal(float, float, int precision) and Assert.Equal(float, float, int precision, MidpointRounding). xunit/xunit#2393

  • BUG: IAssertionException was accidentally always public, even when importing via source and setting XUNIT_VISIBILITY_INTERNAL. xunit/xunit#2703

  • BUG: Restored an inadvertently removed constructor from XunitException which was breaking libraries like Fluent Assertions. xunit/xunit#2737

  • BUG: The type restriction on the obsoleted versions of Assert.ThrowsAny<T> for Func<Task> and Func<ValueTask> was set for ArgumentException, but should've been Exception.

Runners

  • We are now shipping .NET 4.8 and 4.8.1 versions of xunit.console.exe and xunit.console.x86.exe. New MSBuild parameters are available (when referencing the xunit.runner.console NuGet package) which point to the console runner executable on disk:

    • XunitConsole48Path
    • XunitConsole48PathX86
    • XunitConsole481Path
    • XunitConsole481PathX86

  • JSON configuration files can now use the new failSkips configuration element to convert skipped tests into failed tests. This aligns with the -failSkips console runner command line option and the FailSkips MSBuild runner property.

  • Added support for stopOnFail configuration file element in the console and MSBuild runners. This was documented as supported in the v2.3 schema, but complete support in all runners was not added until v2 2.5.

  • BUG: Fixed an issue with the console runner where sometimes, when using newer versions of Windows Terminal, the output of pointer characters (↓ and ↑) would be garbled as �.

Extensibility

  • The DiagnosticMessage class now implements ToString() and outputs the message, for simpler debugging for extensibility authors.

  • Fixed a performance issue with reporter discovery related to scanning every single DLL in your output folder looking for types which implement reporters. Now we only scan libraries named *reporters*.dll which may break any custom reporters you are using from first or third party assemblies without an appropriate name. xunit/visualstudio.xunit#317

  • This release makes ArgumentFormatter and CollectionTracker public (in the assertion library). The former is used to pretty print values in assertion failures as well arguments for display names of data-driven tests. The latter is a newly introduced class whose job is to prevent double enumeration while tracking the last few values visited, so that they can be printed when assertion failures happen without requiring a second enumeration.

  • We added several pragmas to disable warnings that would occur when XUNIT_NULLABLE was not defined, but your project had <nullable>enable</nullable>. You should always define XUNIT_NULLABLE when importing xunit.assert.source (or the Git submodule) when you have nullable enabled, but at least now if you forget to do that, there won't be any compiler warnings as a result of you using our non-nullable path. Note that we had to use pragmas to disable warnings rather than just using #nullable disable in the files, since that pragma didn't exist in C# prior to version 8 (and we set a minimum C# version of 6).

  • Updated TestFrameworkOptions.ForExecution to read the stopOnFail configuration file element and propagate the setting into the execution options.

  • BUG: Fixed an issue where a malformed FactAttribute-derived class could cause tests to become inappropriately ignored due to an exception throw during discovery. Now, when a test method is decorated with a misbehaving attribute, the test will still be discovered, and when run it will fail with information about the failure. Unaffected tests in the same class will no longer be affected by this failure. xunit/xunit#2719

Edit this page