Core Framework v2 2.5.0 Assertion Updates
This page lists several example message updates for assertion failures, comparing 2.4.2 to 2.5.0 output. This list is not comprehensive, but rather a list of some of the more obvious improvements in clarity and consistency.
Assert.All
When more than one failure occurs, the list of failures is printed. Note that the list in 2.4.2 was printed in reverse order; the order is correct now in 2.5.0.
Sample test
[Fact]
public void Sample()
{
var data = new[] { 1, 3, 4 };
Assert.All(data, item => Assert.True(item % 2 == 0));
}
Output from 2.4.2
Assert.All() Failure: 2 out of 3 items in the collection did not pass.
[1]: Item: 3
Xunit.Sdk.TrueException: Assert.True() Failure
Expected: True
Actual: False
at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
at Xunit.Assert.True(Boolean condition) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 87
at Examples.<>c.<Sample>b__0_0(Int32 item) in C:\Dev\repro\comparison\xunit-2.4.2\Examples.cs:line 8
at Xunit.Assert.<>c__DisplayClass11_0`1.<All>b__0(T item, Int32 index) in /_/src/xunit.assert/Asserts/CollectionAsserts.cs:line 34
at Xunit.Assert.All[T](IEnumerable`1 collection, Action`2 action) in /_/src/xunit.assert/Asserts/CollectionAsserts.cs:line 61
[0]: Item: 1
Xunit.Sdk.TrueException: Assert.True() Failure
Expected: True
Actual: False
at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
at Xunit.Assert.True(Boolean condition) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 87
at Examples.<>c.<Sample>b__0_0(Int32 item) in C:\Dev\repro\comparison\xunit-2.4.2\Examples.cs:line 8
at Xunit.Assert.<>c__DisplayClass11_0`1.<All>b__0(T item, Int32 index) in /_/src/xunit.assert/Asserts/CollectionAsserts.cs:line 34
at Xunit.Assert.All[T](IEnumerable`1 collection, Action`2 action) in /_/src/xunit.assert/Asserts/CollectionAsserts.cs:line 61
Output from 2.5.0
Assert.All() Failure: 2 out of 3 items in the collection did not pass.
[0]: Item: 1
Error: Assert.True() Failure
Expected: True
Actual: False
[1]: Item: 3
Error: Assert.True() Failure
Expected: True
Actual: False
Assert.Collection
In 2.4.2, length check happens first and fails fast. In 2.5.0, comparisons happen before length checks, which often makes it easier to spot the rogue item in the collection.
Sample test (correct length)
[Fact]
public void Sample()
{
var data = new[] { 1, 3, 4 };
Assert.Collection(
data,
item => Assert.Equal(1, item),
item => Assert.Equal(2, item),
item => Assert.Equal(5, item)
);
}
Output from 2.4.2
Assert.Collection() Failure
Collection: [1, 3, 4]
Error during comparison of item at index 1
Inner exception: Assert.Equal() Failure
Expected: 2
Actual: 3
Output from 2.5.0
Assert.Collection() Failure: Item comparison failure
↓ (pos 1)
Collection: [1, 3, 4]
Error: Assert.Equal() Failure: Values differ
Expected: 2
Actual: 3
Sample test (mismatched length, with incorrect items)
[Fact]
public void Sample()
{
var data = new[] { 1, 2, 3, 4 };
Assert.Collection(
data,
item => Assert.Equal(1, item),
item => Assert.Equal(2, item),
item => Assert.Equal(5, item)
);
}
Output from 2.4.2
Assert.Collection() Failure
Collection: [1, 2, 3, 4]
Expected item count: 3
Actual item count: 4
Output from 2.5.0
Assert.Collection() Failure: Item comparison failure
↓ (pos 2)
Collection: [1, 2, 3, 4]
Error: Assert.Equal() Failure: Values differ
Expected: 5
Actual: 3
Sample test (mismatched length, with correct items)
[Fact]
public void Sample()
{
var data = new[] { 1, 2, 3, 4 };
Assert.Collection(
data,
item => Assert.Equal(1, item),
item => Assert.Equal(2, item),
item => Assert.Equal(3, item)
);
}
Output from 2.4.2
Assert.Collection() Failure
Collection: [1, 2, 3, 4]
Expected item count: 3
Actual item count: 4
Output from 2.5.0
Assert.Collection() Failure: Mismatched item count
Collection: [1, 2, 3, 4]
Expected count: 3
Actual count: 4
Assert.Contains
In addition to the updated consistency in messaging, this also shows how unnecessary types are no longer printed. When types are important (for example, if two collections are compared and they're of different types), they are still provided.
Sample test
[Fact]
public void Sample()
{
var data = new[] { 1, 2, 3, 4 };
Assert.Contains(5, data);
}
Output from 2.4.2
Assert.Contains() Failure
Not found: 5
In value: Int32[] [1, 2, 3, 4]
Output from 2.5.0
Assert.Contains() Failure: Item not found in collection
Collection: [1, 2, 3, 4]
Not found: 5
Assert.Distinct
This is a simple consistency update.
Sample test
[Fact]
public void Sample()
{
var data = new[] { 1, 2, 3, 2 };
Assert.Distinct(data);
}
Output from 2.4.2
Assert.Distinct() Failure: The item 2 occurs multiple times in [1, 2, 3, 2].
Output from 2.5.0
Assert.Distinct() Failure: Duplicate item found
Collection: [1, 2, 3, 2]
Item: 2
Assert.Empty
This removes unnecessary information along with using a consistent message.
Sample test
[Fact]
public void Sample()
{
var data = new[] { 1, 2, 3, 2 };
Assert.Empty(data);
}
Output from 2.4.2
Assert.Empty() Failure
Expected: <empty>
Actual: [1, 2, 3, 2]
Output from 2.5.0
Assert.Empty() Failure: Collection was not empty
Collection: [1, 2, 3, 2]
Assert.EndsWith
The 2.4.2 version of this assertion only showed the exact end of the string that was being compared. The 2.5.0 version now prints as much of the string as is reasonable, using our standard truncation mechanism.
Sample test
[Fact]
public void Sample()
{
Assert.EndsWith("ending", "This is a relatively long string with a variant end");
}
Output from 2.4.2
Assert.EndsWith() Failure:
Expected: ending
Actual: ···nt end
Output from 2.5.0
Assert.EndsWith() Failure: String end does not match
String: ···"relatively long string with a variant end"
Expected end: "ending"
Assert.Equal
Assert.Equal
is a heavy lifting function, with many overloads (19 and counting!) and many ways data can be handled. Below are a couple examples of those changes.
Sample test (strings)
[Fact]
public void Sample()
{
Assert.Equal(
"This is a relatively long string with a difference embedded in the middle somewhere",
"This is a relatively long string that has a difference embedded in the middle somewhere"
);
}
Output from 2.4.2
Assert.Equal() Failure
↓ (pos 33)
Expected: ···atively long string with a difference embedded in the middle ···
Actual: ···atively long string that has a difference embedded in the mid···
↑ (pos 33)
Output from 2.5.0
Assert.Equal() Failure: Strings differ
↓ (pos 33)
Expected: ···"atively long string with a difference emb"···
Actual: ···"atively long string that has a difference"···
↑ (pos 33)
Sample test (collections)
[Fact]
public void Sample()
{
Assert.Equal(
new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 },
new[] { 1, 2, 3, 4, 0, 6, 7, 8, 9 }
);
}
Output from 2.4.2
Assert.Equal() Failure
↓ (pos 4)
Expected: [..., 3, 4, 5, 6, 7, ...]
Actual: [..., 3, 4, 0, 6, 7, ...]
↑ (pos 4)
Output from 2.5.0
Assert.Equal() Failure: Collections differ
↓ (pos 4)
Expected: [···, 3, 4, 5, 6, 7, ···]
Actual: [···, 3, 4, 0, 6, 7, ···]
↑ (pos 4)
Assert.Multiple
This is a simple consistency update.
Sample test
[Fact]
public void Sample()
{
Assert.Multiple(
() => Assert.Equal(12, 24),
() => Assert.Null("Hello world")
);
}
Output from 2.4.2
Multiple failures were encountered:
---- Assert.Equal() Failure
Expected: 12
Actual: 24
---- Assert.Null() Failure
Expected: (null)
Actual: Hello world
Output from 2.5.0
Assert.Multiple() Failure: Multiple failures were encountered
---- Assert.Equal() Failure: Values differ
Expected: 12
Actual: 24
---- Assert.Null() Failure: Value is not null
Expected: null
Actual: "Hello world"
Assert.Single
This is an extreme example of a needed consistency update.
Sample test
[Fact]
public void Sample()
{
Assert.Single(new[] { 1, 2, 3 });
}
Output from 2.4.2
The collection was expected to contain a single element, but it contained 3 elements.
Output from 2.5.0
Assert.Single() Failure: The collection contained 3 items
Collection: [1, 2, 3]
Assert.Throws
The overload of Assert.Throws
that can test the parameter name for any instance of ArgumentException
now returns a much more meaningful message.
Sample test
[Fact]
public void Sample()
{
void thrower() => throw new ArgumentException("Hello world", "someArg");
Assert.Throws<ArgumentException>("otherArg", thrower);
}
Output from 2.4.2
Assert.Equal() Failure
↓ (pos 0)
Expected: otherArg
Actual: someArg
↑ (pos 0)
Output from 2.5.0
Assert.Throws() Failure: Incorrect parameter name
Exception: typeof(System.ArgumentException)
Expected: "otherArg"
Actual: "someArg"