E
Eirik Tsarpalis
Guest
The 9.0 release of System.Text.Json includes many features, primarily with a focus on JSON schema and intelligent application support. It also includes highly requested enhancements such as nullable reference type support, customizing enum member names, out-of-order metadata deserialization and customizing serialization indentation.
You can try out the new features by referencing the latest build of System.Text.Json NuGet package or the latest SDK for .NET 9.
The new
The resultant schema provides a specification of the JSON serialization contract for the type. As can be seen in this example, it distinguishes between nullable and non-nullable properties and populates the
The generated schema can be further controlled using the
Finally, users are able to apply their own transformations to generated schema nodes by specifying a
Tying it all together, we can now generate a schema that incorporates
This is a particularly useful component when it comes to generating schemas for .NET methods or APIs; it is being used to power ASP.NET Core’s newly released OpenAPI component and we’ve deployed it to a number of AI related libraries and applications with tool calling requirements such as Semantic Kernel, Visual Studio Copilot and the the newly released Microsoft.Extensions.AI library.
This additionally makes it possible to read JSON from payloads that may contain trailing data that is invalid JSON:
When it comes to streaming deserialization, we have included a new
Similarly, this setting adds enforcement on deserialization:
Due to how non-nullable reference types are implemented, this feature comes with an important number of limitations that users need to familiarize themselves with before turning it on. The root of the issue is that reference type nullability has no first-class representation in IL, as such the expressions
If you are looking to add nullability enforcement in these cases, we recommend that you either model your type to be a struct (since they do not admit
Users can turn on the
It should be noted that
This is because STJ treats required and non-nullable properties as orthogonal concepts. This stems from the C# language itself where you can have
And optional properties that are non-nullable:
The same orthogonality applies to constructor parameters:
STJ constructor-based deserialization has historically been treating all constructor parameters as optional, as can be highlighted in the following example:
In .NET 9 we’re including the
Users can turn on the
Both the
The new
Certain features of System.Text.Json such as polymorphism or
By default, the STJ metadata reader requires that the metadata properties
This is known to create problems when needing to deserialize JSON payloads that do not originate from System.Text.Json. The new
Care should be taken when enabling this flag, as it might result in over-buffering (and OOM failures) when performing streaming deserialization of very large JSON objects. This is because metadata properties must be read before the deserialize object is instantiated, meaning that all properties preceding a
The
The
This allows modifications to object instances that can directly influence property order:
The new
The new
For a detailed write-up of System.Text.Json performance improvements in .NET 9, please refer to the relevant section in Stephen Toub’s “Performance Improvements in .NET 9” article.
.NET 9 sees a large number of new features and quality of life improvements with a focus on JSON schema and intelligent application support. In total 46 pull requests contributed to System.Text.Json during .NET 9 development. We’d like you to try the new features and give us feedback on how it improves your applications, and any usability issues or bugs that you might encounter.
Community contributions are always welcome. If you’d like to contribute to System.Text.Json, check out our list of
The post What’s new in System.Text.Json in .NET 9 appeared first on .NET Blog.
Continue reading...
Getting the latest bits
You can try out the new features by referencing the latest build of System.Text.Json NuGet package or the latest SDK for .NET 9.
JSON Schema Exporter
The new
JsonSchemaExporter
class can be used to extract JSON schema documents from .NET types using either JsonSerializerOptions
or JsonTypeInfo
instances:
Code:
using System.Text.Json.Schema;
JsonSerializerOptions options = JsonSerializerOptions.Default;
JsonNode schema = options.GetJsonSchemaAsNode(typeof(Person));
Console.WriteLine(schema.ToString());
//{
// "type": ["object", "null"],
// "properties": {
// "Name": { "type": "string" },
// "Age": { "type": "integer" },
// "Address": { "type": ["string", "null"], "default": null }
// },
// "required": ["Name", "Age"]
//}
record Person(string Name, int Age, string? Address = null);
The resultant schema provides a specification of the JSON serialization contract for the type. As can be seen in this example, it distinguishes between nullable and non-nullable properties and populates the
required
keyword by virtue of a constructor parameter being optional or not. The schema output can be influenced by configuration specified in the JsonSerializerOptions
or JsonTypeInfo
instances:
Code:
JsonSerializerOptions options = new(JsonSerializerOptions.Default)
{
PropertyNamingPolicy = JsonNamingPolicy.KebabCaseUpper,
NumberHandling = JsonNumberHandling.WriteAsString,
UnmappedMemberHandling = JsonUnmappedMemberHandling.Disallow,
};
JsonNode schema = options.GetJsonSchemaAsNode(typeof(MyPoco));
Console.WriteLine(schema.ToString());
//{
// "type": ["object", "null"],
// "properties": {
// "NUMERIC-VALUE": {
// "type": ["string", "integer"],
// "pattern": "^-?(?:0|[1-9]\\d*)$"
// }
// },
// "additionalProperties": false
//}
class MyPoco
{
public int NumericValue { get; init; }
}
The generated schema can be further controlled using the
JsonSchemaExporterOptions
configuration type:
Code:
JsonSerializerOptions options = JsonSerializerOptions.Default;
JsonSchemaExporterOptions exporterOptions = new()
{
// Marks root-level types as non-nullable
TreatNullObliviousAsNonNullable = true,
};
JsonNode schema = options.GetJsonSchemaAsNode(typeof(Person), exporterOptions);
Console.WriteLine(schema.ToString());
//{
// "type": "object",
// "properties": {
// "Name": { "type": "string" }
// },
// "required": ["Name"]
//}
record Person(string Name);
Finally, users are able to apply their own transformations to generated schema nodes by specifying a
TransformSchemaNode
delegate. Here’s an example that incorporates text from DescriptionAttribute
annotations:
Code:
JsonSchemaExporterOptions exporterOptions = new()
{
TransformSchemaNode = (context, schema) =>
{
// Determine if a type or property and extract the relevant attribute provider
ICustomAttributeProvider? attributeProvider = context.PropertyInfo is not null
? context.PropertyInfo.AttributeProvider
: context.TypeInfo.Type;
// Look up any description attributes
DescriptionAttribute? descriptionAttr = attributeProvider?
.GetCustomAttributes(inherit: true)
.Select(attr => attr as DescriptionAttribute)
.FirstOrDefault(attr => attr is not null);
// Apply description attribute to the generated schema
if (descriptionAttr != null)
{
if (schema is not JsonObject jObj)
{
// Handle the case where the schema is a boolean
JsonValueKind valueKind = schema.GetValueKind();
Debug.Assert(valueKind is JsonValueKind.True or JsonValueKind.False);
schema = jObj = new JsonObject();
if (valueKind is JsonValueKind.False)
{
jObj.Add("not", true);
}
}
jObj.Insert(0, "description", descriptionAttr.Description);
}
return schema;
}
};
Tying it all together, we can now generate a schema that incorporates
description
keyword source from attribute annotations:
Code:
JsonNode schema = options.GetJsonSchemaAsNode(typeof(Person), exporterOptions);
Console.WriteLine(schema.ToString());
//{
// "description": "A person",
// "type": ["object", "null"],
// "properties": {
// "Name": { "description": "The name of the person", "type": "string" }
// },
// "required": ["Name"]
//}
[Description("A person")]
record Person([property: Description("The name of the person")] string Name);
This is a particularly useful component when it comes to generating schemas for .NET methods or APIs; it is being used to power ASP.NET Core’s newly released OpenAPI component and we’ve deployed it to a number of AI related libraries and applications with tool calling requirements such as Semantic Kernel, Visual Studio Copilot and the the newly released Microsoft.Extensions.AI library.
Streaming multiple JSON documents
Utf8JsonReader
now supports reading multiple, whitespace-separated JSON documents from a single buffer or stream. By default, Utf8JsonReader
will throw an exception if it detects any non-whitespace characters that are trailing the first top-level document. This behavior can be changed using the JsonReaderOptions.AllowMultipleValues
flag:
Code:
JsonReaderOptions options = new() { AllowMultipleValues = true };
Utf8JsonReader reader = new("null {} 1 \r\n [1,2,3]"u8, options);
reader.Read();
Console.WriteLine(reader.TokenType); // Null
reader.Read();
Console.WriteLine(reader.TokenType); // StartObject
reader.Skip();
reader.Read();
Console.WriteLine(reader.TokenType); // Number
reader.Read();
Console.WriteLine(reader.TokenType); // StartArray
reader.Skip();
Console.WriteLine(reader.Read()); // False
This additionally makes it possible to read JSON from payloads that may contain trailing data that is invalid JSON:
Code:
Utf8JsonReader reader = new("[1,2,3] <NotJson/>"u8, new() { AllowMultipleValues = true });
reader.Read();
reader.Skip(); // Success
reader.Read(); // throws JsonReaderException
When it comes to streaming deserialization, we have included a new
JsonSerializer.DeserializeAsyncEnumerable
overload that makes streaming multiple top-level values possible. By default, DeserializeAsyncEnumerable
will attempt to stream elements that are contained in a top-level JSON array. This behavior can be toggled using the new topLevelValues
flag:
Code:
ReadOnlySpan<byte> utf8Json = """[0] [0,1] [0,1,1] [0,1,1,2] [0,1,1,2,3]"""u8;
using var stream = new MemoryStream(utf8Json.ToArray());
await foreach (int[] item in JsonSerializer.DeserializeAsyncEnumerable<int[]>(stream, topLevelValues: true))
{
Console.WriteLine(item.Length);
}
Respecting nullable annotations
JsonSerializer
now adds limited support for non-nullable reference type enforcement in serialization and deserialization. This can be toggled using the RespectNullableAnnotations
flag:
Code:
#nullable enable
JsonSerializerOptions options = new() { RespectNullableAnnotations = true };
MyPoco invalidValue = new(Name: null!);
JsonSerializer.Serialize(invalidValue, options);
// System.Text.Json.JsonException: The property or field 'Name'
// on type 'MyPoco' doesn't allow getting null values. Consider
// updating its nullability annotation.
record MyPoco(string Name);
Similarly, this setting adds enforcement on deserialization:
Code:
JsonSerializerOptions options = new() { RespectNullableAnnotations = true };
string json = """{"Name":null}""";
JsonSerializer.Deserialize<MyPoco>(json, options);
// System.Text.Json.JsonException: The constructor parameter 'Name'
// on type 'MyPoco' doesn't allow null values. Consider updating
// its nullability annotation.
Limitations
Due to how non-nullable reference types are implemented, this feature comes with an important number of limitations that users need to familiarize themselves with before turning it on. The root of the issue is that reference type nullability has no first-class representation in IL, as such the expressions
MyPoco
and MyPoco?
are indistinguishable from the perspective of run-time reflection. While the compiler will try to make up for that by emitting attribute metadata where possible, this is restricted to non-generic member annotations that are scoped to a particular type definition. It is for this reason that the flag only validates nullability annotations that are present on non-generic properties, fields, and constructor parameters. System.Text.Json does not support nullability enforcement on- Top-level types, aka the type that is passed when making the first
JsonSerializer.(De)serialize
call. - Collection element types, aka we cannot distinguish between
List<string>
andList<string?>
types. - Any properties, fields, or constructor parameters that are generic.
If you are looking to add nullability enforcement in these cases, we recommend that you either model your type to be a struct (since they do not admit
null
values) or author a custom converter that overrides its HandleNull
property to true
.Feature switch
Users can turn on the
RespectNullableAnnotations
setting globally using the System.Text.Json.Serialization.RespectNullableAnnotationsDefault
feature switch, which can be set via your project configuration:
Code:
<ItemGroup>
<RuntimeHostConfigurationOption Include="System.Text.Json.Serialization.RespectNullableAnnotationsDefault" Value="true" />
</ItemGroup>
On the relation between nullable and optional parameters
It should be noted that
RespectNullableAnnotations
does not extend enforcement to unspecified JSON values:
Code:
JsonSerializerOptions options = new() { RespectNullableAnnotations = true };
var result = JsonSerializer.Deserialize<MyPoco>("{}", options); // No exception!
Console.WriteLine(result.Name is null); // True
class MyPoco
{
public string Name { get; set; }
}
This is because STJ treats required and non-nullable properties as orthogonal concepts. This stems from the C# language itself where you can have
required
properties that are nullable:
Code:
MyPoco poco = new() { Value = null }; // No compiler warnings
class MyPoco
{
public required string? Value { get; set; }
}
And optional properties that are non-nullable:
Code:
class MyPoco
{
public string Value { get; set; } = "default";
}
The same orthogonality applies to constructor parameters:
Code:
record MyPoco(
string RequiredNonNullable,
string? RequiredNullable,
string OptionalNonNullable = "default",
string? OptionalNullable = "default");
Respecting non-optional constructor parameters
STJ constructor-based deserialization has historically been treating all constructor parameters as optional, as can be highlighted in the following example:
Code:
var result = JsonSerializer.Deserialize<Person>("{}");
Console.WriteLine(result); // Person { Name = , Age = 0 }
record Person(string Name, int Age);
In .NET 9 we’re including the
RespectRequiredConstructorParameters
flag that changes the behavior such that non-optional constructor parameters are now treated as required:
Code:
JsonSerializerOptions options = new() { RespectRequiredConstructorParameters = true };
string json = """{"Optional": "value"}""";
JsonSerializer.Deserialize<MyPoco>(json, options);
record MyPoco(string Required, string? Optional = null);
// JsonException: JSON deserialization for type 'MyPoco'
// was missing required properties including: 'Required'.
Feature switch
Users can turn on the
RespectRequiredConstructorParameters
setting globally using the System.Text.Json.Serialization.RespectRequiredConstructorParametersDefault
feature switch, which can be set via your project configuration:
Code:
<ItemGroup>
<RuntimeHostConfigurationOption Include="System.Text.Json.Serialization.RespectRequiredConstructorParametersDefault" Value="true" />
</ItemGroup>
Both the
RespectNullableAnnotations
and RespectRequiredConstructorParameter
properties were implemented as opt-in flags to avoid breaking existing applications. If you’re writing a new application, it is highly recommended that you enable both flags in your code.Customizing enum member names
The new
JsonStringEnumMemberName
attribute can be used to customize the names of individual enum members for types that are serialized as strings:
Code:
JsonSerializer.Serialize(MyEnum.Value1 | MyEnum.Value2); // "Value1, Custom enum value"
[Flags, JsonConverter(typeof(JsonStringEnumConverter))]
enum MyEnum
{
Value1 = 1,
[JsonStringEnumMemberName("Custom enum value")]
Value2 = 2,
}
Out-of-order metadata reads
Certain features of System.Text.Json such as polymorphism or
ReferenceHandler.Preserve
require emitting metadata properties over the wire:
Code:
JsonSerializerOptions options = new() { ReferenceHandler = ReferenceHandler.Preserve };
Base value = new Derived("Name");
JsonSerializer.Serialize(value, options); // {"$id":"1","$type":"derived","Name":"Name"}
[JsonDerivedType(typeof(Derived), "derived")]
record Base;
record Derived(string Name) : Base;
By default, the STJ metadata reader requires that the metadata properties
$id
and $type
must be defined at the start of the JSON object:
Code:
JsonSerializer.Deserialize<Base>("""{"Name":"Name","$type":"derived"}""");
// JsonException: The metadata property is either not supported by the
// type or is not the first property in the deserialized JSON object.
This is known to create problems when needing to deserialize JSON payloads that do not originate from System.Text.Json. The new
AllowOutOfOrderMetadataProperties
can be configured to disable this restriction:
Code:
JsonSerializerOptions options = new() { AllowOutOfOrderMetadataProperties = true };
JsonSerializer.Deserialize<Base>("""{"Name":"Name","$type":"derived"}""", options); // Success
Care should be taken when enabling this flag, as it might result in over-buffering (and OOM failures) when performing streaming deserialization of very large JSON objects. This is because metadata properties must be read before the deserialize object is instantiated, meaning that all properties preceding a
$type
property must be kept in the buffer for subsequent property binding.Customizing indentation
The
JsonWriterOptions
and JsonSerializerOptions
types now expose APIs for configuring indentation. The following example enables single-tab indentation:
Code:
JsonSerializerOptions options = new()
{
WriteIndented = true,
IndentCharacter = '\t',
IndentSize = 1,
};
JsonSerializer.Serialize(new { Value = 42 }, options);
JsonObject
property order manipulation
The
JsonObject
type is part of the mutable DOM and is used to represent JSON objects. Even though the type is modelled as an IDictionary<string, JsonNode>
, it does encapsulate an implicit property order that is not user-modifiable. The new release exposes additional APIs that effectively model the type as an ordered dictionary:
Code:
public partial class JsonObject : IList<KeyValuePair<string, JsonNode?>>
{
public int IndexOf(string key);
public void Insert(int index, string key, JsonNode? value);
public void RemoveAt(int index);
}
This allows modifications to object instances that can directly influence property order:
Code:
// Adds or moves the $id property to the start of the object
var schema = (JsonObject)JsonSerializerOptions.Default.GetJsonSchemaAsNode(typeof(MyPoco));
switch (schema.IndexOf("$id", out JsonNode? idValue))
{
case < 0: // $id property missing
idValue = (JsonNode)"https://example.com/schema";
schema.Insert(0, "$id", idValue);
break;
case 0: // $id property already at the start of the object
break;
case int index: // $id exists but not at the start of the object
schema.RemoveAt(index);
schema.Insert(0, "$id", idValue);
}
DeepEquals
methods in JsonElement
and JsonNode
The new
JsonElement.DeepEquals
method extends deep equality comparison to JsonElement
instances, complementing the pre-existing JsonNode.DeepEquals
method. Additionally, both methods include refinements in their implementation, such as handling of equivalent JSON numeric representations:
Code:
JsonElement left = JsonDocument.Parse("10e-3").RootElement;
JsonElement right = JsonDocument.Parse("0.01").RootElement;
JsonElement.DeepEquals(left, right); // True
JsonSerializerOptions.Web
The new
JsonSerializerOptions.Web
singleton can be used to quickly serialize values using JsonSerializerDefaults.Web
settings:
Code:
JsonSerializerOptions options = JsonSerializerOptions.Web; // used instead of new(JsonSerializerDefaults.Web);
JsonSerializer.Serialize(new { Value = 42 }, options); // {"value":42}
Performance improvements
For a detailed write-up of System.Text.Json performance improvements in .NET 9, please refer to the relevant section in Stephen Toub’s “Performance Improvements in .NET 9” article.
Closing
.NET 9 sees a large number of new features and quality of life improvements with a focus on JSON schema and intelligent application support. In total 46 pull requests contributed to System.Text.Json during .NET 9 development. We’d like you to try the new features and give us feedback on how it improves your applications, and any usability issues or bugs that you might encounter.
Community contributions are always welcome. If you’d like to contribute to System.Text.Json, check out our list of
help wanted
issues on GitHub.The post What’s new in System.Text.Json in .NET 9 appeared first on .NET Blog.
Continue reading...