What’s new in System.Text.Json in .NET 9

[Eirik Tsarpalis]

The 9.0 release of System.Text.Json includes many features, primarily with a focus on JSON schema and intelligent application support. It also includes highly requested enhancements such as nullable reference type support, customizing enum member names, out-of-order metadata deserialization and customizing serialization indentation.

Getting the latest bits​

You can try out the new features by referencing the latest build of System.Text.Json NuGet package or the latest SDK for .NET 9.

JSON Schema Exporter​

The new JsonSchemaExporter class can be used to extract JSON schema documents from .NET types using either JsonSerializerOptions or JsonTypeInfo instances:

using System.Text.Json.Schema;

JsonSerializerOptions options = JsonSerializerOptions.Default;
JsonNode schema = options.GetJsonSchemaAsNode(typeof(Person));
Console.WriteLine(schema.ToString());
//{
//  "type": ["object", "null"],
//  "properties": {
//    "Name": { "type": "string" },
//    "Age": { "type": "integer" },
//    "Address": { "type": ["string", "null"], "default": null }
//  },
//  "required": ["Name", "Age"]
//}

record Person(string Name, int Age, string? Address = null);

The resultant schema provides a specification of the JSON serialization contract for the type. As can be seen in this example, it distinguishes between nullable and non-nullable properties and populates the required keyword by virtue of a constructor parameter being optional or not. The schema output can be influenced by configuration specified in the JsonSerializerOptions or JsonTypeInfo instances:

JsonSerializerOptions options = new(JsonSerializerOptions.Default)
{
    PropertyNamingPolicy = JsonNamingPolicy.KebabCaseUpper,
    NumberHandling = JsonNumberHandling.WriteAsString,
    UnmappedMemberHandling = JsonUnmappedMemberHandling.Disallow,
};

JsonNode schema = options.GetJsonSchemaAsNode(typeof(MyPoco));
Console.WriteLine(schema.ToString());
//{
//  "type": ["object", "null"],
//  "properties": {
//    "NUMERIC-VALUE": {
//      "type": ["string", "integer"],
//      "pattern": "^-?(?:0|[1-9]\\d*)$"
//    }
//  },
//  "additionalProperties": false
//}

class MyPoco
{
    public int NumericValue { get; init; }
}

The generated schema can be further controlled using the JsonSchemaExporterOptions configuration type:

JsonSerializerOptions options = JsonSerializerOptions.Default;
JsonSchemaExporterOptions exporterOptions = new()
{
    // Marks root-level types as non-nullable
    TreatNullObliviousAsNonNullable = true,
};

JsonNode schema = options.GetJsonSchemaAsNode(typeof(Person), exporterOptions);
Console.WriteLine(schema.ToString());
//{
//  "type": "object",
//  "properties": {
//    "Name": { "type": "string" }
//  },
//  "required": ["Name"]
//}

record Person(string Name);

Finally, users are able to apply their own transformations to generated schema nodes by specifying a TransformSchemaNode delegate. Here’s an example that incorporates text from DescriptionAttribute annotations:

JsonSchemaExporterOptions exporterOptions = new()
{
    TransformSchemaNode = (context, schema) =>
    {
        // Determine if a type or property and extract the relevant attribute provider
        ICustomAttributeProvider? attributeProvider = context.PropertyInfo is not null
            ? context.PropertyInfo.AttributeProvider
            : context.TypeInfo.Type;

        // Look up any description attributes
        DescriptionAttribute? descriptionAttr = attributeProvider?
            .GetCustomAttributes(inherit: true)
            .Select(attr => attr as DescriptionAttribute)
            .FirstOrDefault(attr => attr is not null);

        // Apply description attribute to the generated schema
        if (descriptionAttr != null)
        {
            if (schema is not JsonObject jObj)
            {
                // Handle the case where the schema is a boolean
                JsonValueKind valueKind = schema.GetValueKind();
                Debug.Assert(valueKind is JsonValueKind.True or JsonValueKind.False);
                schema = jObj = new JsonObject();
                if (valueKind is JsonValueKind.False)
                {
                    jObj.Add("not", true);
                }
            }

            jObj.Insert(0, "description", descriptionAttr.Description);
        }

        return schema;
    }
};

Tying it all together, we can now generate a schema that incorporates description keyword source from attribute annotations:

JsonNode schema = options.GetJsonSchemaAsNode(typeof(Person), exporterOptions);
Console.WriteLine(schema.ToString());
//{
//  "description": "A person",
//  "type": ["object", "null"],
//  "properties": {
//    "Name": { "description": "The name of the person", "type": "string" }
//  },
//  "required": ["Name"]
//}

[Description("A person")]
record Person([property: Description("The name of the person")] string Name);

This is a particularly useful component when it comes to generating schemas for .NET methods or APIs; it is being used to power ASP.NET Core’s newly released OpenAPI component and we’ve deployed it to a number of AI related libraries and applications with tool calling requirements such as Semantic Kernel, Visual Studio Copilot and the the newly released Microsoft.Extensions.AI library.

Streaming multiple JSON documents​

Utf8JsonReader now supports reading multiple, whitespace-separated JSON documents from a single buffer or stream. By default, Utf8JsonReader will throw an exception if it detects any non-whitespace characters that are trailing the first top-level document. This behavior can be changed using the JsonReaderOptions.AllowMultipleValues flag:

JsonReaderOptions options = new() { AllowMultipleValues = true };
Utf8JsonReader reader = new("null {} 1 \r\n [1,2,3]"u8, options);

reader.Read();
Console.WriteLine(reader.TokenType); // Null

reader.Read();
Console.WriteLine(reader.TokenType); // StartObject
reader.Skip();

reader.Read();
Console.WriteLine(reader.TokenType); // Number

reader.Read();
Console.WriteLine(reader.TokenType); // StartArray
reader.Skip();

Console.WriteLine(reader.Read()); // False

This additionally makes it possible to read JSON from payloads that may contain trailing data that is invalid JSON:

Utf8JsonReader reader = new("[1,2,3]    <NotJson/>"u8, new() { AllowMultipleValues = true });

reader.Read();
reader.Skip(); // Success
reader.Read(); // throws JsonReaderException

When it comes to streaming deserialization, we have included a new JsonSerializer.DeserializeAsyncEnumerable overload that makes streaming multiple top-level values possible. By default, DeserializeAsyncEnumerable will attempt to stream elements that are contained in a top-level JSON array. This behavior can be toggled using the new topLevelValues flag:

ReadOnlySpan<byte> utf8Json = """[0] [0,1] [0,1,1] [0,1,1,2] [0,1,1,2,3]"""u8;
using var stream = new MemoryStream(utf8Json.ToArray());

await foreach (int[] item in JsonSerializer.DeserializeAsyncEnumerable<int[]>(stream, topLevelValues: true))
{
    Console.WriteLine(item.Length);
}

Respecting nullable annotations​

JsonSerializer now adds limited support for non-nullable reference type enforcement in serialization and deserialization. This can be toggled using the RespectNullableAnnotations flag:

#nullable enable
JsonSerializerOptions options = new() { RespectNullableAnnotations = true };

MyPoco invalidValue = new(Name: null!);
JsonSerializer.Serialize(invalidValue, options);
// System.Text.Json.JsonException: The property or field 'Name'
// on type 'MyPoco' doesn't allow getting null values. Consider
// updating its nullability annotation. 

record MyPoco(string Name);

Similarly, this setting adds enforcement on deserialization:

JsonSerializerOptions options = new() { RespectNullableAnnotations = true };
string json = """{"Name":null}""";
JsonSerializer.Deserialize<MyPoco>(json, options);
// System.Text.Json.JsonException: The constructor parameter 'Name'
// on type 'MyPoco' doesn't allow null values. Consider updating
// its nullability annotation.

Limitations​

Due to how non-nullable reference types are implemented, this feature comes with an important number of limitations that users need to familiarize themselves with before turning it on. The root of the issue is that reference type nullability has no first-class representation in IL, as such the expressions MyPoco and MyPoco? are indistinguishable from the perspective of run-time reflection. While the compiler will try to make up for that by emitting attribute metadata where possible, this is restricted to non-generic member annotations that are scoped to a particular type definition. It is for this reason that the flag only validates nullability annotations that are present on non-generic properties, fields, and constructor parameters. System.Text.Json does not support nullability enforcement on

• Top-level types, aka the type that is passed when making the first JsonSerializer.(De)serialize call.
• Collection element types, aka we cannot distinguish between List<string> and List<string?> types.
• Any properties, fields, or constructor parameters that are generic.

If you are looking to add nullability enforcement in these cases, we recommend that you either model your type to be a struct (since they do not admit null values) or author a custom converter that overrides its HandleNull property to true.

Feature switch​

Users can turn on the RespectNullableAnnotations setting globally using the System.Text.Json.Serialization.RespectNullableAnnotationsDefault feature switch, which can be set via your project configuration:

<ItemGroup>
  <RuntimeHostConfigurationOption Include="System.Text.Json.Serialization.RespectNullableAnnotationsDefault" Value="true" />
</ItemGroup>

On the relation between nullable and optional parameters​

It should be noted that RespectNullableAnnotations does not extend enforcement to unspecified JSON values:

JsonSerializerOptions options = new() { RespectNullableAnnotations = true };
var result = JsonSerializer.Deserialize<MyPoco>("{}", options); // No exception!
Console.WriteLine(result.Name is null); // True

class MyPoco
{
    public string Name { get; set; }
}

This is because STJ treats required and non-nullable properties as orthogonal concepts. This stems from the C# language itself where you can have required properties that are nullable:

MyPoco poco = new() { Value = null }; // No compiler warnings

class MyPoco
{
    public required string? Value { get; set; }
}

And optional properties that are non-nullable:

class MyPoco
{
    public string Value { get; set; } = "default";
}

The same orthogonality applies to constructor parameters:

record MyPoco(
    string RequiredNonNullable,
    string? RequiredNullable,
    string OptionalNonNullable = "default",
    string? OptionalNullable = "default");

Respecting non-optional constructor parameters​

STJ constructor-based deserialization has historically been treating all constructor parameters as optional, as can be highlighted in the following example:

var result = JsonSerializer.Deserialize<Person>("{}");
Console.WriteLine(result); // Person { Name = , Age = 0 }

record Person(string Name, int Age);

In .NET 9 we’re including the RespectRequiredConstructorParameters flag that changes the behavior such that non-optional constructor parameters are now treated as required:

JsonSerializerOptions options = new() { RespectRequiredConstructorParameters = true };
string json = """{"Optional": "value"}""";
JsonSerializer.Deserialize<MyPoco>(json, options);

record MyPoco(string Required, string? Optional = null);
// JsonException: JSON deserialization for type 'MyPoco' 
// was missing required properties including: 'Required'.

Feature switch​

Users can turn on the RespectRequiredConstructorParameters setting globally using the System.Text.Json.Serialization.RespectRequiredConstructorParametersDefault feature switch, which can be set via your project configuration:

<ItemGroup>
  <RuntimeHostConfigurationOption Include="System.Text.Json.Serialization.RespectRequiredConstructorParametersDefault" Value="true" />
</ItemGroup>

Both the RespectNullableAnnotations and RespectRequiredConstructorParameter properties were implemented as opt-in flags to avoid breaking existing applications. If you’re writing a new application, it is highly recommended that you enable both flags in your code.

Customizing enum member names​

The new JsonStringEnumMemberName attribute can be used to customize the names of individual enum members for types that are serialized as strings:

JsonSerializer.Serialize(MyEnum.Value1 | MyEnum.Value2); // "Value1, Custom enum value"

[Flags, JsonConverter(typeof(JsonStringEnumConverter))]
enum MyEnum
{
    Value1 = 1,
    [JsonStringEnumMemberName("Custom enum value")]
    Value2 = 2,
}

Out-of-order metadata reads​

Certain features of System.Text.Json such as polymorphism or ReferenceHandler.Preserve require emitting metadata properties over the wire:

JsonSerializerOptions options = new() { ReferenceHandler = ReferenceHandler.Preserve };
Base value = new Derived("Name");
JsonSerializer.Serialize(value, options); // {"$id":"1","$type":"derived","Name":"Name"}

[JsonDerivedType(typeof(Derived), "derived")]
record Base;
record Derived(string Name) : Base;

By default, the STJ metadata reader requires that the metadata properties $id and $type must be defined at the start of the JSON object:

JsonSerializer.Deserialize<Base>("""{"Name":"Name","$type":"derived"}""");
// JsonException: The metadata property is either not supported by the
// type or is not the first property in the deserialized JSON object.

This is known to create problems when needing to deserialize JSON payloads that do not originate from System.Text.Json. The new AllowOutOfOrderMetadataProperties can be configured to disable this restriction:

JsonSerializerOptions options = new() { AllowOutOfOrderMetadataProperties = true };
JsonSerializer.Deserialize<Base>("""{"Name":"Name","$type":"derived"}""", options); // Success

Care should be taken when enabling this flag, as it might result in over-buffering (and OOM failures) when performing streaming deserialization of very large JSON objects. This is because metadata properties must be read before the deserialize object is instantiated, meaning that all properties preceding a $type property must be kept in the buffer for subsequent property binding.

Customizing indentation​

The JsonWriterOptions and JsonSerializerOptions types now expose APIs for configuring indentation. The following example enables single-tab indentation:

JsonSerializerOptions options = new()
{
    WriteIndented = true,
    IndentCharacter = '\t',
    IndentSize = 1,
};

JsonSerializer.Serialize(new { Value = 42 }, options);

JsonObject property order manipulation​

The JsonObject type is part of the mutable DOM and is used to represent JSON objects. Even though the type is modelled as an IDictionary<string, JsonNode>, it does encapsulate an implicit property order that is not user-modifiable. The new release exposes additional APIs that effectively model the type as an ordered dictionary:

public partial class JsonObject : IList<KeyValuePair<string, JsonNode?>>
{
    public int IndexOf(string key);
    public void Insert(int index, string key, JsonNode? value);
    public void RemoveAt(int index);
}

This allows modifications to object instances that can directly influence property order:

// Adds or moves the $id property to the start of the object
var schema = (JsonObject)JsonSerializerOptions.Default.GetJsonSchemaAsNode(typeof(MyPoco));
switch (schema.IndexOf("$id", out JsonNode? idValue))
{
    case < 0: // $id property missing
       idValue = (JsonNode)"https://example.com/schema";
       schema.Insert(0, "$id", idValue);
       break;

    case 0: // $id property already at the start of the object
        break; 

    case int index: // $id exists but not at the start of the object
        schema.RemoveAt(index);
        schema.Insert(0, "$id", idValue);
}

DeepEquals methods in JsonElement and JsonNode​

The new JsonElement.DeepEquals method extends deep equality comparison to JsonElement instances, complementing the pre-existing JsonNode.DeepEquals method. Additionally, both methods include refinements in their implementation, such as handling of equivalent JSON numeric representations:

JsonElement left = JsonDocument.Parse("10e-3").RootElement;
JsonElement right = JsonDocument.Parse("0.01").RootElement;
JsonElement.DeepEquals(left, right); // True

JsonSerializerOptions.Web​

The new JsonSerializerOptions.Web singleton can be used to quickly serialize values using JsonSerializerDefaults.Web settings:

JsonSerializerOptions options = JsonSerializerOptions.Web; // used instead of new(JsonSerializerDefaults.Web);
JsonSerializer.Serialize(new { Value = 42 }, options); // {"value":42}

Performance improvements​

For a detailed write-up of System.Text.Json performance improvements in .NET 9, please refer to the relevant section in Stephen Toub’s “Performance Improvements in .NET 9” article.

Closing​

.NET 9 sees a large number of new features and quality of life improvements with a focus on JSON schema and intelligent application support. In total 46 pull requests contributed to System.Text.Json during .NET 9 development. We’d like you to try the new features and give us feedback on how it improves your applications, and any usability issues or bugs that you might encounter.

Community contributions are always welcome. If you’d like to contribute to System.Text.Json, check out our list of help wanted issues on GitHub.

The post What’s new in System.Text.Json in .NET 9 appeared first on .NET Blog.

Continue reading...