GHSA-qjvr-435c-5fjh Nerdbank.MessagePack has a memory amplification DoS in collection deserialization

Nerdbank.MessagePack deserializers for many collection-shaped types trusted the element count declared in MessagePack array and map headers when allocating destination storage. A crafted payload could therefore force large arrays, pooled buffers, dictionaries, or collection instances to be allocated before the deserializer had consumed the corresponding elements.

The same allocation pattern existed across strongly typed arrays, primitive arrays, mutable and immutable dictionaries, mutable enumerables, span-backed enumerable construction, JsonNode, MessagePackValue, and the object/dynamic primitive converters.

Because MessagePack array and map headers carry an attacker-controlled element count, any converter that immediately allocates count elements or constructs a collection with capacity count can turn a payload that is merely large into a much larger managed heap allocation. The reader's residency checks reduce the most extreme header-only attack shape, but they do not remove the memory amplification: minimal MessagePack elements can be one or two bytes on the wire while the managed representation may require object references, dictionary buckets, entries, array headers, or over-allocated collection internals.

Vulnerability Pattern

Affected converters followed one or both of these patterns:

int count = reader.ReadArrayHeader();
TElement[] array = new TElement[count];

int count = reader.ReadMapHeader();
Dictionary<TKey, TValue> map = new(count);

or, for streaming and span-backed construction:

TElement[] elements = ArrayPool<TElement>.Shared.Rent(count);
TCollection collection = getCollection(state, count);

In all affected cases, the allocation size was derived from the untrusted header count before the converter had read the elements. This made deserialization vulnerable to memory amplification and process availability attacks.

Affected Scope

The vulnerable logic was present in multiple converter families:

This means the issue affects normal typed deserialization as well as object/dynamic APIs. Any endpoint or protocol surface that accepts untrusted MessagePack and deserializes collection-shaped contracts can be affected.

Attack Mechanics

MessagePack encodes array and map lengths up front. For array32 and map32, the declared count can be very large. The reader checks that enough bytes remain to plausibly contain the declared number of elements, so an attacker must provide real payload bytes. However, the managed allocation can still be much larger than the payload.

Examples:

At larger counts, the attack can trigger OutOfMemoryException, force full blocking garbage collections, or degrade service through repeated moderate allocations. The impact is availability loss rather than confidentiality or integrity compromise.

Impact

An attacker who can deliver crafted MessagePack data to an endpoint that deserializes collections can:

• Crash the target process via OutOfMemoryException from a single large payload.
• Exhaust available memory through repeated moderate payloads.
• Induce severe GC pressure, increasing latency and reducing throughput.
• Affect typed DTO and framework integration paths, not only dynamic or untyped deserialization paths.

Concrete affected configurations include:

• ASP.NET Core, SignalR, RPC, queue, or storage endpoints that deserialize MessagePack request bodies or messages into DTOs with arrays, lists, sets, dictionaries, immutable dictionaries, JsonNode, MessagePackValue, object, or dynamic members.
• Code calling typed Deserialize<T>() where T contains collection-shaped members.
• Code calling DeserializePrimitives() or DeserializeDynamicPrimitives() on untrusted input.
• Applications registering WithObjectConverter() or WithDynamicObjectConverter() for framework integration or ad hoc object graphs.

Severity Rationale

Attack complexity is low. Once an application accepts untrusted MessagePack for an affected collection type, exploitation only requires a crafted array or map payload with a large declared count and minimal encoded elements.

Privileges required depend on deployment. Public endpoints are exploitable without authentication. Internal or authenticated message-processing systems reduce exposure but remain vulnerable to any caller who can submit MessagePack data.

User interaction is not required. The attack is triggered during server-side or service-side deserialization.

Availability impact is low. The practical outcome is memory pressure which can slow down a process or machine, or cause a network request to fail.

Proof of Concept

The following sample demonstrates the original object/dynamic shape and a typed array shape. Both rely on the same underlying bug: allocation is derived from the MessagePack header count before the elements are consumed.

using System.Buffers;
using System.Buffers.Binary;
using Nerdbank.MessagePack;

Console.WriteLine("=== Memory Amplification DoS - Nerdbank.MessagePack ===");
Console.WriteLine();

var serializer = new MessagePackSerializer();

// 5-byte array32 header + 1 byte per nil element.
// Deserializing as object?[] allocates one managed reference per element.
const int ObjectArrayCount = 1_000_000;
byte[] objectArrayPayload = BuildArray32Payload(ObjectArrayCount, 0xC0);

Measure("object?[] array32 nil", objectArrayPayload, () =>
{
    var sequence = new ReadOnlySequence<byte>(objectArrayPayload);
    var reader = new MessagePackReader(sequence);
    return serializer.DeserializePrimitives(ref reader);
});

// 5-byte array32 header + 1 byte per integer element.
// A typed int[] target allocates four bytes per element plus array overhead.
const int IntArrayCount = 1_000_000;
byte[] intArrayPayload = BuildArray32Payload(IntArrayCount, 0x00);

Measure("int[] array32 fixint", intArrayPayload, () =>
{
    var sequence = new ReadOnlySequence<byte>(intArrayPayload);
    var reader = new MessagePackReader(sequence);