CVE-2022-49394 blk-iolatency: Fix inflight count imbalances and IO hangs on offline

Medium 5.5 Published Updated

A race condition in the Linux kernel's blk-iolatency module can cause IO hangs and denial of service when cgroups are removed while IO is active.

Exploited in the wild
Not known
Login needed
Any signed-in user
Attackable
Local or same network only
Victim action needed
Not needed
Fix
Available

What is CVE-2022-49394?

CVE-2022-49394 is a medium-severity denial of service vulnerability in Linux kernel, published on 26 Feb 2025. It can only be exploited from the same machine or local network, with no action from the victim.

Official description

In the Linux kernel, the following vulnerability has been resolved:

blk-iolatency: Fix inflight count imbalances and IO hangs on offline

iolatency needs to track the number of inflight IOs per cgroup. As this tracking can be expensive, it is disabled when no cgroup has iolatency configured for the device. To ensure that the inflight counters stay balanced, iolatency_set_limit() freezes the request_queue while manipulating the enabled counter, which ensures that no IO is in flight and thus all counters are zero.

Unfortunately, iolatency_set_limit() isn't the only place where the enabled counter is manipulated. iolatency_pd_offline() can also dec the counter and trigger disabling. As this disabling happens without freezing the q, this can easily happen while some IOs are in flight and thus leak the counts.

This can be easily demonstrated by turning on iolatency on an one empty cgroup while IOs are in flight in other cgroups and then removing the cgroup. Note that iolatency shouldn't have been enabled elsewhere in the system to ensure that removing the cgroup disables iolatency for the whole device.

The following keeps flipping on and off iolatency on sda:

echo +io > /sys/fs/cgroup/cgroup.subtree_control while true; do mkdir -p /sys/fs/cgroup/test echo '8:0 target=100000' > /sys/fs/cgroup/test/io.latency sleep 1 rmdir /sys/fs/cgroup/test sleep 1 done

and there's concurrent fio generating direct rand reads:

fio --name test --filename=/dev/sda --direct=1 --rw=randread \ --runtime=600 --time_based --iodepth=256 --numjobs=4 --bs=4k

while monitoring with the following drgn script:

while True: for css in css_for_each_descendant_pre(prog['blkcg_root'].css.address_of_()): for pos in hlist_for_each(container_of(css, 'struct blkcg', 'css').blkg_list): blkg = container_of(pos, 'struct blkcg_gq', 'blkcg_node') pd = blkg.pd[prog['blkcg_policy_iolatency'].plid] if pd.value_() == 0: continue iolat = container_of(pd, 'struct iolatency_grp', 'pd') inflight = iolat.rq_wait.inflight.counter.value_() if inflight: print(f'inflight={inflight} {disk_name(blkg.q.disk).decode("utf-8")} ' f'{cgroup_path(css.cgroup).decode("utf-8")}') time.sleep(1)

The monitoring output looks like the following:

inflight=1 sda /user.slice inflight=1 sda /user.slice ... inflight=14 sda /user.slice inflight=13 sda /user.slice inflight=17 sda /user.slice inflight=15 sda /user.slice inflight=18 sda /user.slice inflight=17 sda /user.slice inflight=20 sda /user.slice inflight=19 sda /user.slice <- fio stopped, inflight stuck at 19 inflight=19 sda /user.slice inflight=19 sda /user.slice

If a cgroup with stuck inflight ends up getting throttled, the throttled IOs will never get issued as there's no completion event to wake it up leading to an indefinite hang.

This patch fixes the bug by unifying enable handling into a work item which is automatically kicked off from iolatency_set_min_lat_nsec() which is called from both iolatency_set_limit() and iolatency_pd_offline() paths. Punting to a work item is necessary as iolatency_pd_offline() is called under spinlocks while freezing a request_queue requires a sleepable context.

This also simplifies the code reducing LOC sans the comments and avoids the unnecessary freezes which were happening whenever a cgroup's latency target is newly set or cleared.

Which versions are affected by CVE-2022-49394?

It affects 1 product.

Products include Linux kernel.

How do I fix CVE-2022-49394?

Apply the vendor's patch.

Upgrade, then put limits on request size, rate and nesting depth so one client cannot exhaust the service. More about Denial of service.

Is CVE-2022-49394 being exploited?

There is no known exploitation of CVE-2022-49394 in the wild and no public exploit code.

All Linux kernel vulnerabilities

Sources

Also known as DEBIAN-CVE-2022-49394, GHSA-896c-jr9v-xc8g, UBUNTU-CVE-2022-49394.

Is CVE-2022-49394 in your software?

Upload an SBOM and sbomify checks it against every vulnerability here, every day.

Check your SBOMs for free