Firecracker is an open source virtualization technology that powers AWS Lambda and other serverless compute services. An issue in Firecracker v0.23.0 through v1.16.1 can, under certain host memory conditions, let a root user in a guest read or modify memory of the Firecracker process after a failed guest memory discard.
Firecracker v0.23.0 through v1.16.1 are affected when a VM is restored from a snapshot using the File memory backend and has a balloon or virtio-mem device. Under certain host memory conditions, a failed memory discard can leave part of the guest memory range unmapped while Firecracker keeps running, and memory that Firecracker later allocates there can become readable or writable by a root user in the guest. VMs started without a snapshot, or restored using the Uffd memory backend, are not affected.
Impacted versions: Firecracker v0.23.0 through v1.16.1, inclusive.
Within the currently supported release lines, v1.16.0 and v1.16.1 are affected. The fix is included in v1.16.2 and v1.17.0.
This issue is fixed in Firecracker v1.16.2 and v1.17.0. Firecracker now retains the original mapping’s MAP_NORESERVE flag when replacing discarded memory. If the replacement mapping fails, Firecracker terminates the process and its VM instead of continuing with an invalid guest memory mapping.
If you restore VMs using the File memory backend with a balloon or virtio-mem device, we recommend upgrading to Firecracker v1.16.2, v1.17.0 or later. Operators maintaining forks or derivative code should incorporate the corresponding fixes.
If upgrading is not immediately possible, restore snapshots using the Uffd memory backend, following its documented requirements for handling discarded pages.
The following host settings also avoid the known conditions:
Use heuristic memory overcommit (vm.overcommit_memory=0) rather than strict overcommit, and keep each VM's guest memory, including any virtio-mem region, no larger than the host's total RAM plus swap.
Keep the out-of-memory (OOM) killer enabled for Firecracker's memory cgroup.
These settings reduce but do not fully address the issue.
PR #6174 and its v1.16 backport PR #6178
If you have any questions or this advisory, we ask that you AWS Security via our vulnerability reporting page or directly via email to [email protected] . Please do not create a public GitHub issue.
We thank Juan Pablo Martinez Kuhn from Coinspect for collaborating on this issue through the coordinated vulnerability disclosure process.
The full story
This article is one source in a clustered incident — the cluster page carries the summary, timeline and every other outlet covering it.
