Linux Module Addresses PCIe Resizable BAR Limitations for GPUs
Article Content
- •The gpu-rebar-layout module enables full 32 GB Resizable BARs for GPUs on Linux.
- •Issues arose when multiple GPUs were installed, causing POST failures due to address space limitations.
- •The module modifies PCIe resources at boot, allowing for peer-to-peer communication between GPUs.
A new Linux kernel module, gpu-rebar-layout, has been developed to enable full 32 GB Resizable BARs for GPUs when BIOS settings are insufficient. The module is designed for systems with multiple GPUs, specifically four AMD Radeon AI PRO R9700s and two RTX 3090s, which faced issues during POST due to address space limitations. The module modifies PCIe resources at boot to ensure that the GPUs can utilize their full memory capacity, which is for enabling peer-to-peer communication between the R9700s. The initial implementation was tested on a specific machine configuration and relies on kernel internals that may change in future releases. Users are advised to read the README before loading the module due to potential compatibility issues. The situation highlights the ongoing challenges with PCIe resource allocation in Linux, particularly for high-performance computing setups.
Ask AI about this cluster
Answers cite the sources they use