A4: Microarchitecture-Aware LLC Management for Datacenter Servers with Emerging I/O Devices
Emerging I/O devices inject data straight into the shared LLC via direct cache access, colliding with CPU workloads — and with each other — inside structures software cannot see.
High-speed I/O devices and CPU cores contend for the last-level cache in ways rooted in the microarchitecture — not the ISA. A4 identifies these hidden interference channels and manages LLC allocation around them, improving high-priority workload performance by 51% without notably compromising low-priority workloads.
Why it matters
In modern server CPUs, the last-level cache serves not only as a victim cache for the private caches but also as a buffer for low-latency DMA transfers between cores and I/O devices through direct cache access (DCA). Prior work showed that high-bandwidth network I/O can flood the LLC with packets and contend with co-running workloads — but the story runs deeper than that.
Key findings
- Exploring hidden microarchitectural properties of Intel Xeon CPUs, we uncover two previously unrecognized LLC contentions triggered by emerging high-bandwidth I/O devices.
- Directory contention: DMA-written cache lines in DCA ways silently migrate to two hidden inclusive ways — an artifact of the non-inclusive cache directory — when CPU cores read them, evicting the data of workloads allocated there.
- Storage-vs-network contention: high-bandwidth NVMe storage gains almost nothing from DCA, yet its blocks evict latency-sensitive network packets from DCA ways, inflating network tail latency.
- A4 is a runtime framework built on CAT, performance counters, and a little-known per-PCIe-port knob that selectively disables DCA. It improves high-priority workload performance by 51% while preserving low-priority throughput.
BibTeX
@inproceedings{park2025a4,
author = {Haneul Park and Jiaqi Lou and Sangjin Lee and Yifan Yuan
and Kyoung Soo Park and Yongseok Son and Ipoom Jeong and Nam Sung Kim},
title = {{A4}: Microarchitecture-Aware {LLC} Management for Datacenter
Servers with Emerging {I/O} Devices},
booktitle = {Proceedings of the 52nd Annual International Symposium on
Computer Architecture (ISCA)},
year = {2025}
}