[Proposal] Add a "Low-Resource / HDD-Optimized" Compilation Mode with Explicit Memory and Thread Limits

[1person280]

背景 / Background

Rust 编译器在高性能硬件（NVMe SSD、32GB+ 内存、多核 CPU）上表现优异，但仍有大量开发者在资源受限环境中工作，尤其是使用机械硬盘（HDD）和低内存（如 4-8GB）的用户。当前 cargo 的默认配置倾向于最大化并行度和吞吐量，这在 HDD 和低内存机器上反而会导致严重的 I/O 争用和 OOM 问题。

Rust compiler performs excellently on high-end hardware (NVMe SSDs, 32GB+ RAM, multi-core CPUs), but many developers still work in constrained environments, especially those using mechanical hard drives (HDDs) and limited memory (e.g., 4-8GB). Current cargo defaults prioritize maximum parallelism and throughput, which can cause severe I/O contention and OOM issues on HDDs and low-memory machines.

提案内容 / Proposal

建议在 cargo 中新增一个可选的低资源编译模式（例如  --profile low-resource  或环境变量  CARGO_LOW_RESOURCE=1 ），该模式强制应用以下配置约束：

I propose adding an optional low-resource compilation mode to cargo (e.g.,  --profile low-resource  or environment variable  CARGO_LOW_RESOURCE=1 ) that enforces the following constraints:

1. 最大编译内存限制 / Maximum Compilation Memory Limit

配置项： max-compilation-memory （整数 GB，范围 1-1024）

建议默认值： 4 （即 4GB）

作用：限制编译器进程的内存使用上限，防止在低内存机器上触发 OOM，同时为 IDE（如 rust-analyzer）留出足够的内存空间。

Configuration:  max-compilation-memory  (integer GB, range 1-1024)

Suggested default:  4  (i.e., 4GB)

Purpose: Cap the compiler process memory usage to prevent OOM on low-memory machines, while leaving sufficient memory for IDEs (e.g., rust-analyzer).

2. 线程数限制 / Thread Count Limit

配置项： compiler-threads （整数，范围 1-16）

建议默认值： 2 

作用：限制编译器可申请的并发线程数。在机械硬盘上，过多的并发线程会导致磁头频繁寻道（thrashing），反而降低整体 I/O 效率。双线程模型可以在保持基本并行度的同时，最小化磁盘争用。

Configuration:  compiler-threads  (integer, range 1-16)

Suggested default:  2 

Purpose: Limit the number of concurrent threads the compiler can request. On mechanical hard drives, excessive concurrent threads cause frequent head seeking (thrashing), reducing overall I/O efficiency. A dual-thread model minimizes disk contention while maintaining basic parallelism.

3. 缓存块大小优化 / Cache Block Size Optimization

配置项： cache-block-size （固定 16MB）

作用：将增量编译缓存文件对齐到 16MB 的固定块大小，减少机械硬盘上的文件碎片化，将随机读取转化为更接近顺序读取的模式，从而提升 HDD 上的缓存命中率。

Configuration:  cache-block-size  (fixed at 16MB)

Purpose: Align incremental compilation cache files to a fixed 16MB block size, reducing file fragmentation on mechanical hard drives and converting random reads into more sequential-like access patterns, thereby improving cache hit rates on HDDs.

预期收益 / Expected Benefits

HDD 用户编译速度提升约 2x：通过减少磁盘 I/O 争用和文件碎片化。

低内存机器稳定性提升：4GB 内存限制可防止 OOM，让 4-8GB 内存的机器能够正常编译中小型项目。

可预测的编译时间：减少因磁盘争用导致的编译时间波动。

~2x speedup for HDD users: By reducing disk I/O contention and file fragmentation.

Improved stability on low-memory machines: The 4GB memory cap prevents OOM, enabling normal compilation of small-to-medium projects on 4-8GB RAM machines.

Predictable build times: Reduced variance caused by disk contention.

注意：此模式明确不适用于 SSD/高端用户，在这些环境下其性能可能低于默认的高并行模式。
Note: This mode is explicitly not intended for SSD/high-end users, where its performance may be lower than the default high-parallelism mode.

实施建议 / Implementation Suggestions

作为可选的 profile 或 flag 实现，不影响现有默认行为。

在  .cargo/config.toml  中提供对应的配置键，方便用户手动启用。

收集 HDD 和低内存环境下的 benchmark 数据，验证加速效果。

Implement as an optional profile or flag, without affecting existing default behavior.

Provide corresponding configuration keys in  .cargo/config.toml  for manual enablement.

Collect benchmark data in HDD and low-memory environments to validate the speedup.

开放问题 / Open Questions

编译器团队是否愿意接受一个针对特定硬件场景的"受限模式"？

现有的增量编译缓存系统是否支持固定块大小的对齐策略？

Would the compiler team be open to a "constrained mode" targeting specific hardware scenarios?

Does the existing incremental compilation cache system support fixed block size alignment?

[theemathas]

I recommend asking at https://internals.rust-lang.org/. More people will see it there.

[bjorn3]

You can already limit the amount of jobs that will run concurrently using -j. Limiting to specific amounts of memory is not possible outside of manually choosing a max amount of jobs that would ensure this memory limit given that cargo can't predict how much memory rustc will use for a given crate.

For cache-block-size, your filesystem should already be trying to prevent fragmentation by delaying writes a bit and then writing as a single chunk. I don't see how it would help with memory usage at all.

[1person280]

English Version

Thanks for the detailed explanation, @bjorn3. I'd like to share some deeper thoughts on these points:

1. On concurrency limits (-j):
   My concern is that even with -j 1, a large dependency graph still triggers a significant number of process creations and context switches. The overhead isn't just about parallelism; it's about the sheer volume of spawned processes and their baseline memory footprint.

2. On memory management architecture:
   Ideally, we could see a more modular integration between Cargo and rustc. Imagine a unified memory manager that operates on larger granules (e.g., 16MB). It could prioritize critical allocations and evict less important data when memory pressure is high, rather than relying on the OS to handle fragmentation blindly. This would give us deterministic control over memory growth.

3. On cache block size and I/O:
   Regarding the block size, while filesystems do try to coalesce writes, explicit control helps. Large projects often involve massive amounts of external references. If we stick to small blocks (like 64KB), we risk high fragmentation and excessive random I/O, which disks (especially HDDs) hate. Bumping this to something like 16MB would align better with modern storage behaviors, significantly reducing fragmentation and the memory overhead associated with managing scattered I/O buffers.

中文版本

感谢 @bjorn3 的详细解释，我想针对这几点分享一些更深层的看法：

1. 关于并发限制 (-j)：
   我的顾虑在于，即使设置了 -j 1，庞大的依赖图依然会触发大量的进程创建和上下文切换。开销不仅仅来自于并行计算，更来自于生成的进程数量及其基础的内存占用。

2. 关于内存管理架构：
   理想情况下，我们可以看到 Cargo 和 rustc 之间有更模块化的整合。想象一个统一的内存管理器，它以较大的粒度（例如 16MB）运作。当内存压力高时，它可以优先保障关键分配，并挤出次要数据，而不是盲目依赖操作系统来处理碎片。这将赋予我们对内存增长确定性的控制权。

3. 关于缓存块大小与 I/O：
   关于块大小，虽然文件系统确实会尝试合并写入，但显式的控制会更有帮助。大型项目通常涉及大量的外部引用。如果我们坚持使用小块（如 64KB），可能会导致严重的碎片化和过度的随机 I/O，而这是磁盘（尤其是机械硬盘）最忌讳的。将其提升到 16MB 左右将更好地契合现代存储的行为模式，从而显著减少碎片化以及管理分散 I/O 缓冲区所带来的内存开销。

[bjorn3]

My concern is that even with -j 1, a large dependency graph still triggers a significant number of process creations and context switches. The overhead isn't just about parallelism; it's about the sheer volume of spawned processes and their baseline memory footprint.

You get one process for cargo (unavoidable) + one process per build script (unavoidable) + one process per crate (avoidable only if cargo were to dynamically link rustc, which it can't as cargo needs to build on stable, and would break the RUSTC env var for cargo either way) + one process per linked crate (for the linker, unavoidable).

Ideally, we could see a more modular integration between Cargo and rustc. Imagine a unified memory manager that operates on larger granules (e.g., 16MB). It could prioritize critical allocations and evict less important data when memory pressure is high, rather than relying on the OS to handle fragmentation blindly. This would give us deterministic control over memory growth.

I don't get at all what you want.

Regarding the block size, while filesystems do try to coalesce writes, explicit control helps. Large projects often involve massive amounts of external references. If we stick to small blocks (like 64KB), we risk high fragmentation and excessive random I/O, which disks (especially HDDs) hate. Bumping this to something like 16MB would align better with modern storage behaviors, significantly reducing fragmentation and the memory overhead associated with managing scattered I/O buffers.

We don't use blocks when writing at all. Object files are written to the disk in one go I believe (unless LLVM streams them), rlibs are written one member at a time. Only the crate metadata and parts of the incr comp cache are encoded in smaller parts (seems like we buffer 64kb currently, increasing the size would increase memory usage).

[1person280]

English Version

I propose a fully open-source approach that integrates these functionalities directly as internal modules. This architecture offers distinct advantages in both performance and accessibility:

Performance via Direct Memory Access: Direct memory access is significantly faster than inter-process communication (IPC) via the OS, effectively bypassing system bottlenecks. For the linking phase specifically, passing an internal pointer to a shared 16MB cache block would completely eliminate the overhead of spawning multiple processes.

OS Optimization: A unified memory manager operating under a single PID and visible to the Task Manager allows the OS to better optimize memory layout and reduce fragmentation, rather than treating it as scattered external processes.

I/O Reality Check: I remain skeptical about the "single-pass write" assumption. Even if true, it ignores a critical reality: HDDs suffer severely from random reads, not just random writes. Sequential access patterns are vital for mechanical drives.

Crucial Context: Accessibility for Developing Regions
While reducing memory footprint is a valid goal, we must consider the hardware reality in developing countries. Many developers there are still constrained by mechanical hard drives (HDDs), 4GB of RAM, and dual-core processors. Optimizing strictly for modern SSDs and high-RAM environments inadvertently excludes this demographic. We should aim for a solution that remains viable on legacy hardware while scaling up for modern systems.

 

中文版本

我建议采用完全开源的方式，将这些功能直接作为内部模块进行整合。这种架构在性能和可访问性上都有显著优势：

通过直接内存访问提升性能： 直接访问内存比通过操作系统进行进程间通信（IPC）要快得多，能有效绕过系统瓶颈。特别是在连接阶段，通过在内部传递指向共享 16MB 缓存块的指针，可以完全消除生成多个进程的开销。

利于操作系统优化： 一个统一且处于单一 PID 下的内存管理器（对任务管理器可见），能让操作系统更好地优化内存布局并减少碎片，而不是将其视为分散的外部进程来处理。

I/O 的现实情况： 我对“一次性写入”的假设持保留态度。即使假设成立，它也忽略了一个关键事实：机械硬盘（HDD）不仅怕随机写入，更怕随机读取。顺序访问模式对机械硬盘至关重要。

关键背景：发展中地区的可访问性
虽然降低内存占用是一个合理的目标，但我们必须考虑到发展中国家的硬件现实。许多开发者仍受限于机械硬盘、4GB 内存和双核处理器。如果仅仅针对现代 SSD 和大内存环境进行优化，实际上会将这部分开发者排除在外。我们的目标应该是寻找一种既能适应现代系统，又能在老旧硬件上存活的解决方案。

[weihanglo]

FWIW, some of the ideas here may be better started in rust-lang/cargo repository. We have some already to be honest

• Add peak memory of each build unit during --timing and build analysis report cargo#17520
• Tracking Issue for integration with --jobs options in rustc cargo#17483
• Limiting the parallelism automatically cargo#12912
• Introduce 'nice' value under cargo.toml -> [build] cargo#9250
• Allow restricting the number of parallel linker invocations cargo#9157
• #t-cargo > cargo#9157: Restrict parallel linker executions

However, there are too many ideas in this issue. IMHO, a smaller scoped more focused issue on a targeted issue is more likely to make progress. Like rust-lang/cargo#17520 we are discussing the monitoring first so Cargo can be smarter about the resource usage.