fix compute buffer estimate: reserve 300 MiB VRAM to avoid potential OOM

This commit is contained in:
Li, Zonghang
2025-06-26 20:45:45 +04:00
parent a05022c05a
commit aacfa8a231
+2 -2
View File
@@ -22043,8 +22043,8 @@ void llama_model_compute_buf_size(
// context GPU memory usage, i.e. the initial memory cost of creating a CUDA context,
// even before you launch any kernels or allocate your own buffers.
// this value may vary by GPU and CUDA version, but it's lower than 400 MiB in most cases,
// another 200 MiB is used to prevent accidental OOM.
*gpu_buf += 600 * 1024 * 1024;
// another 300 MiB is used to prevent accidental OOM.
*gpu_buf += 700 * 1024 * 1024;
}
}