electron-rare/prima-cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add q2_k fast and modify q2_k logic, add preliminary support for rms_norm_aware_importance but currently assumption is wrong and will lead to corrupted result

Browse Source
This commit is contained in:
DandinPower
2025-12-24 21:16:11 +08:00
parent eb7675f923
commit b35e13d4f2
5 changed files with 167 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/arg.cpp
+1 -1
View File
@@ -2120,7 +2120,7 @@ gpt_params_context gpt_params_parser_init(gpt_params & params, llama_example ex,
));
add_opt(llama_arg(
{"--comm-datatype"}, "TYPE",
format("Datatype for communication, currently support f32, bf16, fp16, q8_0, q4_0, q2_k, iq2_s, iq2_xs, iq2_xxs, fp8, fp4, mxfp8, mxfp4, nvfp4, nf4, nf4_dq, or f32_sparsity (default: %s)", params.comm_datatype.c_str()),
format("Datatype for communication, currently support f32, bf16, fp16, q8_0, q4_0, q2_k, q2_k_fast, iq2_s, iq2_xs, iq2_xxs, fp8, fp4, mxfp8, mxfp4, nvfp4, nf4, nf4_dq, or f32_sparsity (default: %s)", params.comm_datatype.c_str()),
[](gpt_params & params, const std::string & value) {
params.comm_datatype = value;
}