2 Commits

Author	SHA1	Message	Date
DandinPower	b35e13d4f2	feat: add q2_k fast and modify q2_k logic, add preliminary support for rms_norm_aware_importance but currently assumption is wrong and will lead to corrupted result	2025-12-24 21:16:11 +08:00
DandinPower	73969b44b1	feat: use bitsqueeze as communication compression library to replace the previous quantization/k_quantization/sparsity. Also add a new argument call comm_compression_threshold to allow user can set a threshold for compression (good for only compress prefilling tokens)	2025-12-06 21:08:10 +08:00