inclusionAI releases Humming, a high-performance quantized inference GEMM kernel library
Original title:inclusionAI/humming
AI summary
inclusionAI has open-sourced Humming, a high-performance, lightweight just-in-time compiled GEMM kernel library designed for quantized inference. It supports inference with arbitrary weight types below 8 bits under various activation data types such as FP16, BF16, and FP8, is compatible with multiple quantization strategies and scaling types, and supports both dense GEMM and mixture-of-experts GEMM operations. The library is compatible with all NVIDIA GPUs from SM75 and above, delivering industry-leading throughput and efficiency in various computing scenarios. It has minimal dependencies, requiring only PyTorch and NVCC, and the package size is only about 100 KB, facilitating ultra-lightweight deployment.
Key Takeaways
- inclusionAI has open-sourced Humming,a high-performance,lightweight just-in-time compiled GEMM…
- It supports inference with arbitrary weight types below 8 bits under various activation data types…
- The library is compatible with all NVIDIA GPUs from SM75 and above,delivering industry-leading…
- It has minimal dependencies,requiring only PyTorch and NVCC,and the package size is only about 100…
Quick Facts
- Published
- Updated
- Source
- 蚂蚁 inclusionAI:GitHub 新仓库
- Category
- Products & Apps
- AI score
- 61
蚂蚁 inclusionAI 开源了一个 100KB 级的量化 GEMM 库,支持从 INT1 到 FP8 全家桶,SM75+ 全覆盖,做推理部署的工程师值得花半小时跑一下 benchmark,看看能不能替换掉现有的 Marlin 方案。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: inclusionAI releases Humming, a high-performance quantized inference GEMM kernel library. February 11, 2026. https://new.nat6.net/en/news/cmorb7ik3006oslhfmf6g1irj/