Products & Apps

inclusionAI releases Humming, a high-performance quantized inference GEMM kernel library

Original title:inclusionAI/humming

AI summary

inclusionAI has open-sourced Humming, a high-performance, lightweight just-in-time compiled GEMM kernel library designed for quantized inference. It supports inference with arbitrary weight types below 8 bits under various activation data types such as FP16, BF16, and FP8, is compatible with multiple quantization strategies and scaling types, and supports both dense GEMM and mixture-of-experts GEMM operations. The library is compatible with all NVIDIA GPUs from SM75 and above, delivering industry-leading throughput and efficiency in various computing scenarios. It has minimal dependencies, requiring only PyTorch and NVCC, and the package size is only about 100 KB, facilitating ultra-lightweight deployment.

Key Takeaways

  • inclusionAI has open-sourced Humming,a high-performance,lightweight just-in-time compiled GEMM…
  • It supports inference with arbitrary weight types below 8 bits under various activation data types…
  • The library is compatible with all NVIDIA GPUs from SM75 and above,delivering industry-leading…
  • It has minimal dependencies,requiring only PyTorch and NVCC,and the package size is only about 100…

Quick Facts

Published
Updated
Source
蚂蚁 inclusionAI:GitHub 新仓库
Category
Products & Apps
AI score
61
Read the original Read the full article at the original source
Why it matters

蚂蚁 inclusionAI 开源了一个 100KB 级的量化 GEMM 库,支持从 INT1 到 FP8 全家桶,SM75+ 全覆盖,做推理部署的工程师值得花半小时跑一下 benchmark,看看能不能替换掉现有的 Marlin 方案。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: inclusionAI releases Humming, a high-performance quantized inference GEMM kernel library. February 11, 2026. https://new.nat6.net/en/news/cmorb7ik3006oslhfmf6g1irj/

View all →