1.58-bit large language model: Difference between revisions

Revision as of 17:16, 7 July 2025

Revision as of 20:47, 9 July 2025

Line 5:

== BitNet ==

{{redirect|BitNet|a computer network|BITNET}}

In 2024, Ma et al., researchers at [[Microsoft]], declared that their 1.58-bit model, '''''BitNet''' b1.58'' is comparable in performance to the 16-bit [[Llama 2]] and opens the era of 1-bit LLM.{{sfn|Huyen|2024|p=330}} BitNet creators did not use the post-training quantization of weights but instead relied on the new ''BitLinear'' transform that replaced the ''nn.Linear'' layer of the traditional transformer design.{{sfn|Wang|Ma|Dong|Huang|2023|p=1}}