Revision as of 18:44, 29 May 2025 edit Unis Tankos (talk \| contribs) Extended confirmed users 1,408 edits →Sources ← Previous edit		Revision as of 19:44, 29 May 2025 edit undo Alenoach (talk \| contribs) Extended confirmed users 5,805 edits removed red links, there is an article Open-source artificial intelligence but there probably won't be an article on these more specific topics Tag: Visual edit Next edit →
Line 7: In 2024, Ma et al., researchers at [[Microsoft]], declared that their 1.58-bit model, '''''BitNet''' b1.58'' is comparable in performance to the 16-bit [[Llama 2]] and opens the era of 1-bit LLM.{{sfn\|Huyen\|2024\|p=330}} BitNet creators did not use the post-training quantization of weights but instead relied on the new ''BitLinear'' transform that replaced the ''nn.Linear'' layer of the traditional transformer design.{{sfn\|Wang\|Ma\|Dong\|Huang\|2023\|p=1}} In 2025, Microsoft researchers had released an [[open-weights]] and [[open inference code]] model ''BitNet b1.58 2B4T'' demonstrating performance competitive to the full precision models at 2B parameters and 4T training tokens.{{sfn\|Ma\|Wang\|Huang\|Zhang\|2025\|p=}} == Critique ==

1.58-bit large language model: Difference between revisions