Revision as of 20:18, 29 April 2025 edit Headbomb (talk \| contribs) Edit filter managers, Autopatrolled, Extended confirmed users, Page movers, File movers, New page reviewers, Pending changes reviewers, Rollbackers, Template editors 472,907 edits m →Sources: clean up Tag: AWB ← Previous edit		Revision as of 20:19, 29 April 2025 edit undo Citation bot (talk \| contribs) Bots 5,861,762 edits Alter: template type, url. URLs might have been anonymized. Add: date, title, class, eprint, authors 1-10. Removed parameters. Some additions/deletions were parameter name changes. \| Use this bot. Report bugs. \| Suggested by Headbomb \| #UCB_toolbar Next edit →
Line 15: ==Sources== * {{cite arXiv \|~~last~~last1=Ma \|~~first~~first1=Shuming \|last2=Wang \|first2=Hongyu \|last3=Ma \|first3=Lingxiao \|last4=Wang \|first4=Lei \|last5=Wang \|first5=Wenhui \|last6=Huang \|first6=Shaohan \|last7=Dong \|first7=Li \|last8=Wang \|first8=Ruiping \|last9=Xue \|first9=Jilong \|last10=Wei \|first10=Furu \|title=The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits \|~~arxiv~~eprint=2402.17764 \|date=2024-02-27 \|class=cs.CL }} * {{cite arXiv \|eprint=2504.12285 \|last1=Ma \|first1=Shuming \|last2=Wang \|first2=Hongyu \|last3=Huang \|first3=Shaohan \|last4=Zhang \|first4=Xingxing \|last5=Hu \|first5=Ying \|last6=Song \|first6=Ting \|last7=Xia \|first7=Yan \|last8=Wei \|first8=Furu \|title=BitNet b1.58 2B4T Technical Report \|date=2025 \|class=cs.CL }} * {{cite arxiv \|arxiv=2504.12285}} * {{cite journal \|~~last~~last1=Friha \|~~first~~first1=Othmane \|last2=Amine Ferrag \|first2=Mohamed \|last3=Kantarci \|first3=Burak \|last4=Cakmak \|first4=Burak \|last5=Ozgun \|first5=Arda \|last6=Ghoualmi-Zine \|first6=Nassira \|title=LLM-Based Edge Intelligence: A Comprehensive Survey on Architectures, Applications, Security and Trustworthiness \|journal=IEEE Open Journal of the Communications Society \|volume=5 \|date=2024 \|issn=2644-125X \|doi=10.1109/OJCOMS.2024.3456549 \|doi-access=free \|pages=5799–5856}} * {{cite journal \|title=1-bit LLMs Could Solve AI's Energy Demands \|journal=IEEE Spectrum \|date=2024-05-30 \|url=https://spectrum.ieee.org/1-bit-llm \|first=Matthew\|last=Hutson\|access-date=2025-04-22}} * {{cite book \|last=Huyen \|first=Chip \|title=AI Engineering \|publisher="O'Reilly Media, Inc." \|date=2024-12-04 \|isbn=978-1-0981-6627-4 \|url=https://~~www~~books.google.com/books~~/edition/AI_Engineering/S7M1EQAAQBAJ~~?hlid=~~en&gbpv=1~~S7M1EQAAQBAJ&pg=PA330 \|access-date=2025-04-22}} * {{cite arXiv \|eprint=2411.04330 \|last1=Kumar \|first1=Tanishq \|last2=Ankner \|first2=Zachary \|last3=Spector \|first3=Benjamin F. \|last4=Bordelon \|first4=Blake \|last5=Muennighoff \|first5=Niklas \|last6=Paul \|first6=Mansheej \|last7=Pehlevan \|first7=Cengiz \|last8=Ré \|first8=Christopher \|last9=Raghunathan \|first9=Aditi \|title=Scaling Laws for Precision \|date=2024 \|class=cs.LG }} * {{cite arxiv \|arxiv=2411.04330}} * {{cite web \|last=Morales \|first=Jowi \|title=Microsoft researchers build 1-bit AI LLM with 2B parameters \|website=Tom's Hardware \|date=2025-04-17 \|url=https://www.tomshardware.com/tech-industry/artificial-intelligence/microsoft-researchers-build-1-bit-ai-llm-with-2b-parameters-model-small-enough-to-run-on-some-cpus \|access-date=2025-04-21}} * {{cite arXiv \|eprint=2411.17691 \|last1=Ouyang \|first1=Xu \|last2=Ge \|first2=Tao \|last3=Hartvigsen \|first3=Thomas \|last4=Zhang \|first4=Zhisong \|last5=Mi \|first5=Haitao \|last6=Yu \|first6=Dong \|title=Low-Bit Quantization Favors Undertrained LLMS: Scaling Laws for Quantized LLMS with 100T Training Tokens \|date=2024 \|class=cs.LG }} * {{cite arxiv \|arxiv=2411.17691}} * {{cite arXiv \|eprint=2310.11453 \|last1=Wang \|first1=Hongyu \|last2=Ma \|first2=Shuming \|last3=Dong \|first3=Li \|last4=Huang \|first4=Shaohan \|last5=Wang \|first5=Huaijie \|last6=Ma \|first6=Lingxiao \|last7=Yang \|first7=Fan \|last8=Wang \|first8=Ruiping \|last9=Wu \|first9=Yi \|last10=Wei \|first10=Furu \|title=BitNet: Scaling 1-bit Transformers for Large Language Models \|date=2023 \|class=cs.CL }} * {{cite arxiv \|arxiv=2310.11453}} [[Category:Large language models]]

1.58-bit large language model: Difference between revisions