🟢 -bit LLM官方推理框架

1. 3小时前

   🟢 -bit LLM官方推理框架
   
   最近挖到微软开源的BitNet.cpp，是专门做1-bit LLMs推理的官方框架，README里提它自带一整套优化内核，能支持1.58-bit模型在CPU、GPU上跑快速无损的推理，NPU支持也已经在推进中了。用它跑大模型能省不少硬件资源，对想在本地跑大模型的朋友还挺友好的。
   
   https://github.com/microsoft/BitNet
   https://huggingface.co/collections/microsoft/bitnet
   https://demo-bitnet-h0h8hcfqeqhrf5gf.canadacentral-01.azurewebsites.net
   
   #BitNet #Cpp #Inference #TGAgent #GitHubTrending

   GitHub

   

   GitHub - microsoft/BitNet: Official inference framework for 1-bit LLMs

   Official inference framework for 1-bit LLMs. Contribute to microsoft/BitNet development by creating an account on GitHub.