Run GGUF models directly with transformers. This work brings ggml's Metal kernels to the transformers ecosystem, increasing compatibility and performance. More info below

Transformers has supported loading GGUF files for a few years now, by unquantizing them. Thanks to @_marcsun, we're now using GGML kernels through the `kernels` library to run at the same performance as llama.cpp Huge kudos to the entire @ggml_org for making these kernels!

Millions of GGUF downloads later, those same llama.cpp checkpoints can now run in 🤗 transformers. Same models, more ways to use them, and fast local inference on Mac powered by ggml kernels! Blog: huggingface.co/blog/transfor… ggml kernels: huggingface.co/ggml-org/kern…

Super happy to welcome @jundotkim, oMLX creator, to Hugging Face! 🥳 We are invested in Local AI and support MLX since @awnihannun and @angeloskath released it in Christmas 2023. oMLX gets stability and speed. It remains a community project led by Jun. huggingface.co/blog/omlx

The wait is finally over 🚀 !! Today we're finally shipping Halo A new training framework built to make training simpler, faster, and more reliable. Go try it at: github.com/whitecircle/halo ∇

Tokenizers v1's first release candidate is out 🔥 absolutely huge release across the board: Up to 30x faster tokenization, rivaling the best in the ecosystem. Scaling across threads, much better latency, less memory consumed, and microscopic crate size. huggingface.co/blog/tokenize…

Happy to officially bring you the new SOTA `tokenization` library. We focused on all languages, multi-thread scaling, minimal package size and memory usage. huggingface-tokenizers-v1.st… We're thankful for the players of this ecosystem that have pushed us to give the best we could!

Exciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university, industry and country to build with, customize and benefit from AI. Thank you @ClementDelangue for coming to me. NVIDIA is going to be a great home for Hugging Face, its community and the future of open models. 🤗 blogs.nvidia.com/blog/nvidia…

everyone is talking about open-weight models. at @huggingface we're invested in open source AI. like, a lot. and more every year. i pulled the public github numbers for our orgs. good excuse to introduce you to some of our hugging faces 🧵 (maybe you should follow them?)

I have HUGE news about the Transformers modelling backend for @vllm_project v0.25.0 🚀 It has reached performance parity with native vLLM model implementations 🤯 The Transformers modelling backend has just become a zero-effort, zero-compromise way to deploy to vLLM!

Over 7 days we've had Fable locked and an MIT opus-lvl model is out: @Zai_org GLM 5.2 I've been switching to open models everywhere: chat (HuggingChat), code (Pi) etc I learned valuable lessons on them and where they shine. If you work with them on your own tools, read this 🫵

This is not a day for celebrating, Andrej. It's a very dark and very sad day, and the damage may be impossible to undo.

Gemma-4 QAT just dropped! We found if you naively convert from QAT Q4_0 BF16, you will lose accuracy since the conversion to llama.cpp has a different lattice. Unsloth dynamic GGUFs recovers most of it! 26B-A4B: 85.6% top-1 % from 70.2% (+15.4%) 31B: 96.7% from 87.9% (+8.8%)

Introducing Gemma 4 QAT 🤏 - Quantization aware training to reduce models' precision while preserving quality - Introducing a new mobile quantization format that reduces memory footprint of E2B to 1GB - Q4 for all your favorite libraries ✨

We just dropped Gemma 4 Quantization-Aware Training (QAT) checkpoints on Hugging Face! All Gemma 4 model sizes and their drafters are now optimized with QAT to cut memory requirements and maximize on-device performance!

Meet Gemma 4 12B! A unified, encoder-free multimodal model designed to bring high-performance intelligence directly to your laptop, and released under an Apache 2.0 license. Bridging the gap between edge efficiency and advanced reasoning. Here is what’s new with Gemma 4 12B: 👇

🚀 open sourced metalBLAS, hand-tuned Metal matmul kernels for Apple Silicon, callable from PyTorch on mps. Matches/beats MPS Graph (torch) matmuls on bf16/fp16, 2-3x faster on fp32 (TF32-relaxed) across the bench suite on M5 Pro. Next step is to upstream this to PyTorch! github.com/Isalia20/metalBLA…

multi-turn RL and the "tito" problem keeps coming up. we've been working on it for a while, and the takeaway is that it's much easier than people are making it. it takes 1 implementation rule, and 1 chat-template property that all models already comply with. **that's all you need to do it right** qgallouedec-tito.hf.space

This is what we have been working on for the last 6 months or so at the AI Snowflake Research: Zero Redundancy Rollouts (ZoRRo): snowflake.com/en/blog/engine… If you do RL and you want it to be much faster make sure to have a look.