LlamaFactory — Unified LLM Fine-Tuning Framework

AI Tools hiyouga/LlamaFactory 70,823 stars Updated 3 days ago

TL;DR: Unified fine-tuning framework supporting 100+ models. Training Llama, Qwen, or DeepSeek takes minutes with zero code. Production-ready, battle-tested, and free.

LlamaFactory is the most comprehensive fine-tuning framework for LLMs. It supports 100+ models including Qwen, Llama, Gemma, DeepSeek, and 95+ others. Fine-tune your own model in minutes with zero code.

What It Does

LlamaFactory handles the entire fine-tuning pipeline:

Data preparation — Convert your data to the right format
Training configuration — Set up LoRA, QLoRA, P-Tuning, and more
Training execution — Run on single GPU or distributed clusters
Evaluation — Test your model with built-in benchmarks
Export — Save as GGUF, safetensors, or export to Inference API

Key Features

100+ Models

Support for Qwen, Llama, Gemma, DeepSeek, Yi, Mistral, and 95+ others

Multiple Methods

LoRA, QLoRA, P-Tuning v2, Full Fine-tuning, and more

Zero Code

Configure everything via UI or simple YAML files

Multi-GPU

Distributed training with DeepSpeed, FSDP, and ZeRO

Evaluation

Built-in benchmarks: MMLU, GSM8K, HumanEval, and more

Export Formats

GGUF, safetensors, and export to HuggingFace Inference API

Quick Start

Train a model in 3 steps:

Prepare your data in JSONL format
Run llamafactory-cli train with your config
Export your model with llamafactory-cli export

That's it. No complex setup, no boilerplate code.

Supported Models

Most Popular

🦙 Llama 3 / 2 / 1.x
🐯 Qwen 2 / 1.x
💎 Gemma / Gemma 2
🧠 DeepSeek V2 / V3
🔥 Yi / Yi-1.5
⚡ Mistral / Mixtral

Use Cases

Domain adaptation — Fine-tune for your specific industry or use case
Style transfer — Match your brand voice or writing style
Task-specific models — Specialized models for coding, math, reasoning
Private models — Fine-tune open-source models with your data

Key Takeaways

Verdict

LlamaFactory is the best fine-tuning framework for most use cases. Zero code setup, 100+ model support, and battle-tested. If you need to fine-tune an LLM, start here.

Pros

100+ models supported
Zero code configuration
Multiple training methods
Multi-GPU distributed training
Built-in evaluation
Active development

Cons

Learning curve for advanced features
Resource-intensive training
Not a production inference solution

Final thoughts: If you need to fine-tune an LLM, this is the framework you want. It handles everything from data prep to evaluation with minimal setup.

Z.AI — GLM Models & Claude Code Support

Access GLM-5, GLM-4, and 30+ models. Free tier available.