LLaMA-Factory

Part I: The mental model

llamafactory-cli train cfg.yaml   console entry -> src/llamafactory/cli.py
      |
      v
get_train_args(args)              hparams/parser.py -> five dataclasses
      |                           (ModelArguments, DataArguments, Seq2SeqTrainingArguments,
      |                            FinetuningArguments, GeneratingArguments)
      v
run_exp(...)                      train/tuner.py dispatches on finetuning_args.stage
      |                           stage in {pt, sft, rm, ppo, dpo, kto}
      v
tokenizer + template             model/loader.py, data/template.py
      |                           get_template_and_fix_tokenizer(...)
      v
get_dataset(...)                 data/loader.py: parse dataset_info.json ->
      |                          align (alpaca/sharegpt) -> template -> tokenize -> mask
      v
load_model + init_adapter         model/loader.py, model/adapter.py (full/freeze/lora/qlora)
      |
      v
CustomSeq2SeqTrainer.train()      train/sft/trainer.py (subclass of HF Trainer / TRL)
      |
      v
save adapter + logs               output_dir/  (adapter weights, trainer_state, loss plot)

The one-sentence identity. LLaMA-Factory is an orchestration layer that makes the three things which normally change from one fine-tune to the next, the chat template, the dataset format, and the training algorithm, into named, registered, swappable parts, so a run becomes a config rather than a script. Almost every serious open-source model can be fine-tuned with the same commands, because the parts that differ between Llama, Qwen, Gemma, Mistral, ChatGLM, and the rest are captured as a registered template and a model-loading patch rather than as forked training code. The training loop itself is Hugging Face's Trainer and TRL's RLHF trainers, lightly subclassed, so the project does not reinvent backpropagation, it composes existing pieces and removes the wiring you would otherwise write by hand.

Two consequences follow. First, the barrier to a first fine-tune is genuinely low. The repository ships example configs and demo datasets, auto-selects sensible defaults such as targeting every linear layer for LoRA, and offers a web UI for people who do not want to touch a terminal, so the distance from a base checkpoint to a trained adapter is one command. Second, because the CLI, the Python API, and the web UI all funnel into the same run_exp function, learning one path teaches all three. Everything in this chapter is described against the project as of mid 2026. LLaMA-Factory moves quickly and tracks new models and methods aggressively, so where an exact file path or field name is likely to have shifted I describe the component by its role and say so.

Part II: Using it

LLaMA-Factory installs from source, which is also the best way to read it. A recent Python and a PyTorch build matched to your CUDA are the only hard requirements, and the extras in the install pull in the training and metrics stack.

git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"

# sanity checks
llamafactory-cli version
llamafactory-cli env      # prints torch, transformers, trl, peft, and CUDA versions

The console entry point is llamafactory-cli, wired to llamafactory.cli:main in the package metadata. Its subcommands are the whole surface of the project. train runs a fine-tune, chat opens an interactive terminal chat with a trained model, export merges a LoRA adapter back into its base and optionally quantizes the result, api serves an OpenAI-compatible endpoint, eval runs benchmark evaluation, webui launches LlamaBoard, and webchat launches a single-model chat demo. The intended first run is a LoRA SFT of Llama 3 on the bundled demo data, driven by one of the ready-made configs under examples/.

# one command, one config file
llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml

That YAML is worth reading in full, because every field maps directly to a dataclass attribute you will meet in Part IV. It names the base model, selects the algorithm and the tuning method, points at registered datasets and a chat template, and sets the ordinary Hugging Face training knobs.

### model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct

### method
stage: sft                 # pt | sft | rm | ppo | dpo | kto
do_train: true
finetuning_type: lora      # full | freeze | lora
lora_target: all           # attach LoRA to every linear module

### dataset
dataset: identity,alpaca_en_demo   # names resolved via data/dataset_info.json
template: llama3                    # chat format registered in data/template.py
cutoff_len: 2048
max_samples: 1000
overwrite_cache: true

### output
output_dir: saves/llama3-8b/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true

When the run finishes, output_dir holds the LoRA adapter (adapter_model.safetensors and adapter_config.json), the tokenizer, a trainer_state.json with the full log history, and, if plot_loss is on, a loss-curve image. You can talk to the result immediately by pointing an inference config at the base model plus the adapter, or merge and export a standalone model.

# chat with the freshly trained adapter
llamafactory-cli chat examples/inference/llama3_lora_sft.yaml

# merge the adapter into the base weights and write a standalone model
llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml

# launch the LlamaBoard web UI (Gradio) for a point-and-click run
llamafactory-cli webui

Everything the YAML expresses can also be passed as flags, which is what the web UI generates under the hood, and every flag maps to a dataclass field.

llamafactory-cli train \
  --stage sft --do_train \
  --model_name_or_path meta-llama/Meta-Llama-3-8B-Instruct \
  --dataset alpaca_en_demo --template llama3 \
  --finetuning_type lora --lora_target all \
  --output_dir saves/llama3-lora-sft \
  --per_device_train_batch_size 1 --gradient_accumulation_steps 8 \
  --learning_rate 1e-4 --num_train_epochs 3 --bf16

For multiple GPUs the launcher handles the plumbing. Setting FORCE_TORCHRUN=1 makes the CLI re-exec itself under torchrun, and DeepSpeed ZeRO is a matter of adding one config, for example --deepspeed examples/deepspeed/ds_z3_config.json. The multi-GPU mechanics live under Transformers, Accelerate, and DeepSpeed rather than in LLaMA-Factory itself, and the parallel computing notes cover what those degrees mean.

There is also a small Python API, useful for wrapping a fine-tune in a larger script. The training entry point takes an optional dict of the same arguments.

from llamafactory.train.tuner import run_exp

run_exp(dict(
    stage="sft",
    do_train=True,
    model_name_or_path="meta-llama/Meta-Llama-3-8B-Instruct",
    dataset="alpaca_en_demo",
    template="llama3",
    finetuning_type="lora",
    lora_target="all",
    output_dir="saves/llama3-lora-sft",
    per_device_train_batch_size=1,
    gradient_accumulation_steps=8,
    learning_rate=1e-4,
    num_train_epochs=3.0,
    bf16=True,
))

# inference from Python via the same registries
from llamafactory.chat import ChatModel
chat = ChatModel(dict(
    model_name_or_path="meta-llama/Meta-Llama-3-8B-Instruct",
    adapter_name_or_path="saves/llama3-lora-sft",
    template="llama3",
    finetuning_type="lora",
))
print(chat.chat([{"role": "user", "content": "Who are you?"}])[0].response_text)

Now the mistakes newcomers make. The first and most common is a template that does not match the model. The template field selects the exact chat format, special tokens, and stop words the base model was trained with, and using llama3 on a Qwen checkpoint, or the base template on an instruct model, produces fluent-looking but subtly wrong output and an EOS that never fires. Match the template to the model family. The second is confusing the two orthogonal choices. stage picks the algorithm (SFT, DPO, and so on) while finetuning_type picks how much of the model moves (full, freeze, or LoRA), and they combine freely. The third is using an unregistered dataset. A dataset name has to resolve through data/dataset_info.json, so custom data means adding one JSON entry that declares its format and column mapping, covered in Part V. The fourth is asking for a preference method on plain instruction data. DPO, KTO, and reward modeling need a dataset that carries preference signal, chosen and rejected responses for DPO and RM or a binary label for KTO, and pointing them at an SFT dataset fails at data-loading time rather than training a bad model quietly.

Part III: When it is the right tool

LLaMA-Factory is the right tool when you want to fine-tune a known open model on your own data with a standard recipe, quickly, without writing a training loop. That covers a very large fraction of practical work. Teams adapting an instruct model to a domain with LoRA or QLoRA SFT, researchers running DPO or KTO alignment studies over many base models with one consistent harness, and newcomers who want a first fine-tune to succeed on the first try are all squarely in its sweet spot. The breadth is the point. One interface reaches a hundred-plus architectures and six training stages, and the web UI lets a non-programmer drive the same machinery.

The honest cases for alternatives. Axolotl is the closest peer, another YAML-driven fine-tuning framework over the same Hugging Face stack, and the choice between them is mostly ecosystem and taste. LLaMA-Factory leans on its built-in template and dataset registries and its web UI, while Axolotl leans on a large library of community configs. Unsloth rewrites the hot paths with custom Triton kernels for the fastest, lowest-memory single-GPU LoRA, at the cost of narrower model coverage, and LLaMA-Factory can call it as an optional backend rather than competing with it. If you want full control and minimal indirection, use TRL directly, since it is the very library LLaMA-Factory wraps for its RLHF trainers, and you trade convenience for a training script you own line by line. torchtune is the PyTorch-native, recipe-as-code option for people who prefer explicit Python over declarative YAML. ms-swift from ModelScope covers comparable breadth with tighter ties to that ecosystem. And for large-scale distributed RLHF where PPO throughput on many nodes is the whole game, OpenRLHF and verl are built for that regime in a way LLaMA-Factory's convenient single-node PPO is not.

Two scope boundaries are worth stating plainly. LLaMA-Factory fine-tunes existing checkpoints, it does not pretrain from scratch at cluster scale. That job belongs to a platform like torchtitan, which composes N-dimensional parallelism over a plain model definition. And LLaMA-Factory is a serving-adjacent tool, not a serving engine. Its api command and vLLM inference backend are for evaluation and demos, and for production throughput you would export the merged model and serve it with vLLM or SGLang.

The architecture-shaped warning is about where failures actually live. Because LLaMA-Factory orchestrates rather than reimplements, a crash deep in a training step is almost always in the layer underneath, Transformers, PEFT, TRL, bitsandbytes, or DeepSpeed, and the stack trace will say so.

your run:   llamafactory-cli train cfg.yaml
              |
   thin      | LLaMA-Factory: parse args, pick template + dataset + stage, wire trainer
   layer     v
   -------------------------------------------------------------------
   heavy     Transformers Trainer / TRL trainers   <- the real training loop
   lifting   PEFT (LoRA)  bitsandbytes (quant)      <- adapters and 4-bit math
             DeepSpeed / Accelerate / FSDP          <- multi-GPU sharding
             flash-attention kernels                <- attention

The practical implication is that LLaMA-Factory pins the versions of these libraries it is tested against, and the common source of mysterious breakage is a manually upgraded transformers or trl that no longer matches. When debugging, read the traceback for the layer that raised, and treat LLaMA-Factory's own code as the wiring diagram that tells you which knob fed that layer.

Part IV: The full life of one fine-tune

The specimen is one LoRA SFT run of Llama 3 8B Instruct on the demo data, launched with llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml. The same skeleton runs for every stage, and where the path forks by stage I say so.

Stage 1: the CLI and the launcher

llamafactory-cli resolves to main in src/llamafactory/cli.py. It reads the first token of the command line as a subcommand and dispatches. For train it must decide between a plain in-process run and a distributed one. If FORCE_TORCHRUN is set or more than one GPU is visible, it re-executes itself under torchrun with the resolved node and process counts, so that the same file becomes an SPMD program across ranks. Otherwise it calls run_exp directly in the current process. On a single GPU our specimen takes the direct path, which is also the path to step through with a debugger on a first read.

Stage 2: parsing into five dataclasses

run_exp begins by calling get_train_args in src/llamafactory/hparams/parser.py, which uses a Hugging Face HfArgumentParser to turn the YAML file (or the raw flags) into five typed argument objects, ModelArguments, DataArguments, Seq2SeqTrainingArguments, FinetuningArguments, and GeneratingArguments. This is also where consistency checks live, for example that a full fine-tune is not requested alongside quantization, that a template is present, and that the requested stage is compatible with the chosen options. The payoff of parsing to dataclasses up front is that every later component receives a small, typed slice of the configuration rather than a bag of strings. The sibling functions get_infer_args and get_eval_args serve the chat and eval commands the same way.

Stage 3: dispatch on the stage

With arguments in hand, run_exp in src/llamafactory/train/tuner.py branches on finetuning_args.stage. It is a small, honest strategy switch, and it is the spine of the whole project.

# train/tuner.py, in spirit
if finetuning_args.stage == "pt":
    run_pt(model_args, data_args, training_args, finetuning_args, callbacks)
elif finetuning_args.stage == "sft":
    run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
elif finetuning_args.stage == "rm":
    run_rm(...)
elif finetuning_args.stage == "ppo":
    run_ppo(...)
elif finetuning_args.stage == "dpo":
    run_dpo(...)
elif finetuning_args.stage == "kto":
    run_kto(...)

Our specimen has stage: sft, so control enters run_sft in src/llamafactory/train/sft/workflow.py. Each stage has a workflow.py that assembles the run and a trainer.py that customizes the trainer, and reading one workflow top to bottom is the fastest way to see how the parts snap together.

Stage 4: tokenizer and template

run_sft first loads the tokenizer. load_tokenizer in src/llamafactory/model/loader.py returns a small module holding the tokenizer and, for multimodal models, a processor. Then get_template_and_fix_tokenizer in src/llamafactory/data/template.py looks up the named template and, crucially, fixes the tokenizer in place. Fixing means adding a pad token if the model lacks one, registering the template's stop words as special tokens so generation halts correctly, and reconciling the EOS. The returned Template object carries the formatting rules that the data pipeline will apply. Getting this step right is why a matched template matters so much, since the template and the tokenizer fixes together define the exact byte sequence the model sees.

Stage 5: building the dataset

get_dataset in src/llamafactory/data/loader.py turns the dataset names into tokenized tensors in three moves. First it resolves each name through data/dataset_info.json, parsing the entry into a DatasetAttr (in src/llamafactory/data/parser.py) that records where the data lives, whether it is alpaca or sharegpt formatted, and how its columns map to prompts, responses, system text, and tools. Then it loads the raw rows, from the Hugging Face Hub, a local file, or a loading script, and align_dataset in src/llamafactory/data/aligner.py normalizes every format into one internal message schema, so that all downstream code is format-agnostic. Finally a stage-specific preprocessing function tokenizes each example. For SFT that means concatenating the templated prompt and response into input_ids and building labels that copy the response tokens but replace every prompt token with the ignore index, so loss is only taken on the assistant's turn.

raw row (alpaca)            {instruction, input, output}
      |  align_dataset       normalize to a common message list
      v
_prompt = [{role: user, content: ...}]     _response = [{role: assistant, content: ...}]
      |  template.encode      apply llama3 chat format + special tokens
      v
input_ids = [ <prompt tokens> <response tokens> <eos> ]
labels    = [ -100 ... -100   <response tokens> <eos> ]   # prompt masked out

The ignore index is the standard -100 that PyTorch's cross-entropy skips, defined among the project's constants. Preference stages diverge here. DPO and reward modeling build a chosen and a rejected sequence per row, and KTO builds a single sequence with a binary desirability label, which is why those stages demand ranking-style datasets.

Stage 6: loading the model and attaching the adapter

load_model in src/llamafactory/model/loader.py builds the network. It first patches the configuration through src/llamafactory/model/patcher.py, which is where rope scaling for longer context, the attention implementation such as FlashAttention 2, and quantization settings are applied. If the run is QLoRA, the weights load in 4-bit through bitsandbytes here. Then init_adapter in src/llamafactory/model/adapter.py realizes the finetuning_type. For full it leaves every parameter trainable, for freeze it unfreezes only a chosen subset of layers, and for lora it constructs a PEFT LoraConfig targeting the modules named by lora_target and wraps the base model with get_peft_model. With lora_target: all the code discovers every linear module automatically, so you do not have to know a model family's internal names. After this step only the adapter's small matrices carry gradients, which is why a LoRA fine-tune of an 8B model fits where a full one would not.

Stage 7: the trainer

run_sft then constructs a CustomSeq2SeqTrainer from src/llamafactory/train/sft/trainer.py, a subclass of the Transformers Seq2SeqTrainer with a handful of targeted changes, including how the loss is computed and how predictions are generated for evaluation. It receives the model, the tokenizer, the processed dataset, a data collator that pads variable-length sequences, and a set of callbacks. One of those callbacks writes a running JSON-lines log that the web UI tails to draw its live loss curve. Nothing here reimplements backpropagation. The heavy lifting is the inherited Hugging Face training loop, and LLaMA-Factory's contribution is the correct assembly of its inputs.

Stage 8: train, save, and close the loop

trainer.train() runs the ordinary loop, forward, loss, backward, optimizer step, logging, and periodic checkpoints. Because only the adapter has gradients, the optimizer state and the saved artifact are both tiny. At the end trainer.save_model() writes the LoRA adapter and its config, the tokenizer, and the training arguments, and a plotting helper renders the loss curve when plot_loss is set. The trainer_state.json preserves the full metric history. That closes the loop of one fine-tune, data in through two registries, a base model quietly wrapped with an adapter, and a small, mergeable set of weights out. Running llamafactory-cli export afterward calls export_model, which loads the base plus the adapter, merges them, and writes a standalone model that any inference engine can serve.

Part V: Internals deep dives

Deep dive: the template registry

The template system is the first of the two load-bearing ideas. A model's chat format is not cosmetic. The exact placement of system text, the role markers, the separators, and the special tokens are part of what the model learned, and feeding it a different arrangement degrades it quietly. LLaMA-Factory captures each family's format as a registered Template in src/llamafactory/data/template.py. A template is a dataclass of formatters, one each for the user turn, the assistant turn, the system prompt, function calls, tool observations, and the separator, plus metadata such as the default system message, the stop words, and whether the format needs an efficient EOS. The formatters themselves are small classes, StringFormatter for literal templated text, FunctionFormatter and ToolFormatter for tool-calling, and EmptyFormatter for the degenerate case.

# register_template(...), in spirit
register_template(
    name="llama3",
    format_user=StringFormatter(slots=["<|start_header_id|>user<|end_header_id|>\n\n{{content}}<|eot_id|>..."]),
    format_assistant=StringFormatter(slots=["{{content}}<|eot_id|>"]),
    format_system=StringFormatter(slots=["<|start_header_id|>system<|end_header_id|>\n\n{{content}}<|eot_id|>"]),
    stop_words=["<|eot_id|>"],
    ...
)

At use time get_template_and_fix_tokenizer returns the named template and mutates the tokenizer so its special tokens and stop words agree with the format. The template then exposes encoding methods that turn a list of role-tagged messages into token ids, with the multi-turn variant laying out an entire conversation so the loss mask can cover every assistant turn while hiding every user turn. Because the format is data rather than code, supporting a new model family is usually one register_template call, and the same dataset trains correctly against any model once you name the matching template. The trap to internalize is that the template and the base checkpoint are a pair. The registry frees you from writing formatting code, but it cannot guess which format a checkpoint expects, so choosing the template remains your responsibility.

Deep dive: the dataset registry

The dataset registry is the second load-bearing idea, and it is deliberately just a JSON file, data/dataset_info.json, plus the code that reads it. Each entry gives a dataset a name and declares how to find and interpret it. A minimal alpaca-style local dataset needs only a file name, a formatting tag, and a column mapping.

"my_dataset": {
    "file_name": "my_data.json",
    "formatting": "alpaca",
    "columns": { "prompt": "instruction", "query": "input", "response": "output" }
}

"my_chat": {
    "file_name": "conversations.json",
    "formatting": "sharegpt",
    "columns": { "messages": "conversations" },
    "tags":    { "role_tag": "from", "content_tag": "value",
                 "user_tag": "human", "assistant_tag": "gpt" }
}

"my_prefs": {
    "file_name": "prefs.json",
    "formatting": "sharegpt",
    "ranking": true,
    "columns": { "messages": "conversations", "chosen": "chosen", "rejected": "rejected" }
}

The two formatting styles cover most of the world. alpaca is the instruction, optional input, and output triple, while sharegpt is a list of role-tagged turns, which the tags block maps onto standard roles. A ranking flag with chosen and rejected columns marks a preference dataset for DPO or reward modeling. The parser in src/llamafactory/data/parser.py reads each entry into a DatasetAttr, and the aligner in src/llamafactory/data/aligner.py converts every raw row into the single internal message schema. This is the decisive design move. Because alignment collapses many external formats into one, the template code, the preprocessing code, and the trainers never branch on where the data came from. Adding your own data is adding one JSON entry and a file, not writing a loader, which is a large part of why a first fine-tune on custom data is achievable in an afternoon.

Deep dive: trainer dispatch and the six stages

The run_exp switch from Part IV fans out into six self-contained stage packages under src/llamafactory/train/, and each rests on a well-known base trainer rather than a bespoke loop.

SFT and PT are thin subclasses of the Hugging Face trainers. Reward modeling uses a custom pairwise trainer that runs the model with a value head over the chosen and rejected sequences and takes a margin loss on their scalar scores. PPO wraps TRL's PPOTrainer and loads the policy as an AutoModelForCausalLMWithValueHead, sampling responses, scoring them with a reward model, and taking the clipped policy-gradient step. DPO and KTO wrap TRL's respective trainers, comparing the policy against a frozen reference to move probability mass toward preferred outputs. The reinforcement learning ideas behind these objectives are developed in the RL section and the deep reinforcement learning notes. One subtlety worth flagging. ORPO and SimPO do not have their own stages. They run under stage: dpo with a different pref_loss, because the custom DPO trainer supports a family of preference losses, and ORPO in particular needs no separate reference model. The dispatch is a clean strategy pattern. One switch, six workflows, each delegating the actual optimization to a battle-tested base trainer, which is how the project supports the full post-training spectrum without owning a single training loop.

Deep dive: model loading, adapters, and quantization

The model/ package is where breadth is bought cheaply. patcher.py normalizes the differences between architectures before weights load, setting the attention implementation, rope scaling, and dtype, and applying quantization configuration. loader.py then instantiates the model through the ordinary Transformers Auto classes, which is why any architecture the Transformers library knows is a candidate for fine-tuning. adapter.py realizes the tuning strategy. QLoRA is simply LoRA plus a quantization_bit that loads the frozen base in 4-bit through bitsandbytes, so the large weights sit quantized and only the small adapter trains in higher precision. A generous set of optional accelerations plug in around this core, including FlashAttention 2 via flash_attn, the Unsloth kernels via use_unsloth, the Liger kernels via enable_liger_kernel, memory-efficient full-parameter optimizers such as GaLore and BAdam, and variants like DoRA and rank-stabilized LoRA. The attention-memory reasoning that makes these kernels fast is derived in the FlashAttention chapter. The lesson of this package is that model-specific knowledge is confined to patches and a template, so the trainers above never need to know which family they are training.

Deep dive: LlamaBoard and the web UI

LlamaBoard, under src/llamafactory/webui/, is a Gradio front end that generates and runs the very same commands. interface.py builds the component tree of dropdowns and fields, and a Runner in runner.py collects the current UI state, assembles the argument set, and launches training as a subprocess through the same entry point, then tails the run's log file to stream loss and progress back into the browser. The manager and engine pieces coordinate component state and preview the exact command before you run it. Nothing about training lives in the UI. It is a configuration builder over run_exp, which is why the command it previews is one you could have typed, and why the UI and the CLI can never drift in behavior. This is the architectural reason the project can offer a friendly on-ramp without maintaining two code paths.

Part VI: Reading the repository

The tree is larger than a minimal trainer but very navigable, because the layers are clean. Read it in stages from the entry points inward. Paths below reflect the layout as of mid 2026, and a few may have moved.

Stage 0, orientation. Read the README, then open two or three example configs under examples/, for instance train_lora/llama3_lora_sft.yaml, a full fine-tune config, and a DPO config, comparing the fields that change. Glance at pyproject.toml to confirm the llamafactory-cli console entry point. Questions to hold. Which fields select the algorithm versus the tuning method, and what does a dataset name have to resolve to?

Stage 1, the front door. Read src/llamafactory/cli.py. Notice the subcommand dispatch and the decision to re-exec under torchrun for multi-GPU. Questions. What does FORCE_TORCHRUN change, and which functions do the train, chat, and export commands ultimately call?

Stage 2, the argument surface. Read src/llamafactory/hparams/parser.py and skim the argument dataclasses model_args.py, data_args.py, finetuning_args.py, and generating_args.py. This is the full menu of what the project can do. Questions. Where do the cross-field validations live, and what are the fields on FinetuningArguments that select stage, tuning type, and preference loss?

Stage 3, one workflow end to end. Read src/llamafactory/train/tuner.py for the dispatch, then train/sft/workflow.py from top to bottom, with train/sft/trainer.py beside it. This single file threads together tokenizer, template, dataset, model, adapter, and trainer. Questions. In what order are the pieces built, and what does the custom trainer change relative to the Hugging Face base?

Stage 4, the data pipeline. Read the data/ package in the order the pipeline runs, parser.py, then aligner.py, then template.py, then loader.py, and finally collator.py and the preprocessing functions. Questions. How does an alpaca row and a sharegpt row end up in the same internal shape, and where exactly does the prompt get masked out of the labels?

Stage 5, the model layer. Read model/loader.py, model/adapter.py, and model/patcher.py. Questions. Where does QLoRA's 4-bit loading happen, how does lora_target: all discover the linear modules, and which accelerations are switched on by which flags?

Stage 6, the rest of the stages and the UI. Read the train/dpo, train/rm, train/ppo, and train/kto packages to see how each leans on its base trainer, then webui/runner.py to see the UI generate the same run. Questions. Which base trainer does each stage subclass, and how do ORPO and SimPO fit under the DPO stage?

Where not to start. The multimodal plumbing, the plugins that handle image and audio inputs, adds a processor and a lot of format handling that only makes sense once the text path is solid. The api/ server and the eval/ harness are useful but peripheral to how training works. And the DeepSpeed and FSDP config files under examples/ are best read after you have run a single-GPU job, since they are about scaling a pipeline you already understand.

Part VII: Hands-on labs

Labs 1 through 3 run on a single modest GPU, and several shrink further with max_samples. Log formats and exact defaults drift with the fast pace of the project.

Lab 1: a first LoRA SFT. Concept: the eight-stage lifecycle of Part IV.

llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml \
  --max_samples 200 --num_train_epochs 1

# then talk to the result
llamafactory-cli chat examples/inference/llama3_lora_sft.yaml

Watch the startup logs in order, argument parsing, template and tokenizer setup, dataset alignment and tokenization, model and adapter construction, then the training steps. Match each phase to a stage of Part IV. Inspect output_dir and confirm it holds an adapter, not a full model.

Lab 2: register your own dataset. Concept: the dataset registry and the aligner.

# 1. put an alpaca-style file at data/my_data.json:
#    [{"instruction": "...", "input": "", "output": "..."}, ...]
# 2. add an entry to data/dataset_info.json:
#    "my_data": { "file_name": "my_data.json", "formatting": "alpaca" }
# 3. train on it
llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml --dataset my_data

Observe that no code changed, only one JSON entry and a data file. Then break it deliberately by mislabeling formatting as sharegpt and read how the aligner fails, which teaches what alignment expects.

Lab 3: watch the wrong template break generation. Concept: the template is load-bearing.

# right template
llamafactory-cli chat examples/inference/llama3_lora_sft.yaml
# wrong template on the same model, expect a missing stop and odd formatting
llamafactory-cli chat examples/inference/llama3_lora_sft.yaml --template qwen

Compare the two conversations. With the mismatched template the special tokens and stop words no longer line up, and the model rambles past where it should stop. This is the single most common silent misconfiguration made real.

Lab 4: DPO, then ORPO with no reference model. Concept: preference stages and the DPO trainer's loss family.

# DPO on a preference dataset
llamafactory-cli train examples/train_lora/llama3_lora_dpo.yaml --max_samples 200
# ORPO runs under the dpo stage with a different preference loss
llamafactory-cli train examples/train_lora/llama3_lora_dpo.yaml \
  --pref_loss orpo --max_samples 200

Note that the dataset now carries chosen and rejected responses, and that ORPO trains without loading a separate reference model, which is visible in the memory footprint and the startup logs.

Lab 5: QLoRA on a small budget. Concept: 4-bit base plus a trainable adapter.

llamafactory-cli train examples/train_qlora/llama3_lora_sft_otfq.yaml \
  --max_samples 200
# compare peak GPU memory against the bf16 LoRA run from Lab 1

Observe the far lower memory. The frozen base sits in 4-bit through bitsandbytes while the LoRA matrices train in higher precision, which is exactly the QLoRA idea. Confirm the exact example filename in examples/, as the names evolve.

Lab 6: LlamaBoard equals the CLI. Concept: the UI is a command builder over run_exp.

llamafactory-cli webui
# in the browser: pick the model, template, dataset, stage, and method,
# then use the preview to see the exact command before running it

Configure a run in the UI, read the previewed command, and notice it is a llamafactory-cli train invocation you could have typed. Run it from the UI, then run the identical command in a terminal, and confirm they behave the same. That equivalence is the whole design.

Part VIII: Questions and model answers

Understanding checks. Answer aloud before reading.

1. What is LLaMA-Factory, in one sentence?

An orchestration layer over Transformers, PEFT, and TRL that turns fine-tuning a known open model with a standard method into a single config, by registering the chat template, the dataset format, and the training stage as swappable parts reachable from a CLI, a Python call, or the LlamaBoard web UI.

2. What do the two orthogonal choices stage and finetuning_type control?

stage selects the algorithm, pt, sft, rm, ppo, dpo, or kto, and therefore which workflow and base trainer run. finetuning_type selects how much of the model moves, full, freeze, or lora. They combine freely, so LoRA DPO and full SFT are both ordinary combinations.

3. Why is choosing the right template so important?

The template fixes the exact chat format, special tokens, and stop words the base model was trained with, and it also drives the tokenizer fixes and the loss-mask boundaries. A mismatched template yields fluent-looking but degraded output and an EOS that never fires, and the registry cannot infer which format a checkpoint expects.

4. Trace how a dataset name becomes tokenized tensors.

The name resolves through data/dataset_info.json into a DatasetAttr, the raw rows load from the hub, a file, or a script, the aligner normalizes alpaca or sharegpt into one message schema, and a stage-specific preprocessing function tokenizes each example, building labels that mask the prompt for SFT or a chosen and rejected pair for preference stages.

5. What does run_exp actually do?

It parses arguments into five dataclasses, then switches on finetuning_args.stage to call the matching workflow, run_sft, run_dpo, and so on. It is the single dispatch point that fans out into the six stage packages.

6. Where does the real training loop live?

In the base trainers underneath. SFT and PT subclass the Hugging Face trainers, DPO, KTO, and PPO wrap the TRL trainers, and reward modeling uses a custom pairwise trainer. LLaMA-Factory assembles their inputs and customizes small behaviors, it does not reimplement backpropagation.

7. How does QLoRA differ from LoRA in this codebase?

QLoRA is LoRA plus a quantization_bit that loads the frozen base weights in 4-bit through bitsandbytes during model patching, so the large weights stay quantized while only the small adapter trains in higher precision. It is the same adapter path with a quantized base.

8. Where do ORPO and SimPO live, and why?

Under the dpo stage, selected by pref_loss, because the custom DPO trainer supports a family of preference losses. ORPO in particular needs no separate reference model, so folding it into the DPO workflow avoids a redundant stage.

9. How can one command fine-tune a hundred different model families?

Because model-specific knowledge is confined to a registered template and a loading patch, while the model itself instantiates through the Transformers Auto classes. The trainers above never branch on family, so adding support is usually a template and a patch rather than new training code.

10. Why does a preference method fail on plain SFT data?

DPO and reward modeling need chosen and rejected responses per example, and KTO needs a binary desirability label. An SFT dataset has neither, so the ranking-aware preprocessing has nothing to build the pair or the label from and fails at data loading.

11. When would you reach for TRL, Unsloth, or OpenRLHF instead?

TRL when you want to own the training script directly with minimal indirection, since it is the library being wrapped. Unsloth when single-GPU LoRA speed and memory dominate, though LLaMA-Factory can call it as a backend. OpenRLHF or verl when large-scale distributed PPO throughput is the goal, which is beyond LLaMA-Factory's convenient single-node RLHF.

12. A training step crashes deep in a stack trace. Where do you look first?

At the library that raised, since LLaMA-Factory orchestrates rather than reimplements. Most step-time failures come from Transformers, PEFT, TRL, bitsandbytes, or DeepSpeed, and the common root cause is a manually upgraded dependency that no longer matches the pinned versions.

13. What is the relationship between the CLI and LlamaBoard?

The web UI is a configuration builder that assembles the same arguments and launches the same entry point as a subprocess, then tails the log file for a live loss curve. The command it previews is one you could type, so the two can never diverge in behavior.

14. Where does LLaMA-Factory stop, and what takes over?

It fine-tunes and exports checkpoints, it does not pretrain at cluster scale, which is a platform like torchtitan, and it is not a production serving engine, which is vLLM or SGLang on the merged model. It hands off cleanly at both ends.

Part IX: Design lessons

Orchestrate, do not reimplement. The project delegates the training loop to Transformers and TRL and spends its own code on assembling their inputs correctly. Reusing hardened components instead of rewriting them is why one small team can keep pace with a hundred models and six methods. The same instinct underlies good glue code everywhere, own the wiring, borrow the engine.

Turn the variable parts into registries. The two things that change between fine-tunes, the chat format and the data format, become a template registry and a JSON dataset registry, so supporting a new model or dataset is a declaration rather than a code change. Whenever a system has a small number of axes that vary a lot, naming those axes as data pays off for years.

One dispatch point, many strategies. run_exp is a single switch on the stage that fans out into self-contained workflows, each delegating to a base trainer. The strategy pattern keeps the six algorithms from tangling, and new methods slot in as new branches. Centralizing the branch also centralizes the place to reason about what a stage requires.

Normalize early so everything downstream is simple. The aligner collapses many external data formats into one internal schema at the boundary, so the template code, the preprocessing, and the trainers never branch on provenance. Pushing format variety to the edge and keeping the core uniform is the same discipline as parsing untyped input into typed structures at a service boundary.

Let every interface hit the same core. The CLI, the Python API, and the web UI all funnel into run_exp, so there is one behavior to test and no risk of the friendly path drifting from the powerful one. A UI that generates the command it runs is honest, inspectable, and cheap to maintain.

Lower the barrier with batteries included. Demo datasets, ready-made example configs, and defaults like targeting every linear layer for LoRA turn a first fine-tune into one command. Making the easy case trivial, without hiding the knobs the hard case needs, is what actually brings new people into a field.

Part X: Memorization framework

The one-sentence summary. LLaMA-Factory parses a config into five dataclasses, dispatches on the stage into one of six workflows, builds the run from a registered template and a registered dataset, wraps a Transformers-loaded model with the chosen adapter, and hands the whole thing to a lightly subclassed Hugging Face or TRL trainer, so post-training becomes a config rather than a script.

llamafactory-cli train cfg.yaml -> cli.py (dispatch, maybe torchrun)
  -> get_train_args -> 5 dataclasses (model, data, training, finetuning, generating)
  -> run_exp -> switch on stage {pt, sft, rm, ppo, dpo, kto}
  -> template + tokenizer -> dataset (parse -> align -> template -> tokenize -> mask)
  -> load_model + init_adapter (full | freeze | lora | qlora)
  -> Custom trainer.train() -> save adapter + logs + loss plot

The chain mapped to source.

entry             src/llamafactory/cli.py
args              src/llamafactory/hparams/parser.py (+ *_args.py)
dispatch          src/llamafactory/train/tuner.py (run_exp)
workflow          src/llamafactory/train/{sft,pt,rm,ppo,dpo,kto}/workflow.py
template          src/llamafactory/data/template.py (get_template_and_fix_tokenizer)
dataset           data/dataset_info.json -> data/parser.py -> data/aligner.py -> data/loader.py
model + adapter   src/llamafactory/model/{loader,patcher,adapter}.py
trainer           src/llamafactory/train/{stage}/trainer.py (HF / TRL subclass)
web ui            src/llamafactory/webui/{interface,runner}.py

Memorize these blocks.

• Six stages: pt, sft, rm, ppo, dpo, kto, dispatched by run_exp on finetuning_args.stage. ORPO and SimPO ride under dpo via pref_loss.
• Three tuning types: full, freeze, lora, orthogonal to the stage, with QLoRA being lora plus a 4-bit base.
• Two registries: the template registry in data/template.py for chat formats, and the dataset registry in data/dataset_info.json for data formats and columns.
• Two data formats: alpaca (instruction, input, output) and sharegpt (role-tagged turns via tags), both normalized by the aligner into one internal schema.
• Base trainers: SFT and PT on Hugging Face trainers, DPO, KTO, and PPO on TRL, reward modeling on a custom pairwise trainer.
• Three front doors, one core: CLI, Python run_exp, and LlamaBoard all funnel into the same dispatch.

Part XI: Papers and further reading

The methods this walkthrough keeps naming come from a small set of papers, and each one rewards a direct read. Where this site works the same idea in depth, the companion link points there.

1. Zheng et al., LlamaFactory, Unified Efficient Fine-Tuning of 100+ Language Models, ACL 2024. The system paper behind this repository, describing the unified fine-tuning framework and the LlamaBoard UI.
2. Hu et al., LoRA, Low-Rank Adaptation of Large Language Models, 2021. The adapter method behind finetuning_type: lora, realized here through PEFT and covered in the PEFT walkthrough on this site.
3. Dettmers et al., QLoRA, Efficient Finetuning of Quantized LLMs, 2023. The 4-bit NormalFloat base plus trainable adapter recipe that quantization_bit switches on through bitsandbytes.
4. Schulman et al., Proximal Policy Optimization Algorithms, 2017. The clipped objective the ppo stage runs through TRL, derived step by step in the PPO note and the deep reinforcement learning class.
5. Ouyang et al., Training language models to follow instructions with human feedback, 2022. The SFT then reward model then PPO pipeline that the pt through ppo stages retrace.
6. Rafailov et al., Direct Preference Optimization, Your Language Model is Secretly a Reward Model, 2023. The preference objective behind the dpo stage, worked in full in the DPO note on this site.
7. Ethayarajh et al., KTO, Model Alignment as Prospect Theoretic Optimization, 2024. The binary-feedback objective behind the kto stage, which is why that stage wants a desirability label rather than a chosen and rejected pair.
8. Hong et al., ORPO, Monolithic Preference Optimization without Reference Model, 2024. The reference-free preference loss that rides under the dpo stage via pref_loss.
9. Meng et al., SimPO, Simple Preference Optimization with a Reference-Free Reward, 2024. The other loss in the DPO trainer's family, using average log probability as the implicit reward.
10. Zhao et al., GaLore, Memory-Efficient LLM Training by Gradient Low-Rank Projection, 2024. The gradient-projection optimizer that makes full-parameter tuning fit in adapter-sized memory, one of the optional accelerations in Part V.

Part XII: Final takeaway

If the pieces underneath LLaMA-Factory are the gap, the paths out are short. The base models come from Transformers, the multi-GPU degrees are the subject of the parallel computing notes, the RLHF objectives are developed in the RL section, and once you have a merged model the serving story continues in vLLM. Come back and read train/sft/workflow.py once more, and it will read like a checklist of the eight stages in Part IV, which is exactly the point.