How to Fine-Tune a Hugging Face Model and Share It on the Hub

Full fine-tuning a 7B model needs 8 GPUs. LoRA fine-tuning the same model needs one 24 GB card and about 40 minutes for a 5,000-example dataset. That gap is why LoRA is the default for anything short of a research project.

Direct answer: to fine-tune a Hugging Face model, use PEFT with LoRA adapters — set rank 16, alpha 32, learning rate 2e-4, and train only the attention projections. The adapter file is typically 20-80 MB instead of 14 GB, and you push it to the Hub with model.push_to_hub().

Should you fine-tune at all?

Usually no. Prompt engineering and few-shot examples solve most formatting and tone problems, and they cost nothing to iterate on. Fine-tuning earns its keep in three cases:

• You need a consistent output schema across thousands of calls and prompting keeps drifting.
• The task is narrow and repetitive enough that a 3B model matches a 70B one after training.
• You need to strip a specific behaviour (a refusal pattern, a chatty preamble).

If none of those apply, you are about to spend two days to beat a better prompt. Try the prompt first.

How do you prepare the dataset?

A JSONL file, one object per line, with a messages array. About 500 examples is the floor for a visible effect; 2,000-5,000 is where it becomes reliable. Quality dominates quantity — 500 carefully written examples beat 10,000 scraped ones every time.

{"messages": [{"role": "user", "content": "Classify: refund request"}, {"role": "assistant", "content": "{\"intent\": \"billing\", \"urgency\": \"medium\"}"}]}

Split 90/10 train/eval before you start, and never let evaluation examples appear in training. That sounds obvious; it is also the single most common way people get impressive-looking loss curves and useless models.

What settings actually matter?

from peft import LoraConfig, get_peft_model

cfg = LoraConfig(
    r=16, lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05, task_type="CAUSAL_LM",
)
model = get_peft_model(base_model, cfg)

Setting	Typical	Effect
r (rank)	8-32	capacity; 16 covers most tasks
lora_alpha	2× rank	scaling; higher = stronger adaptation
learning rate	1e-4 to 3e-4	2e-4 is a safe default
epochs	2-3	beyond 3 it memorises
batch size	4 + grad accum 4	effective batch 16
target_modules	q_proj, v_proj	adding k_proj, o_proj helps on hard tasks

Rank 64 is almost always wasted parameters on a task under 10,000 examples. Start at 16, check eval loss, and only raise it if the model underfits.

Use trl‘s SFTTrainer rather than a hand-rolled loop. It handles the chat template, packing, and gradient checkpointing correctly:

from trl import SFTTrainer, SFTConfig
trainer = SFTTrainer(model=model, train_dataset=ds, args=SFTConfig(
    output_dir="./out", num_train_epochs=3, per_device_train_batch_size=4,
    gradient_accumulation_steps=4, learning_rate=2e-4, logging_steps=10))
trainer.train()

How do you know it worked?

Loss going down is not evidence. Run the base model and the tuned model on the same 100 held-out prompts and compare on the thing you actually care about — exact-match on the JSON schema, or a human rating on tone.

A realistic expectation for a well-scoped classification task: 60% schema compliance before, 95%+ after. For open-ended writing, expect much less; LoRA does not teach a model new knowledge, only new behaviour.

Watch for regression on general capability. A model tuned hard on 2,000 examples of one task gets noticeably worse at everything else. If you need both, keep two models or use a router.

How do you publish to the Hub?

huggingface-cli login

model.push_to_hub("yourname/model-lora-intent")
tokenizer.push_to_hub("yourname/model-lora-intent")

Merge first if you want a standalone model rather than an adapter:

merged = model.merge_and_unload()
merged.push_to_hub("yourname/model-merged-intent")

Adapters are the better default. They are one file, load on top of the base model in a second, and let you ship five task-specific variants without five full copies.

Write a real model card. State the base model, the dataset size and source, the evaluation numbers, and the known failure cases. Models with honest limitations sections get used; models with a three-line card do not.

What goes wrong in practice

• Training loss spikes to NaN: learning rate too high. Halve it.
• Eval loss rises while train loss falls: overfitting at epoch 2. Stop earlier.
• Output ignores the prompt: chat template mismatch between training and inference.
• Adapter loads but output is garbage: target_modules did not match the architecture.

Set save_strategy="epoch" and keep the checkpoint with the best eval loss, not the last one. The final checkpoint is rarely the best.

Key Takeaways

• LoRA on one 24 GB GPU handles what full fine-tuning needs 8 GPUs for.
• Rank 16, alpha 32, learning rate 2e-4, two to three epochs — start there, adjust one thing at a time.
• 500 good examples is the floor; 2,000-5,000 is where results stabilise.
• Evaluate on held-out data with task-specific metrics, not loss curves.
• Push adapters, not merged weights, unless you need a standalone artifact.