Reading Loss Curves: What You’re Training Run is Trying to Tell You

Abstract

A loss curve is the closest thing you have to a window into what your model is learning during training. But most people either ignore it or misread it. A loss curve that looks good can still mean your model is broken. A curve that looks scary can still mean your run is fine. This post explains what loss curves actually show, the patterns that matter, and how to know when to keep training versus when to stop and fix something first.

You kicked off your training run. The logs are scrolling. The loss number is dropping.

And you're not sure if that's good or not.

Most people in this situation do one of two things. They either walk away and hope for the best, or they stare at the numbers without really knowing what they're looking for. Neither one works.

A loss curve is your training run talking to you. It's telling you whether your model is learning, overfitting, memorizing garbage, or slowly going off the rails. The problem is it doesn't speak plain English. You have to know what the patterns mean.

So let's go through them.

What Loss Actually Measures

Loss is a number that measures how wrong your model is on each prediction. High loss means the model is making bad predictions. Low loss means it's making better ones.

During training, your loss should go down. That's the whole point. The model is updating its weights to get better at predicting your training data.

But there's a second number you need to watch at the same time. Your validation loss. This measures how wrong the model is on data it hasn't seen during training.

Train loss and validation loss tell two different stories. And the gap between them tells a third.

The Healthy Pattern

A healthy training run looks like this. Both train loss and validation loss drop steadily in the early steps. Then they start to flatten out as the model approaches the limits of what it can learn from your data.

The two curves track each other closely. Train loss is usually slightly lower than validation loss, and that's fine. A small gap is normal.

If you see this pattern, your model is learning. The data is clean. The config is reasonable. You're in good shape.

The Overfitting Pattern

Here's where people get fooled.

Your train loss keeps dropping. Looking great. But your validation loss stops dropping and starts creeping back up.

That's overfitting. The model has stopped learning general patterns from your data and started memorizing the training examples themselves. It's getting better at your training set and worse at everything else.

This is one of the most common problems in fine-tuning and it's easy to miss if you're only watching train loss.

What do you do about it? A few options. Stop training earlier — your best checkpoint is probably from before the validation loss started rising. Add more data. Reduce your number of epochs. Or lower your lora_r to reduce the model's capacity to memorize.

The Flat Line Pattern

Your loss drops fast in the first few steps and then flatlines. Both curves. They just stop moving.

This usually means one of three things.

Your learning rate is too low and the model has stalled. Try bumping it up by a factor of 5 and rerunning.

Your dataset is too small and the model learned everything it can in the first pass. If you're training on a few hundred examples, this happens fast.

Or your dataset format is wrong. If the model is training on tokenization artifacts instead of real content, it will plateau fast because there's nothing meaningful to learn. This is the one that's hardest to catch because the loss doesn't spike. It just stops.

The Spiking Pattern

Your loss is dropping smoothly and then — spike. A sudden jump upward. Then it recovers and keeps dropping. Or it spikes and doesn't recover.

A single spike that recovers is usually a bad batch in your dataset. One malformed example that threw off that step. Not a crisis. Keep going and watch for more.

Repeated spikes mean your learning rate is too high. The model is overcorrecting on each step. Bring the learning rate down and rerun. For QLoRA, 2e-4 is a reasonable starting point. If you're spiking, try 1e-4.

A spike that doesn't recover and the loss just climbs from there is a sign something is more fundamentally wrong. Stop the run. Check your dataset for corruption. Check your config for mismatched settings.

The Suspiciously Perfect Pattern

Loss drops fast. Really fast. Both curves. It looks beautiful.

Be suspicious.

If your loss drops to near zero in the first epoch, the model is probably memorizing your data rather than learning from it. This happens when your dataset is too small, when there's data leakage between your train and validation sets, or when your dataset format is set up in a way that makes the task trivially easy.

A model that hits near-zero loss in training usually performs terribly on real inputs. The curve looked great. The model isn't.

What to Actually Watch For

You don't need to stare at every step. But there are three checkpoints worth building into your workflow.

First 10 percent of steps: Is the loss actually dropping? If it's flat or climbing from the start, stop. Something is wrong with your data or config. Don't burn the rest of the compute.

Midpoint: Is validation loss still tracking train loss? If the gap is starting to widen, you're approaching overfit territory. Note the step number. That's probably where your best checkpoint will be.

End of training: Did both curves flatten out smoothly or did something weird happen in the last stretch? A sudden drop at the very end sometimes means the model overfit hard in the final epochs.

One Thing People Skip

Most people set up training and forget to log validation loss at all.

If you only log train loss you are flying blind. You can have a perfectly shaped train loss curve with a completely broken model underneath it and you won't know until you try to use it.

Set val_set_size to at least 0.05 in your Axolotl config. Make sure eval_steps is set so validation runs regularly throughout training, not just at the end. And log both curves somewhere you can actually look at them — Weights and Biases, TensorBoard, or even just a CSV.

The loss curve is only useful if you're watching the right numbers.