LLM Training and Fine-Tuning Tools
Model Training is for the step after prompting: adapting or running a model yourself. The dataset validator checks a fine-tuning file line by line in the common chat, preference, Alpaca and ShareGPT formats, reports token counts and duplicates, converts between formats and splits off a validation set. The eval scorer compares model outputs with expected answers using exact match, token F1, ROUGE-L and other checks, so you can tell whether a fine-tune or a prompt change helped.
4 free tools · no sign-up · no ads
All Model Training (4)
Fine-Tuning Dataset Validator
Check and convert JSONL chat datasets before fine-tuningLLM GPU Memory Calculator
VRAM for inference, full fine-tuning, LoRA and QLoRALLM Training Compute Calculator
FLOPs, Chinchilla tokens, GPU hours and cost for pre-trainingLLM Eval Scorer
Score model outputs against expected answers with exact match, F1 and ROUGE
Which tool do I need?
Make sure a training file will be accepted
Find bad JSON, wrong roles and empty turns before you upload and pay for a run.
Find out if a model fits on your GPU
Pick the model, precision and context length, and see memory for inference or LoRA training.
Budget a pre-training run
Get FLOPs, GPU-hours, wall-clock days and cost from model size and tokens.
Tell whether a fine-tune improved answers
Score outputs from two models against the same expected answers and compare wins and losses.
About Model Training
The two calculators cover hardware. The GPU memory calculator adds up weights, KV cache, gradients and optimizer state for inference, full fine-tuning, LoRA and QLoRA, and shows which GPUs the job fits on. The training compute calculator applies the standard 6 x parameters x tokens estimate and the Chinchilla rule of thumb to give FLOPs, GPU-hours, days and cost for a pre-training run at a chosen utilisation.
Guides
- Fine-Tuning Datasets and LLM Evals: A Practical GuideSFT versus preference tuning, the JSONL formats trainers expect, what makes training data good, and how to hold out an eval set and read metrics like exact match, F1 and ROUGE-L.
- LLM GPU Memory and Training Compute, Worked OutHow much VRAM a model needs to serve, fine-tune or train with LoRA and QLoRA, how the KV cache grows with context, and how 6ND and Chinchilla turn into GPU-hours and cost.
Frequently asked questions
Should I fine-tune a model or improve the prompt first?
Improve the prompt and add retrieval first; they are cheaper and quicker to change. Fine-tuning pays off when you need a consistent format or style across many calls, or a smaller model that matches a larger one on a narrow task.
What is the difference between pre-training and post-training?
Pre-training teaches a model language and knowledge from a very large text corpus. Post-training, such as supervised fine-tuning and preference tuning, shapes an existing model to follow instructions, use a format or prefer certain answers, with far less data and compute.
How much data do I need to fine-tune a model?
Providers often suggest starting with tens to a few hundred good examples and adding more if evals keep improving. Quality and coverage of real cases matter more than volume, and a held-out set is needed to know if it worked.
Why does a model need more memory than its weights?
Inference also stores the KV cache, which grows with context length and batch size. Training adds gradients, optimizer state and activations, which is why full fine-tuning needs several times the memory of the weights alone and why LoRA and QLoRA are popular.