Community handbook · Music source separation
Training Audio Source Separation Models
How to train a vocal remover — the whole path, from choosing a GPU and building a dataset to fine-tuning a pre-trained model, reading the metrics and renting cloud compute.
Everything here was learned the slow way: failed runs, mismatched stem lengths, out-of-memory errors and a lot of listening. It is written for people who have never trained a model before and who want a straight answer to one question — how do I actually do this?
No prior machine-learning knowledge required. Comfort with a terminal, file paths and audio files is enough.
> python train.py --model_type mel_band_roformer --config_path config_vocals.yaml --results_path results/ --data_path /data/2stem_dataset/ --dataset_type 2 --num_workers 4 --device_ids 0 --start_check_point results/model.ckpt --valid_path /data/validation/ --metric_for_scheduler sdr --metrics sdr fullness bleedless
.ckpt + .yamlThe pipeline in one picture
Seven stages, in the order you will meet them. Everything on this site is one of these boxes in more detail.
- Step 01DatasetFLAC or WAV stems of the songs you care about
- Step 02Validation setA few tracks in 16-bit WAV: mixture + stems
-
Step 03Repo + modelMSST with a
.ckptinresults/and its YAML - Step 04ConfigChunk size, learning rate, instruments, epochs
- Step 05Training runYour GPU, or a rented one that finishes in hours
- Step 06MetricsSDR, l1_freq, fullness, bleedless — per epoch
- Step 07Your modelCheckpoints you can separate audio with
Not the training itself — the two stages either side of it. A dataset whose stems do not match in length, or a validation folder exported as FLAC, will stop a run before a single epoch completes. Part one exists to prevent exactly that.
Should you train at all?
Fine-tuning exists for the cases where no published model does what you need. Knowing when that is saves weeks.
A model misses your material
Some genres break generic separators. Metal is the classic example: so many overlapping sounds that most models smear instruments into each other. Roformers do fairly well, but they come with trade-offs — bleed, dullness, or a target that does not match what you want.
An existing model is already close
If a released checkpoint gets you 90% of the way, an ensemble or a different model will usually beat a from-scratch training run that costs GPU rent and weeks of your attention.
Fine-tune first, pre-train last
Fine-tuning shifts an existing model toward your data and converges in days. Training from nothing is dramatically slower and its early metrics look terrible — by design. Start at the shallow end.
What you need, in one line
A computer with an NVIDIA GPU (required — AMD and Apple GPUs are not supported by the training stack), at least 8 GB of VRAM and a reasonably fast card; or a rented GPU from vast.ai, runpod.io and similar. Plus Python, which you can grab from python.org.
For pre-training, or for shifting an existing model's target, a small card can work — slowly. For real training runs, no: rent cloud compute. A single Roformer at batch size 4 can fill an entire 140 GB H200, so treat small VRAM as a prototyping budget, not a training budget.
Can you do this with no prior knowledge?
Yes. That is the entire reason this guide exists. The repository's own documentation describes the training scripts accurately but never spells out the actual "how to" — which file goes where, which argument matters, and what the numbers scrolling past your screen mean. That is the gap this guide fills.
This covers everything learned over months of training, plus the fixes contributed by the community. It is not the final word: separation research moves fast, and there is always more to learn. Treat it as a solid starting map, not a contract.
The 60-second version
If you only read one list on this site, read this one. Every step links to the chapter that explains it properly.
-
Check your hardware
NVIDIA GPU, 8 GB+ VRAM, CUDA-capable. Otherwise budget for cloud rental — Chapter 12 covers it end to end.
-
Build a dataset of official stems
Instrumentals and vocals in FLAC or WAV (never MP3), arranged in one of two layouts — Chapter 03. Aim well past 200 files.
-
Build a validation set
A handful of tracks with
mixtureplus their stems, exported as 16-bit WAV — Chapter 04. -
Install the training repository
Download Music-Source-Separation-Training, install PyTorch and the requirements, then drop a checkpoint into
results/and its YAML at the repo root — Chapter 05. -
Point the train command at your data
Fill in the paths, dataset type and model arguments — Chapter 06, then tune the config in Chapter 07.
-
Run it and read the metrics
Watch loss fall, compare validation metrics per epoch, and listen to the checkpoints it stores — Chapters 11 and Appendix B.
What's inside
Five sections, thirteen chapters and four appendices. Each chapter follows the same shape: overview, numbered steps, then the caveats and the errors you are likely to hit.
Before you train
Hardware and software requirements, the two dataset layouts, how many files you actually need, and how to build a validation set that will not throw errors at you.
Open section → Chapters 05–07Setup & configuration
The training repository, config vs. checkpoint, the full command and argument reference, and how to tune the YAML — including the chunk size conversion table.
Open section → Chapters 08–12Training runs
Training from scratch, shifting a model's focus from one stem to another, fullness models and model fusion, then local runs and a full cloud walkthrough.
Open section → Chapter 13 + A–DReference
An error-to-fix index, a glossary of the vocabulary, a metric reference explaining what you are actually looking at, and the credits.
Open section → Appendix BWhat the numbers mean
SDR, L1 frequency, SI-SDR, Aura STFT, bleedless, fullness — what each metric measures and how to read them together instead of in isolation.
Open appendix → Chapter 12Cloud cheat sheet
Renting a GPU on vast.ai, wiring up Google Drive, the exact terminal sequence, the module rabbit hole, and the two fixes for the failed-wheel error.
Open chapter →Local or cloud?
The only decision you have to make before you start. Both paths use the same commands; they differ in cost, patience and how much you have to babysit.
Your own machine
- Zero rental cost, no upload time.
- Free for pre-training and for shifting a model's target on a smaller card.
- Your GPU is unusable for anything else while it runs — no games, no renders.
- Large batch sizes and long runs are usually out of reach.
A rented GPU
- Fast cards with large VRAM; runs finish in hours instead of weeks.
- Pay by the hour, so a stuck run costs real money.
- Setup is fiddlier: drive sync, terminal sequence, missing modules.
- Best paired with a small local test run so you know the command is right.
Validate your command locally for a few hundred steps first. An error in the config, a mismatched stem length or a wrong dataset type wastes cloud money at a much faster rate than it wastes local time.