goldhub/YuE2-RS-Finetunes

YuE2-RS · Ruach Studio's fine-tunes of YuE2-3B

Ruach Studio is open music generation that runs on your own hardware: the model, the score, the adapters and every take stay yours, and nothing is rented by the token. Open models are now good enough to stand on their own, and what was missing is the studio around them: that is what we build, in the open. Music should be free to make and to share: whoever has no voice, no instrument and no studio still has the right to lay their thoughts and feelings onto sound.

⚠ Not on free cloud GPUs. Ruach Studio does not run on Google Colab, Kaggle or any free notebook GPU, so don't spend an evening on it there: it is a whole studio, not a notebook. It builds its C++/CUDA engine on the machine it lives on, keeps a server and a lab of Python services running beside it, and takes about 33 GB of disk (60 GB with the Artist). A free session's card, memory, wiped disk and hours-long limit cannot hold that. It wants your own Linux computer, or a server you rent and keep, with an NVIDIA card: 24 GB for everything, 16 GB with the smaller model copies. The weights on this page load wherever YuE2-3B does, ComfyUI among them: the warning is about the studio.

YuE2-RS-FW, the First Wave, is our own training of m-a-p's YuE2-3B on a dataset we assembled. We trained it in Ruach Studio (its page) and publish it as merged weights: seven checkpoint and strength pairs, each as GGUF (BF16, Q8_0, Q6_K) and as BF16 safetensors. Ruach Studio 2.0.0 takes 2.4 × 0.75 as its default backbone.

An experiment, said plainly. As far as we know, this is the first public fine-tune of YuE2: a first wave, trained on one machine with three consumer cards, on a dataset we assembled ourselves, and judged by one pair of ears. It is a beginning, not a finished product. What it does well and where it falls short is written below as we heard and measured it, and the next wave learns from both.

Which one is best: what the ear chose, blind

Viktor heard eight merges blind: the ring's cards below in 20 styles and 9 languages, and one song of his own, the same seeds on each merge, the takes numbered at random and their names sealed until the end. Of the 272 takes, the 248 outside industrial metal were marked one by one, and 242 of them heard (a take without a mark was not heard and counts nowhere): 👎 −1, 👍 +1, ⭐ +2, and 🔒 (kept for good, «over the top») +2 more. Industrial metal's 24 are apart: all eight merges were on fire there, each with a vibe of its own.

merge	heard	points a take	👎	⭐	🔒	styles it led
3.4, music 0.75 / sound 0.6	28	2.25	0	19	2	6
0.7 × 0.75	31	2.19	1	23	2	7
2.4 × 0.75	30	2.13	3	20	5	4
3.4 × 0.75	31	2.10	2	18	5	6
1.5 × 0.75	31	1.97	2	17	4	5
4.0 × 0.7	30	1.93	2	17	4	4
3.4 × 0.7	30	1.80	5	19	3	3
4.0 × 0.65	31	1.77	4	18	2	4

19 of 242 thumbed down (7.9 %), 151 starred, 27 kept for good. Every merge is good. The gaps between them are within what the same marks give when shuffled inside each style (p ≈ 0.87), and on the styles heard with three seeds a seed moved a take more than the merge did. Across the styles the four at × 0.75 (0.7, 1.5, 2.4, 3.4) and 3.4 with the sound half at 0.6 lead, and the two late 4.0 merges and 3.4 × 0.7 trail and drew most of the thumbs down. 2.4 × 0.75 stays the default for its spread across styles; 0.7 × 0.75 is the lightest.

On Viktor's own song the order turned over. A dense, theatrical Russian song of his, two seeds a merge: 3.4 × 0.7, 4.0 × 0.65 and 3.4 with the sound at 0.6 took a 👍 and a ⭐ each, 4.0 × 0.7 a take kept for good, and the default 2.4 × 0.75 two thumbs down. One song is a small sample, so read it as a hint: for a demanding song of your own, try the late saves before you settle. A second song of his, an album track, is left out by his own word: what failed there was the style and the lyrics he wrote for it, not the merges.

merge	epoch	strength, music / sound	dose, music / sound	note
YuE2-3B-RS-FW-2.4x0.75	2.4	0.75 / 0.75	59 / 73	the default of Ruach Studio 2.0.0
YuE2-3B-RS-FW-0.7x0.75	0.7	0.75 / 0.75	29 / 39	the lightest; the cleanest in the ring below
YuE2-3B-RS-FW-1.5x0.75	1.5	0.75 / 0.75	44 / 57
YuE2-3B-RS-FW-3.4x0.7	3.4	0.7 / 0.7	72 / 83
YuE2-3B-RS-FW-3.4x0.75	3.4	0.75 / 0.75	77 / 88
YuE2-3B-RS-FW-4.0x0.65	4.0	0.65 / 0.65	75 / 83	the last save, merged lightest
YuE2-3B-RS-FW-4.0x0.7	4.0	0.7 / 0.7	80 / 90

The dose is the strength times the size of what the save adds to the weights (the Frobenius norm of its low-rank change, summed over a half). A later save adds more: from epoch 0.7 to 4.0 the change grows about three times on the music half and two and a half times on the sound half. So at one strength, a late save pushes much harder than an early one. That is why the late saves are merged lighter: 3.4 × 0.7 and 4.0 × 0.65 land on nearly the same dose, and the ear can then compare what each learned, not how hard it pushes.

What the ear has heard so far. The wave sounds cleaner than stock YuE2-3B, and in the ring below its 0.7 save measured less sand than both closed services. One thing came back from about 1.5 epochs on: a fine grain, a sand, on high female voices, heard most on a gospel card. A lighter strength is the answer under test, and the blind listening also has a merge with the sound half alone lowered (3.4, music 0.75 / sound 0.6).

The merges hold their scores. 320 plans in Full mode (the ring's 20 cards × the saves 0.7, 1.5, 3.4 and 4.0 × strengths 0.6, 0.65, 0.7 and 0.75) went through the engine's score guard, and none was broken; a second seed at 0.75 gave 80 out of 80, with a sung vocal line in every one. The default's save, 2.4, sang the ring's 40 at 0.75 with every score whole.

Why there are no LoRA files here

You don't serve a raw piece of steak in a restaurant instead of the finished dish. An adapter is that raw piece: cut out of the training, without the checkpoint chosen, the strength set and the listening that make it a dish. A merge is what we have tasted and serve. It also stays tied to YuE2-3B and to this card with its licence and credits, while a bare adapter can be laid on any checkpoint and vanish into someone else's mix. So only merges are published, in several strengths, never the adapters.

Files

file	for	size
YuE2-3B-RS-FW-<merge>-BF16.gguf	yue2.cpp and Ruach Studio, the whole merge	7.2 GB
YuE2-3B-RS-FW-<merge>-Q8_0.gguf	the same, quantized from the merged BF16	3.8 GB
YuE2-3B-RS-FW-<merge>-Q6_K.gguf	the same, smaller	2.9 GB
YuE2-3B-RS-FW-<merge>-BF16.safetensors	anything that loads m-a-p/YuE2-3B: m-a-p's own layout and tensor names, BF16	7.3 GB
comfyui/yue2_3b_RS-FW-<merge>_bf16.safetensors	ComfyUI: one checkpoint in Comfy-Org's layout, as Comfy-Org/YuE2 ships it, BF16	7.8 GB
comfyui/yue2_3b_RS-FW-<merge>_int8_convrot.safetensors	the same, ConvRot int8 like Comfy-Org's own yue2_3b_int8_convrot, for a smaller card	4.0 GB
config.json, generation_config.json, yue2_generation_config.json, modeling_yue2.py, qwen.tiktoken	m-a-p's, unchanged, for the safetensors

Each file carries its own name, licence, links, epoch and strengths: the GGUF in its header (general.*, ruach.fw.*, and heresy.merged_loras, the recipe), the safetensors in its metadata. The quants are made from the merged BF16 (yue2.cpp's quantize), not by merging into a quant. Each BF16 GGUF was checked tensor by tensor against the merge it came from, and each safetensors against the GGUF and m-a-p's file.

The VAE is not part of this repo. Use m-a-p's YuE2-Vae, as before; in GGUF form it is in Serveurperso/YuE2-GGUF.

Use

• Ruach Studio: a fresh install of 2.0.0 fetches 2.4 × 0.75 at the size your card takes (BF16 on 24 GB, Q8_0 from 7 GB, Q6_K from 4.5 GB). Engine › Models and LoRAs › More models fetches any of the seven; Engine › Compute chooses among the backbones here.
• yue2.cpp: pass a GGUF as the backbone (--model YuE2-3B-RS-FW-2.4x0.75-Q8_0.gguf), with the stock VAE.
• ComfyUI: the -BF16.safetensors above are in m-a-p's layout, which ComfyUI does not load. Take one from comfyui/ instead, put it in ComfyUI/models/checkpoints/, and use it wherever a workflow loads Comfy-Org's yue2_3b_bf16.safetensors (or its int8 ConvRot); SheetSage2 (audio_encoders/sheetsage2_bf16.safetensors) stays Comfy-Org's. Made the way Comfy-Org made theirs: the merge folded into their bf16 checkpoint, and int8 ConvRot by their recipe (their own bf16 quantized this way gives their int8 file back, every value). Start with 2.4x0.75, the safest.
• Transformers and other shells: put the config files and one safetensors in a folder, the safetensors named model.safetensors, and load the folder as you load m-a-p/YuE2-3B:

from huggingface_hub import hf_hub_download
import os, shutil
repo, merge, d = "goldhub/YuE2-RS-Finetunes", "YuE2-3B-RS-FW-2.4x0.75", "YuE2-RS-FW-2.4x0.75"
os.makedirs(d, exist_ok=True)
for f in ("config.json", "generation_config.json", "yue2_generation_config.json", "modeling_yue2.py", "qwen.tiktoken"):
    shutil.copy(hf_hub_download(repo, f), d)
shutil.copy(hf_hub_download(repo, f"{merge}-BF16.safetensors"), os.path.join(d, "model.safetensors"))
# then load d with trust_remote_code=True, as m-a-p/YuE2-3B

The First Wave: the data and the training

tracks	32,633 (about 1,800 hours)
where from	MTG-Jamendo 13,564 and the Free Music Archive 11,000, kept only above a production-quality bar (Audiobox Aesthetics, PQ ≥ 7.0); songs with lyrics from two community sets on the Hub, both made with Suno: humair025/suno-audio 5,000 and Kukedlc/suno-ai-music-dataset 789; a guard of sung and spoken material, so the model keeps what it already did well: Ruach Studio's own approved takes (962), GTSinger in English and Russian (662), and speech (600); a few duduk and shofar loops and songs of our own album
a score for every track	31,801 tracks have their own ABC score (SheetSage2), so the wave also teaches Full mode, where the model plans the song first and then sings it
style tags in YuE2's own order	language, genre, instruments, mood, key and BPM, plus emotion words read from the audio by LAION's Empathic Insight heads
lyrics	8,849 tracks sing words: from the sources, else heard by Whisper and checked against LRCLIB where it had them
training	LoRA on both halves of YuE2: the music half (the score and the music tokens) at rank 160, the sound half (the latents) at rank 320. Learning rate 5e-5, a moving average of the weights over a quarter epoch, four epochs (two planned, two more while the held-out curves still fell), a save every tenth of an epoch, three RTX 3090s, Ruach Studio's own trainer (ported from Ostris' AI-Toolkit)
held out	16 tracks never trained on. From epoch 0.1 to 4.0 the sound half's loss went from 0.873 to 0.846 and the music half's cross-entropy from 4.428 to 4.207, both falling to the end; the gap between training and held out stayed flat from epoch 2 on

Judged by ear, blind. Every save was played with fixed seeds at several strengths, numbered at random. Machine scores only filtered the data; they don't choose the release.

The vocabulary it learned

Every word in the style lines of the 32,633 training tracks, counted and sorted: genres, instruments, voice, mood, the emotion words read from the audio itself (LAION's Empathic Insight, 32 dimensions), keys and BPM, and the vocabulary of the 31,801 scores the music half learned to plan from (sections, keys, meters, tempi, chords). A word met in hundreds of tracks is a word the model understands; write your style in them, in YuE2's order.

• FW-VOCABULARY.md: the reference, with the most met words of each kind, where they came from and the words they came with
• fw-vocabulary.json: all of it, 4,695 words met in two tracks or more
• Search it on Ruach Studio's page, and build a style line by clicking words; in the studio it is in F1 → Creator → The First Wave's Vocabulary

YuE2 vs Others: the ring

Twenty cards, each a style line and lyrics (twelve language slots, from Russian and Ukrainian to Japanese, Mandarin and Greek), sent unchanged to each contender:

• SUNO v6: the first take of each pair, downloaded as WAV.
• musicful.ai V3.0: the first take of each pair, as WAV, at its defaults.
• Ours: stock YuE2-3B, and YuE2-RS-FW at epochs 0.7 and 1.5 (× 0.75). Two seeds a card, Full mode, raw from the engine: no mastering, no loudness matching.

160 takes, one yardstick for all. Lower is better for the error, the buzz, the stray tones, the clicks, the clipping, the noise floor and the sand; higher for the top and the roll-off; the rest is character, not score.

	SUNO v6 (20)	musicful V3 (20)	YuE2-3B stock (40)	RS-FW 0.7 (40)	RS-FW 1.5 (40)
Whisper error, % (words; ja/zh characters)	23.04	17.17	20.6	18.12	26.19
decoder 25 Hz buzz, dB	8.32	7.95	11.35	11.57	11.51
stray tones	2.2	1.9	4.25	1.95	2.3
clicks	33.4	33.8	27.7	31.9	34.12
clipping runs	0.0	0.0	15.7	16.38	16.3
noise floor, dBFS	-26.61	-25.15	-25.94	-25.86	-25.14
sand 6–16 kHz, dB	-15.7	-14.76	-17.58	-18.07	-17.37
top, kHz	17.89	15.32	15.77	14.8	14.96
roll-off, kHz	2.52	2.64	2.17	2.06	2.13
sub 20–60 Hz share, dB	-8.6	-10.22	-10.04	-9.58	-10.11
bass 60–150 Hz, dB	-5.97	-5.62	-4.34	-4.78	-4.58
presence 2–5 kHz, dB	-17.08	-16.6	-16.02	-16.28	-16.3
crest, dB	14.38	15.74	15.28	15.34	15.16
loudness spread p95−p10, dB	9.36	9.03	11.54	10.81	10.51
low band's spread, dB	23.35	20.04	21.92	21.12	20.77
stereo mid corr	0.67	0.69	0.75	0.77	0.73
LUFS	-15.47	-14.25	-12.62	-12.82	-12.64
LRA	5.94	6.17	7.1	6.55	6.48
true peak, dBTP	-3.38	-0.53	0.32	0.3	0.32
length, s	156.78	154.46	148.6	148.06	154.04

How we measured:

• the engine's own artifact check: the decoder's 25 Hz frame buzz in the highs, stray standing tones, clicks, clipping runs, stereo correlation by band, LUFS, LRA, true peak;
• the noise floor (the quietest tenth of 50 ms moments), the sand (6–16 kHz of those quiet moments against the whole), the top of the spectrum and its 95 % roll-off;
• the low end and the body: sub 20–60 Hz and bass 60–150 Hz shares, presence 2–5 kHz, crest, loudness spread;
• Whisper large-v3 in the card's language against the lyrics as sent: the word error, or the character error for Japanese and Chinese.

Read it with its limits. The closed services deliver mastered files and ours are raw from the engine, so the clipping runs and the true peak above zero are ours to fix in mastering, as Ruach Studio's Refiner does. Other seeds and other prompts sound different on every side. The ring shows the quality and the spread of sound between the contenders, not a final ranking. Its later columns, the merges above among them, will be added after the blind listening.

Credits and licence

• m-a-p for YuE2-3B and its VAE, CC BY-NC 4.0.
• The MTG-Jamendo dataset (Music Technology Group, Universitat Pompeu Fabra) and the Free Music Archive dataset, and every artist who released their music under Creative Commons.
• GTSinger (CC BY-NC-SA 4.0); humair025/suno-audio; Kukedlc/suno-ai-music-dataset.
• Serveurperso for yue2.cpp, its GGUF format, its LoRA merge and its quantizer; Ostris for AI-Toolkit; IronWolve for the YuE2 Kit that Ruach Studio grew from.
• Audiobox Aesthetics (Meta), SheetSage2, LAION's Empathic Insight, Whisper, LRCLIB.

The weights derive from YuE2-3B (CC BY-NC 4.0) and from data that includes GTSinger (CC BY-NC-SA 4.0). They are published under CC BY-NC-SA 4.0: non-commercial use, with credit as above, and anything made from them shared under the same terms. LICENSE is that licence; m-a-p's own licence and notices are in licenses/ and THIRD_PARTY_NOTICES.md, copied unchanged.