Okay, so, the wall that I'm hitting is that training an LLM on music theory, that understands the modern context, based on public domain sources is... well, challenging. The biggest problem, is not using copyrighted work directly.
So how do you get there?
Here's how you do it.
There's a dataset that uses the Musecore sources. The entire thing is public domain, lots of historic songs. Just this really big history book. It covers 20 genres, everything's already in a format that's clearly parsable by AI.
And that's great, but it doesn't give you anything that's really useful for composing modern pop songs. At least by itself.
The trick is to batch it out with a Python script to chat GPT's new reasoning model, over API. And what you want to tell it to do there is write a historical essay that includes historical references, influences, notation, possible re-imaginings of each song in contemporary genres with notation, and a discussion on how the song has influenced specific artists and genres that emerged later. List off notable covers by artists in history, and talk about the social impact of the piece, and why it's notable.
I'm also going to be combining this with the ChatMusician datasets, the same way.
Then, instead of training on the source dataset... train on the essays.
At the end of the day, what you end up with is a situation, where you have a LLAMA based AI that knows pretty much everything about modern music history, the evolution of genres, can make connections about how things fit together, maybe even in unexpected ways, that can create compositions, and have technical music discussions... without breaking anybody's copyright.