God this is interesting!
Okay, so here's what I was doing until 3am.
The initial prompt I came up with didn't work. It confused the machine, and produced bad compositions. That's fine. These things rarely work at the same time. Ran the prompt through both Claude and ChatGPT 1o, and discussed the purpose, testing methodology, and desired outcome with both of them.
And I let Claude and ChatGPT 1o both come up with working solutions to the problem. Claude's solution produced better musical discussion, but ChatGPT 1o's approach consistently produced better notation between both models. Interesting.
The current working prompt is screenshotted here.
The lengths of compositions that seems to work best between both models is three compositions per run. So that's the one I'm going to stick with.
Peer review between models
Once I have a transformed piece of music, the magic here really happens by using the peer-review or jury approach, and passing the new comp from one model to the other to review things like, how organic it sounds, and how interesting the compositions are musically.
We're grading on a couple of key criteria.
1. Technical foundations in music theory
2. How novel and thoughtful the transformed comps are
3. A subjective "fun" score between 1 and 6, graded against an understanding of a theoretical non-academic contemporary audience!
The critique also gives us another really interesting data point to train on that I hadn't considered.
So at the end of this process, we've got 4 key data points:
1. The original piece of public domain music.
2. A transformation document including three entirely new genre adaptations of the music, written like a virtuoso composer using modern genres, styles, and techniques, with performance notes and breakdown.
3. Analysis and critique of those pieces with revisions designed to spot blind spots, correct for unexpected or inorganic sounding choices, and explanation of reasoning as to WHY and how the composition needed to be edited.
4. Revised musical compositions with modifications that correct and build on the original reasoning for revision.
What the experiment taught me
So what did we learn?
1. I think I got a little carried away with the kinds of notation I want the machine to generate at the same time. This is a tall order any way we cut it, and we need to keep the kinds of notation we do a little more focused. Next time, we'll cut out the tablature and Nashville key notation, and just do that in a separate run.
2. Given that we've figured out what the grading system is, we need to adjust the initial prompt to match the grading criteria so that we can produce a more consistent deliverable.
3. Two synthetic reasoning machines peer reviewing each others work are better than one.
4. We still need a sanity check, so I'm going to be reaching out to all of my composer friends for human review of these outputs and assumptions. Incidentally, if you would like to help me do this, I could use all the musically literate sets of eyes I can get.
