Synth Preset Auto-Chooser Idea

[Nunstummy]

I’ve complained about no standard for finding a synth preset.  Figure all Logic instruments = 6000+ presets.  Add Komplete 14 = 22,000 presets, maybe Arturia V Collection 9 = 14,000 presets, plus Vital, DeXEd, XT Surge, u-he Diva and whatever else you happen to own…. might total 100,000 synth/instrument presets! How are you supposed to find the sound you’re looking for?

I propose an app like Shazam, that listens to a sound, and tells you “that’s Bright Brass from an Oberhiem” and if you don’t own that specific virtual synth, it builds a similar sound using the Logic delivered  synths.  How about that?

[Atlas007]

Really interesting!

Indeed, with many thousands of synth sounds presets (which many sound close to one another) it's a daunting task to manage them efficiently and timely... If even humanely possible...

[des99]

It's not practical for a bunch of different reasons, but AI/ML starts to put the possibilities for implementing at least a subset of that dream.

Already Synplant 2 does a really great job of this, with it's own reasonably simple synth engine to use ML to recreate a source sample, but the training needed to achieve this - on a limited sample time, and with a reasonably simple synth engine - is pretty staggering. (Watch the interviews. The guy uses his training system to heat his brother's house in winter...)

To really use ML to understand how to to this on arbitrary third-party synths is a little too far fetched at the moment, but I could see in ten years time some standards being implemented that devs can sign onto to get some engines to understand and train on their engines.

[Nunstummy]

The Native Instruments NKS standard is well adopted, but the approach is to have all vendors comply with the same standard tagging descriptions, allowing you to search across many VSTs for the sounds and presets you want.  The problem is that many great VSTs don't want to comply with NKS - they feel it gives NI too much control and they're at the mercy of NI to expand the standards.

What I'm suggesting is more of a marketing levelling AI model that gives all plugin vendors (big or small) and equal playing field.

It does sound daunting, from a technology point-of-view.  It's a big data model that cross references audio with patch names.  Designed right, it could learn from user input and improve over time, as the big data model grows.

Example:  You want the patch that Duran Duran used in Hungry Like the Wolf for the arpeggiated intro.  The AI tool listens to the intro (or you upload just that audio bit) and it tells you it's a Roland Jupiter 8 with the 'Pluck' patch, and the arp set to random.

Then it looks at the Plugin Manager in Logic to see if you own any VSTs of a Jupiter 8, and shows that at the top of your results.  Then perhaps lists other vendor VSTs that are similar.

Am I dreaming too big?

 

[Nunstummy]

3 hours ago, des99 said:

Already Synplant 2

I'm looking into it!

[des99]

38 minutes ago, Nunstummy said:

What I'm suggesting is more of a marketing levelling AI model that gives all plugin vendors (big or small) and equal playing field.

Yes, I understood what you meant.

39 minutes ago, Nunstummy said:

It's a big data model that cross references audio with patch names.  Designed right, it could learn from user input and improve over time, as the big data model grows.

It's a *lot* more complicated than that. Ideas are easy - implementations, that work, are *hard*.

39 minutes ago, Nunstummy said:

Am I dreaming too big?

Yes, as of now.

However, we're in the early days of AI/ML and related technologies, and the growth potential is big - it will change our world for sure - so who knows what will be possible over the next couple of decades...

[Nunstummy]

3 hours ago, Nunstummy said:

Synplant 2

OK - I downloaded Synplant 2 and gave it a try after watching a good intro video.  I'd call this patch cloning.  I loaded sample from an 80's song that I know used the PPG Wave synth.  Synplant did a pretty good job of replicating the sound.  Impressive!  Rather that figue out which synth and preset was used, Synplant just expands upon the sample with a new preset.

Depending on how clean and isolated the sample is, I could have also created a quick Sampler instrument in Logic.

 

[Nunstummy]

2 hours ago, des99 said:

It's a *lot* more complicated than that. Ideas are easy - implementations, that work, are *hard*.

30 years in the software business, I had a knack for big ideas.  My role was a high level design and requirements, then pitching it to investors.  I create a pro-forma, touched on all the financial details - usually without a detailed estimate of the work.  Because no matter what our estimate was for time, money and complexity, investors wopuld just double the time and double the cost.

Since retiring from that world, it's touch to turn off the big ideas.

 

[Nunstummy]

Just a little more thought about the design.  It IS mutiple functions.

1.  Isolation of sounds.

Any attempt to extract the synth and/or patch of a sound will depend on how well isolated the parts are in the source.  I've used all the leading AI services to isolate parts, and most are designed to eliminate vocals (for Karaoke) or extract drum beats for hip hop.  When it comes to isolating the melodic parts, it's not too clean because frequencies overlap considerably.

2.  Finding the sound patch

I expect AI is better at 'finding a sound' the same way Google finds a place based on a picture, or apps that identity a tree based on a pic of a leaf.  Frankly, I'm not aware of technology that can find a sound.  It seems to me the entire history of recorded music would have to exist server side in some big data model.  You'd expect it to find multiple hits.  Ex. DX7 ePiano used in every hit song from 1984 - 1988.

3.  Finding the synth or instrument that created the sound

For that, you'd need some cross-reference of all the sounds and patches in every synth or sampled instrument.  That might be less effort than I think, as the patch lists of all synths and sampled instrument are usually documented in user manuals (pdf).  Problem is patch names are not unique on their own, but vendor+synth+patch name ARE unique.

4.  Last step is to compare the vendor+synth+patch to what VTSs you might have in Logic (or any DAW)

Still thinking.

Edited October 26, 2023 by Nunstummy
typos

[des99]

Sure, I can break this down into steps too - you still need to implement the steps 🙂, and some of the technology would just result in massive computer models. For example, if you want to know the audio solution space (ie, what sounds can a JP8 make) for a single synthesiser, like the JP8, it's essentially infinite - and that's just one synth. Ok, you don't have to have a complete training set, but you do need a representative one for the model to be useful, so you have to build a model that understands what sounds could possibly be made by which synths, what the overlaps are and so on. That's a lot of training, and with older analog synths that aren't able to be set up via MIDI, you have to tweak the controls *manually* millions of times unless you frankenstein the synth to be able to remote control all the parameters via software.

So even if you have a clean source sound to identify (which itself isn't trivial, but tools are getting better at this year on year), to be able to come up with a) a bunch of candidate synths that could *potentially* make that sound, along with the best matches, would just require a huge amount of model training, which is computationally expensive for what is, albeit a cool feature, a somewhat "frivolous" one.

Given I do quite a bit of sound archeology from classic records, I've got a good idea of what it takes to reverse engineer synth sounds, and it's hard, *especially* as most of the time, you're not even hearing the raw synth, but it's been processed, EQ'd, layered, chorused, delayed, reverbed, compressed - so to get back the initial synth sound, you have to "see through" that stuff as well. Like I say, it's a much harder task than it initially might seem, once you start to analyse the task.

I'm sure we will see more ML models being trained on audio - eg it's easier to identify the target frequencies of a piano to separate from a mix, when the model really understands what pianos sound like, whereas before it was by more dumb frequency/transient analysis etc - there are a lot of use cases for ML in audio, for sure. And Apple for sure are doing a lot of work in this area, and have the resources, some of which is already in Logic, but I expect to see more of this stuff being rolled out into user features over the coming years.

Edited October 26, 2023 by des99

[David Nahmani]

On 10/25/2023 at 7:14 AM, Nunstummy said:

I propose an app like Shazam, that listens to a sound, and tells you “that’s Bright Brass from an Oberhiem” and if you don’t own that specific virtual synth, it builds a similar sound using the Logic delivered  synths.  How about that?

That would really be incredible. 

Meanwhile I enjoy the brain exercise that comes with listening to a sound and coming up with the right synth to reproduce that sound, and programming the synth to sound like the sound you're hearing. 

I remember an old Sound On Sound article where a sound designer recommended that you first program the notes or the riff to be just like the melody you're hearing, same notes, same rhythm, same note lengths, tempo etc.. , then you focus on the way the sound is modulated (any tremolo, vibrato, envelope routed to filter etc....) and then finally in the very end adjust the timbre (source oscillators etc...) so I've generally been following that process and it's a great exercise, even if I'm not always able to perfectly reproduce the original sound obviously. 

But yes I can see how such a tool would be amazing, if only to study how sounds you like in songs you hear are programmed.