I think one of my best ideas was “vocable synthesis”, which my friend Lucy just reminded me of. I wrote a chapter on it in my doctoral thesis (chapter 3 – words), an ICMC paper in 2008 called Vocable Synthesis, and a later 2009 NIME paper called Words, Movement and Timbre. I got a kind of proof of concept working with a triangular waveguide mesh working but the results weren’t very musical. The ICMC paper has had zero citations (from people other than me), the NIME one got a bit more response but not much. But I still think it is a really exciting idea and think someone could have a lot of fun with it.
Basically the idea is that vocal tracts are extremely complicated, and words are a lovely, compact way to describe sounds in terms of articulations/movements, based on our tacit knowledge.
Drums (and other instruments) are less complicated than vocal tracts but could be controlled in the same way. The rich history of using non-lexical vocable syllables in music tradition (bol syllables, cantairreachd etc) point to this being a rich seam for interaction design in music. For practices like live coding, the possibilities of describing sounds with words is exciting in a lot of ways.
Why did this idea go nowhere? Maybe people are just too used to minute control over sound in sequencers/DAWs, and the idea of describing sound in rich, but vague ways doesn’t appeal? Indeed Canntaireachd is barely practiced since the sheet music has become dominant. I really have no interest in staff notation myself, but it has a lot of power, and seems to have taken over oral practices like Canntaireachd in Europe. Maybe this means that people can’t see the possibilities of notation based on speech?
Or maybe the idea is just a bit too easily misunderstood for people to pick up and run with. It’s a bit like onomatopoeia, but translates the idea from mapping from words to sounds via a vocal tract, to mapping from words to sounds via articulations of a drum and drum stick. So the aim isn’t to make drum sounds that sound like speech at all! But to make a way of making drum sounds that work in an analogous way to text-to-speech. Maybe people say ‘ah, this is onomatopoeia, so like those projects making pianos sound like speech”. That isn’t the idea at all.
Similar to the lack of interest in vocable synthesis, articulatory speech synthesis has not taken off beyond early research. It’s easier to do concatenative speech synthesis, and now ‘AI’-driven speech synthesis, rather than model the (again, extremely complex) vocal tract as a physical model. The lovely “pink trombone” software is a lot of fun to play with though, and people have used it in music performance, so maybe we aren’t far from a vocable synthesis breakthrough..
Or maybe it’s just a bad idea.
Here’s some proof-of-concept videos I made at the time anyway, that maybe didn’t really prove the concept..
https://www.youtube.com/watch?v=vreRpT_ov8Q
https://www.youtube.com/watch?v=77TPSrd-tuo
https://www.youtube.com/watch?v=j91d3aPM_jU
https://www.youtube.com/watch?v=034o1mfq-7Q