SoundVis

Frederic Leymarie and I have created a blog called SoundVis to document our research into the visualisation of sound and music. We’ll be adding our findings to it as time allows…

Canntaireachd for sinewaves

An early sketch of a system of vocables for describing manipulations of a sine wave.

The text is a bit small there, it’s better in the original avi version.

Vowels give pitch, and consonants give movements between pitches.

Inspired by the notation of canntaireachd. Made with hsc (Haskell client for scsynth). As ever, code available under GPL
on application.

I’m not sure where I’m going with this. It’s nice to describe a sound in this way but to use it in music the sound has to change over time otherwise it gets repetitive and therefore boring in many situations. I think I either have to develop ways of manipulating these strings programmatically, or ways of manipulating how they are interpreted. Both approaches would involve livecoding of course…

Speechless

A new project:
http://speechless.lurk.org/rhythm.html

The idea is to use festival speech synth to turn what people type into rhythms, giving them a simple multi-user interface for playing words together.

It needs flash 8 or later, but all of the interface is in good old HTML – I’m just using flash for the audio stream and server communication. The javascript and flash was made with the open source haXe language, so I didn’t have to install any dirty macromedia software on my computer.

Please play with it! All feedback very much appreciated. It’ll run until 14th April, after which I’ll release the sourcecode under the GPL for download, plus if anyone’s interested, a DVD containing the audio from the two weeks.

Relatedly, I was excited to find out about Canntaireachd, which is to bagpipes what bols are to tabla. I’m looking forward to getting my own articulatory synthesis working…

[update] This project is now finished, but I wrote a report on it.

Haskell supercollider tutorial

Rohan Drape has made a nice tutorial to getting his “Hsc” Haskell bindings to SuperCollider installed and integrated with emacs. It’s available here (link updated). This is exactly what I needed, I’m hoping to get started with some simple physical model synthesis this coming week.

20010203 (translated)

Peano weave applied to a slub classic for Ade‘s birthday..

Higher quality AVI available at slub.org

Programming in Haskell

Not really a review, just a strong recommendation… Graham Hutton’s Programming in Haskell is published mid January 2007, but Cambridge University Press are shipping already — I got mine just before Christmas and wish I had it earlier… It is by far the best introduction to Haskell I’ve seen, at least for someone new to functional programming such as myself. The chapters on parsing and and IO are a good mark of the book, together clearly yet stealthily introducing monadic programming in an easily digestible form. Well this book has plenty of other aspects I could praise, but like I say this isn’t a review, just go read and enjoy it yourself.

It’s great to read a really clear, concise text book, I could almost feel my brain re-organising itself while I read it. The experience reminded me of reading K & R after some months of confused C hacking, feeling everything clicking into place. That would be over ten years ago now, gah…


Peano curve weaves of whole songs

Some nine months ago I played with weaving images from music, including using a peano curve as a mapping.

I’ve returned to this subject, having many good ideas to explore from recent discussions with Tim Blackwell. We thought rendering some whole songs would work nicely. I didn’t fancy playing with my Java code again so wrote some Haskell, which I’m rather pleased with. The source is available (feedback welcome!). It does the the mapping using seeks on the output file, allowing impressive memory efficiency via Haskell’s lazy evaluation.

Some examples of some indie synth pop, disco, minimal techno (*3) and industrial gabba below, click on the images for the full versions but beware, they are rather large, around 5M each. Mouseover for the original track names.

Boy From School - Hot Chip At Last I Am Free - Chic Ping Pong - Plastikman Ping Pong - Plastikman (different curve)
Ping Pong - Plastikman (with some colours) Unborn Baby - Venetian Snares and Speedranch

BP2-like polymetric syntax

Another experiment with haskell, rather hastily screencasted for your pleasure:

It’s using haskell’s Parsec module to parse the syntax, and sending the sound events to supercollider for rendering.

This is a work in progress, but GPLd source available is on request, as is an AVI version if you don’t have flash. All feedback much appreciated.

Live programming

I thought there wasn’t enough context on this log, so here’s a brief history of my experiences with live programming.

So I’ve been writing music in the Perl language for some years now. For the first few years this involved hacking together text based curses interfaces. However inspired by the work of the SuperCollider and ChucK livecoders, as well as my musical collaborator Ade, I began writing and modifying code during performances. As such, the language is the only interface to the music.

A quick example:

Or download as a slightly easier to read avi.

After a couple of years though, it has become clear that Perl is not the ideal language for music. The interpreter itself is good for it, allowing me to reload bits of code in a slapdash manner, and the TMTOWTDI philosophy behind the language lends itself quite well to applications such as music, where *how* you express yourself is somehow important, as well as the end result. But while expressing a musical idea as a bunch of general purpose while loops, if statements and so on is certainly possible, it does not inspire musical thought and experimentation.

The end result is that when I improvise music with Perl in front of an audience, I either make lots of simple, enmeshed polymetric effects and polyrhythms, or call up and modify scripts I’ve composed under less pressured circumstances. Finding myself exploring a new idea during a performance was possible, but rare. However, according to Jeff Pressing, this is true of all human improvisation — through practice we build up processes for generating musical continuations and apply them, with rare changes, during an improvisation.

So, my library of Perl scripts *is* my musical technique. Any musical technique I have as an human (as an entity separate from my computer) is largely lost to me during a performance. If I have it, I don’t have time to express it while others are waiting to hear or dance to something.

The answer could be to switch to a language designed for music, such as SuperCollider or ChucK. Frederic Oloffson and Nick Collins have reported good results after making themselves practice livecoding from scratch with SuperCollider every day for a month.

What I’m intending to try though is making a language built around the kind of music I want to make, able to cope with programming under tight time constraints, allowing vague specification of sound events but well specified enough to allow other bits of software to reason within the language as well as myself.

More to follow…

Onomatopoeic synthesiser

In the afore-mentioned paper Rationalizing musical time: syntactic and symbolic-numeric approaches, Bernard Bel describes onomatopoeic notation for music, and then later a language for composing similarly structured music, the Bol Processor 2 (BP2). In BP2 however, the sound objects are represented by non-onomatopoeic symbols. That is, as far as the software is concerned, the particular words chosen as symbols for sound objects are of no consequence. Why?

What I’m asking here is really, why can’t I type “krrgrinnngngg!” or “poink?” and have the software synthesise a sound accordingly? Perhaps we should expect more of computers. We could ask someone with a guitar, trumpet or drum to make these sounds, and while they’d make quite different sounds to one another they would likely be interesting and with some identifiable relationship to the original written words.

I can imagine a few different approaches to synthesising onomatopoeic words. One would be to use (well, abuse) a speech synthesiser such as mbrola or festival. Another would be to take the approaches of speech synthesis but remove some constraints to open it up to producing a wider range of sounds. A third would be mapping parameters of an existing synthesiser to properties of a word. For example, how many consecutive vowel sounds the word has, whether the word begins with a hard or soft sound, or whether it ends with a question mark or an exclamation mark.

Anyway I’m still thinking about this, and there’s bound to be plenty of prior art… Please let me know if you know of any!