I’ve been thinking about visual languages and the morphology of symbols (as opposed to words) for a while. I had the opportunity to start putting some of these ideas into code at a really excellent openframeworks workshop this week, run by Joel Gethin Lewis and Arturo Castro.
Here’s what it does:
Makes the point nicely that symbols and spaces can intertwine.
Using opencv blob detection, the regularity, direction and area of the shapes map to envelope modulation, resonance and pitch. The drawing is then sequenced into a melody using the minimum spanning tree (from the boost library) of the shape centroids, where distance maps to inter-onset interval.
It also has a mode for projecting the red circles and highlights back on the drawing surface which worked well.
This is only the second thing I’ve made with openframeworks, and while I don’t really get on with the codeblocks editor recommended for linux, I’m impressed with how accessible it makes opencv and all that.
Update: open frameworks sourcecode
Another update (1st August 2013): I ported this to Python, get the source here
This is quite charming.
But do they function as symbols? or geometric drawings?
I love the idea of drawing to make sound.
Seeing your experiment has got me thinking about a load of possibilities. Nice work.
Hey Graham, symbols are geometric drawings aren’t they? 🙂 But it’s not doing any grapheme detection or anything like that, the computer finds meaning in the shapes.
Yep Jamie I think this approach has a lot of promise, which means it’s probably all been done before, probably in the 50s, documented on a lot of fanfold paper in the attics of forgotten geniuses. Got anything in particular in mind?
Ha, lots of great graphical score stuff in the 1950s, like Earl Brown’s classic “December 1952”:
I love the casual way you manipulate the drawings Alex, the way a gesture’s meaning can change and you can connect previously disconnected shapes, distort the drawing, etc. Having the red balls bouncing around really opens up possibilities as well. You’ve probably seen this oldy but goody:
rad, dude. a nice concept. what happens with really dense drawings?
Thanks Evan.. In theory you should get intricate sequences with lots of warbling 303 glides, although as soon as two blobs a connected with a line they are treated as a single blob. In practice there are bugs and the more blobs there are, the more get missed out of the sequence for some reason…
Must return to this sometime soon
Nice Alex! I especially like how you’ve unlocked the hidden musical potential of the Happy Face. 🙂
I just begin OpenFrameworks and discovered this application. That seems great although I didn’t get to make this work. I’m a mac user and I think I have a problem with several libraries like boost or ofxCvMain.h or “ofxDirList.h or boost/lambda/lambda.hpp
(it isn’t included in your zip file?)
Many thanks for your help, maybe my question or difficulties are really easy but I don’t get everything.
I haven’t used openframeworks a great deal and don’t use macs, but for the missing ofx headers I’d suggest making a new openframeworks project, adding the opencv plugin and copying the code from my project across.
The boost library is not part of openframeworks and you’ll have to install that separately:
Hope that helps,