I wrote a paper with Dave Griffiths and Nick Collins on the visualisation of live code, exploring ideas around live coding interfaces, accepted for the EVA London 2010 conference in July. A HTML version is below, or see the PDF Preprint.
Alex McLean (Goldsmiths), Dave Griffiths (FoAM), Nick Collins (University of Sussex) and Geraint Wiggins (Goldsmiths)
In this paper we outline the issues surrounding live coding which is projected for an audience, and in this context, approaches to code visualisation. This includes natural language parsing techniques, using geometrical properties of space in language semantics, representation of execution flow in live coding environments, code as visual data and computer games as live coding environments. We will also touch on the unifying perceptual basis behind symbols, graphics, movement and sound.
I’ve been through a few linux distros over the years, neatly getting progressively easier to install and configure as I get less willing to spend time recompiling kernels, culminating in ubuntu, enjoying the attention to detail and simplicity of use. Recently though, I’ve had to give ubuntu up and go back upstream to the rather higher maintenance Debian again. Linux suffers from creeping featurism in its layers of audio APIs, it started with OSS, a straightforward API based on files, then came ALSA, a wildly complex API with broken documentation in a wiki you can’t edit, and an architecture that somehow means only one OSS application can write sound at a time. It seems to me that it’s a failing of ALSA that further layers of abstraction are piled on top of it, creating a rather complex landscape for sound hackers to navigate.
Ubuntu has joined in the fun by shipping with PulseAudio, which is probably great for general users but a pain for those needing to work with audio on a low level without using loads of CPU. Pulse is not straightforward to remove, and when I removed it had problems with volume controls not working, and the likelihood that future system upgrades wouldn’t work so well. That’s why I switched to debian sidux, but then I couldn’t get laptop hibernation, or my firewire sound card working, and had the stress of maintaining an unstable distribution.
However this week Puredyne carrot and coriander came out, and it’s really great. The kernel is optimised for realtime sound, and jack audio runs solidly without any drop outs, something I haven’t seen before. My firewire sound works reliably, better than I managed under ubuntu. It has a really nice logo and clean look, with no plump penguins in sight. It comes with all the best a/v software beautifully packaged, including all the live coding languages. The people behind it are super friendly and helpful. It’s downstream from ubuntu, so all the software is available. It’s a dream!
They make a big deal out of it being good for booting off a USB key, and I think have worked out some nice practicalities of working that way. This makes it great for doing workshops and running linux in a non-linux lab etc. It installs and works just as nicely on a permanent hard drive though, and that’s what I’ve done.
Anyway, heartily recommended, a dream come true, congratulations to all those involved.
I’m giving a paper at the CHArt conference in Birkbeck tomorrow. I’ll edit it a little after the conference for publication, but here’s a draft of the paper, here’s the presentation (which I’m currently editing) and here’s the abstract:
Programmers do their work by writing — a piece of software is a structure made from words. These structures are generally too big to comprehend in their entirety, so programmers instead focus on small detail and overall plans; zooming in to find parts to combine and simplify and zooming out to find places to build. But this is not architecture: these structures are more like machines than static buildings. A programmer’s work is set in motion by a program interpreter, with information flowing in and around processing units before being directed outward in response.
Usually a programmer will write some text, and then step back to start it up, watch it work and decide upon the next edit. Live coding programmers however work on their software while it is running, as if they were modifying a machine without switching it off first. Because software is built from words, this is done by editing it as text, adding new routines or changing the character of an existing one. Such a change takes immediate effect, allowing fast creative feedback.
Where a written novel exists to describe human activity, written software exists to simulate it. Therefore the live coder can take the role of an artist, constructing simulators in order to generate patterns of movement, either as music, video animation or both. This can be done in front of a live audience, so that the process of writing software becomes the process of improvising music or video in performance art.
Programmers are finally taking to the stage. Introspecting and encoding their musical thoughts before an audience. A tradition of live coding has quickly formed where computer screens are projected, making the programmer’s reactions to their work visible. Questions of authorship disappear; the performance is live, the programmer improvising through the medium of written language.
I’m still working on my upgrade, hoping to be a proper PhD student by the end of this month…
I’m excited to be organising an evening called transfer at Goldsmiths, London on Friday October 16th. It’ll be fantastic with all my favourite electronic music superheroes — Leafcutter John, Finn Peters, Yeeking and friends, Sarah Angliss as well as slub. Tickets now on sale!
Congratulations to those who made it to the end of the hackpact, sorry for breaking my part of the bargain but I really had to get this draft done, and couldn’t justify taking time away from my family without also taking time away from my nightly practice. Looking forward to spending time on other parts of my life this month, including a rather looser aim of recording and editing one track every week. If I manage it there’ll be an EP at the end of it.
third fourth week of hackpact actually started yesterday, but I didn’t think my contribution then warranted a new entry.
Bit of an error with the screencast, see if you can spot the problem. Pretty happy with the sound though. (will take a while to appear due to vimeo’s encoding queue)
I’m musically humbled by my son who has taught himself how to play the guitar and sing the blues. He’s two today (24th), happy birthday Harvey.
Well I made a blueberry birthday cake for young Harvey on the 24th, which is a hacklet at a push, photo when I find the transfer cable. No creations at all on the 25th or 26th, I was away for the weekend and thankfully didn’t get a moment with my laptop. I did record a session last night thought that I’ll upload at some point…
The hackpact has been really good for making me get bored with my software and develop it further. I need some longer term development time now though to play out some of my frustrations I’ve been feeling over the last month. In particular using a command line interface is feeling like a big limitation, I need to express relationships over more than one line. Maybe I can adapt the yi haskell editor for my needs.
Next month I think I’ll switch to making a fixed recording every week. I’ve never really made fixed recordings so should be interesting.
Now I’ve fallen off the hackpact wagon I’m not sure if I’ll be able to totally get back on, particularly as I need to finish my PhD upgrade report by the end of this month. We’ll see..
It’s the third week of the hackpact. A few have fallen by the wayside, others are doing impressively well. Adam is doing great learning supercollider, Sam is cracking away on a diverse range of ideas, Joe has put a lot of himself into his involved hacks, poetry with a smell of solder, Gabor pushing fluxus in wild new directions every day, Scott still soldiering on with ChucK every day, part of the inspiration for the hackpact and now part of it, and Cormac‘s easy sounding but in reality clearly really challenging rule based photography project that he’s tackling with increasing need for imagination. Honorary mention for Dan who threw himself at the project with some ace daily projects before going offline for a bit, hopefully he’ll rejoin us.
I hope I haven’t missed anyone, sorry if I have — let me know.
Another screencast with some good moments and also a couple of bugs found… I realised I forgot to switch off cpu scaling, so there might be more jumps in the recording.
Preparing for the haskell users group talk, so I tried to make some ultra quick demos of features of my pattern language. Failed really, I kept getting distracted at making the patterns sound better rather than demonstrate how they work, so I will likely just do a live demo instead.
The results are on youtube — I’m doing the presentation in google docs, and youtube is the only way of getting videos in there. In fact this is the only way I found of making a presentation with videos in under linux. Video support in openoffice sucks and didn’t work anyway, the html based s5 gets all slow and glitchy with videos in, the video library used in the python based bruce wouldn’t play any video I found, etc, etc…
Got through the haskell users group talk in one piece, happy with it actually. I managed to do some short sub-minute demos for it today which I think made it easier for people to follow, and I think count as today’s hackpact. Here’s the final presentation in full:
Uploaded my current Haskell stuff. Not very useful at all without instructions for how to install and use it which I’ll get to soon in a proper release, but some might be interested to read through the Pattern module at least.
Big shout out to Kassen, while he is behind on documenting his hackpact stuff he assures that he’s still doing daily hacks in the background. Hope to hear more from his side of our acidpact soon.
I’m pretty tired, going to try and make a new screencast tonight nonetheless.. Ah, here you are. Got rid of need for explicitly calling parser by using the overloaded string literals extension, thanks for the tip Ganesh.
Today’s screencast is here, although it’s not all that. No update tomorrow, we’re off to do some live coding at Plymouth Planetarium, documentation will hopefully surface eventually.
Well the planetarium experiment went well, hopefully will turn into a tour. Some documentation to appear sometime soon.
Too tired to make music, so worked on a poster design. Ended up too gloomy, to rework.
[[blue, blue [lightblue lightskyblue lightblue, lightblue lightskyblue lightblue ~] blue, blue], blue lightblue ~, blue]
I started my hackpact month last night with this screencast, playing around with time offsets and functors. I think the audio gets *slightly* ahead of the video, probably due to some jack-audio drops. If for some reason you want a better quality version you can look at the source mpeg-4 file on blip.tv.
I was happy with applying a sine wave to the time offsets, giving a bit of swing to the rhythm. Combining that with a straight (pure 0.0) pattern grounded it nicely I think, so the rhythm was played straight on top of shifting it forward and backward in time. (although this is live I can play sounds slightly in the past due to a 0.2 second system-wide artificial latency). I also played with a simple chorus effect by having lots of offsets on top of one another amongst other stuff.
Not sure I like the end result of this as music or not, I’ll listen with fresh ears before I decide on that. But then I think the point of my hackpact is to practice live coding rather than doing great music so that doesn’t really matter.
For this one I used nekobee, it took quite a while to get haskell talking to it over DSSI (a nice way to talk to synths over OSC, but it turns out it uses a vaguely non-standard tag to send MIDI bytes). I quite like the results although there’s much to tweak and I didn’t get the levels quite right.
Baas and bleeps… Trying to focus on the music rather than the language for this one. UPDATE: I screwed up the panning, and have updated the below with a mono version. The original stereo version is over here might be better quality due to the transcoding but the panning is annoying.[Rather than choke up my loyal reader’s RSS feed I’ll add further day’s hacks here on this post rather than make new ones, unless I do something major (like actually release some software).]
Oops, forgot to update the blog with this yesterday. Here’s some minimal acid from the 4th.hackpact5
Getting harder to make myself make a screencast but I’m enjoying it more and more. Tried going extra slow for this one. Switched to vimeo hosting due to transcoding problems.
Tried going extra fast after the previous extra slow offering. Some nice bits towards the start but didn’t manage to keep it up really.
A low point today, have some pattern visualisation stuff I wanted to get done but didn’t manage to get it working tonight. Polar coordinates too much for my tired brain. I did manage to make a start on slides for my haskell user group talk next week though, very much a work in progress but any feedback much appreciated. Not a hack but bah.
I’m going to do a live a/v stream from my sofa 10pm GMT this Saturday 13th December ’08, livecoding with Perl and hopefully also a little language parsed with Haskell. You can find info about how to watch, listen to the stream and join the chat over on the toplap site.
I did something similar last weekend, a remote performance to the Piksel festival in Norway, and I enjoyed it so much I had to repeat it. Hopefully it’ll become a regular thing, yeeking has already offered to do the next one.
I’m doing the streaming with gstreamer, I don’t know if it’s possible to do live screencasts in this way with anything else and it offers a huge amount of control. I reached the limits of gst-launch so have written a little gstreamer app to use for this weekend. I’ll be releasing that soon…
Another thing – it’s the xmas dorkboteastlondon tomorrow (thurs) and one of our best line-ups ever. Unmissable if you’re in around…
Two posts rolled in to one, to annoy the aggregators a bit less (sorry haskellers, more haskell stuff soon).
First, dorkcamp is a lovely event in its third year. The idea is for around 60 of us to go to a campsite an hour out of London, well equipped with showers, toilets, a big kitchen and hall, and do fun dorky stuff like soldering and knitting. It happens at the end of August, tickets are running low so grab yours now. More info on the website and wiki.
Second here’s a new demo, this time with two drum simulations, one high and one low:
This entertaining article supporting test-first development has been playing on my mind. The article is beautifully written so it is easy to see the assumed context of working to deadline on well specified problems, most probably in a commercial environment. It saddens me though that we accept this implicit context across all discussion of software development practice all too easily.
Here’s a nice illustration from the article, which appears under the heading “Prevent imagination overrun”.
So there is a fairly clear reason not to write any tests for your code — you will take in more of the problem domain without such directive constraints. What you are left with will be the result of many varied transformations, and be richer as a result. You might argue that this is undesirable if you are coding a stock control system to a tight deadline. If you instead take the example of writing some code to generate a piece of music, then you should see my point. The implicit commercial context does not apply when you are representing artistic rather than business processes as code.
In fact this notional straight line is impossible in many creative tasks — there is no definable end goal to head towards. A musician is often compelled to begin composing by the spark of a musical idea, but after many iterations that idea may be absent from the end result. If they are scoring their piece using a programming language, then there would be no use in formalising this inspirational spark in the form of a test, even if it were even possible to do so.
What this boils down to is the difference between programming to a design, and design while programming. Code is a creative medium for me, and the code is where I want my hands to be while I am making the hundreds of creative decisions that go into making something new. That is, I want to define the problem while I am working on it.
While “end user programming” in artistic domains such as video and music becomes more commonplace and widely understood, then perhaps we will see more discussion about non-goal driven development. After all artist-programmers are to some extent forced to reflect upon their creative processes, in order to externalise them as computer programs. Perhaps this gives a rare opportunity for the magic of creative processes to be gazed upon and shared, rather than jealously guarded for fear that it may escape.