Category: texture

Coding with knots

My first Quipu attempt

Being inspired by Quipu, and also Dave and Julian‘s work on visualising and sonifying Quipu data (their “coding with knots” paper is here, will link to preprint soon..), I thought I’d have a go at making some. Quipu seem to be ancient databases, used to archive and communicate information. They were in use by Andean people over hundreds of years. The goal is to encode Tidal patterns in a similar manner, which I think was also Dave’s idea. So far, I think this should work very well.. Some initial (and of course, naive) thoughts about trying to make Quipu are below.

Representation of a real Quipu (Meyers Konversations-Lexikon, 1888)

I started with four-ply 100% cotton handknit yarn, which was less fluffy and easier to work with than sheep’s wool. It did cross my mind to try to find Alpaca wool for authenticity, but it was not very easy to find, and I read that cotton was used in the Quipu too. After a bit of knotting things weren’t going too well, due to the stretchiness of the yarn it was difficult to get the knots close together, and it was all a bit fiddly.. Especially for the ‘end knot’ in a number which requires you to pass the yarn through its own loop up to nine times, securing itself in neat spiral. I wondered whether this yarn had the right structure for the task..

Looking closer at the diagrams and photographs of Quipu on-line though, it became clear that the Quipu yarn was ‘doubled’ – folded in half and twisted together to create a loop at one end, which could then be used to attach to another piece of yarn with a ‘cow hitch’. I found I could double small pieces of wool quite easily by hand, by twisting it with the direction of the existing twist until I met resistance, folding the yarn in half, and letting it twist together. Happily, the result was a yarn which was not only easy to hitch (and un-hitch) from another piece, but also was much easier to work with, in terms of tying the knots and getting them to sit in the right place, with a little help from a 20mm crochet hook.

It seemed important to start knotting at the bottom of the doubled yarn, otherwise it would start to unravel back to its original 4-ply. In terms of the Quipu representation of natural numbers (which handily, is base ten), this meant starting by knotting the unit, and then the tens, and so on. Once attached, the threads were easy to move along the thread they were attached to, and un-hitch if necessary.

I’ll need a lot more practice to make the knots more ‘readable’. Yarn is twisted in a particular direction – in this case the ‘Z’ rather than ‘S’ direction, so perhaps the knots have to match in terms of whether they’re right or left handed. There is a suggestion that the direction of twist and handedness of the knot is significant in terms of information storage, I’m a bit sceptical about this, as least they appear to be dependent on one another. For now I’m putting a little bit of twist into the yarn as I tie the knot to make sure the ply doesn’t ‘open up’.

Anyway, I think I have some grasp of how this can work now. The next task is to try to notate Tidal patterns. The ability to hitch threads to the side of others should make knotting together the parse tree of a TidalCycles pattern fairly straightforward. I’d also like to use beads to represent things which aren’t directly numerical, such as sound samples — Hama beads look perfect for this, the right size, available in bulk in a wide range of colours, and cheap! Hopefully I’ll have something to show at my lunchtime talk at the ODI this Friday.

PENELOPE

I’ve just realised that I haven’t posted here about my new job. I have left my post as research/teaching fellow in the University of Leeds, and since February 2017 have started work for a Museum, in particular the Research Institute of the Deutsches Museum, an incredible science museum in Munich — although I am still based in Sheffield UK. I’ll be working part time over the next five years on the PENELOPE research project lead by Ellen Harlizius-Klück, following our previous project Weaving Codes, Coding Weaves.

“Our aim is to integrate ancient weaving into the history of science and technology, especially digital technology. The project encompasses the investigation of ancient sources as well as practices and technological principles of ancient weaving. We set up a PENELOPEan laboratory where we detect the models and topologies of weaves and develop codes to make them virtually explorable.”
It’s a great privilege to have this huge chunk of time to really get to the bottom of something, an experience I haven’t had since my PhD. The project has much deeper connection to the world of live coding than it might first appear, being all about computational pattern, and the sharing of thought — but taking a much longer view of live coding than is usual. You can read more on the project website, including a brief exploration of making music from ‘tabby’ weaves.

Colourful texture

Texture v.2 is getting interesting now, reminds me of fabric travelling around a loom..

Everything apart from the DSP is implemented in Haskell. The functional approach has worked out particularly well for this visualisation — because musical patterns are represented as functions from time to events (using my Tidal EDSL), it’s trivial to get at future events across the graph of combinators. Still much more to do though.

Texture 2.0 bug exposure

Texture 2.0 (my Haskell based visual live programming language) is working a bit more. It has reached gabber zero – the point at which a programming language is able to support the production of live techno. Also I’ve made some small steps towards getting some of my live visualisation ideas working. Here’s a video which exposes some nice bugs towards the end:

This is an unsupported, very pre-alpha experiment, but if you want to try to get it working, first install Tidal (and if you want sound, the associated “dirt” sampler). Then download the code from here:

https://github.com/yaxu/hstexture

.. and run it with something like runhaskell Main.hs

 

PhD Thesis: Artist-Programmers and Programming Languages for the Arts

With some minor corrections done, my thesis is finally off to the printers.  I’ve made a PDF available, and here’s the abstract:

We consider the artist-programmer, who creates work through its description as source code. The artist-programmer grandstands computer language, giving unique vantage over human-computer interaction in a creative context. We focus on the human in this relationship, noting that humans use an amalgam of language and gesture to express themselves. Accordingly we expose the deep relationship between computer languages and continuous expression, examining how these realms may support one another, and how the artist-programmer may fully engage with both.

Our argument takes us up through layers of representation, starting with symbols, then words, language and notation, to consider the role that these representations may play in human creativity. We form a cross-disciplinary perspective from psychology, computer science, linguistics, human-computer interaction, computational creativity, music technology and the arts.

We develop and demonstrate the potential of this view to inform arts practice, through the practical introduction of software prototypes, artworks, programming languages and improvised performances. In particular, we introduce works which demonstrate the role of perception in symbolic semantics, embed the representation of time in programming language, include visuospatial arrangement in syntax, and embed the activity of programming in the improvisation and experience of art.

Feedback is very welcome!

BibTeX record:

@phdthesis{McLean2011,
    title = {{Artist-Programmers} and Programming Languages for the Arts},
    author = {McLean, Alex},
    month = {October},
    year = {2011},
    school = {Department of Computing, Goldsmiths, University of London}
}

RIS record:

TY  - THES
ID  - McLean2011
TI  - Artist-Programmers and Programming Languages for the Arts
PB  - Department of Computing, Goldsmiths, University of London
AU  - McLean, Alex
PY  - 2011/10/01

Workshop output

The Text live coding workshop went really well, surprisingly well considering it was the first time anyone apart from me had used it and (so I found out after) most of the participants didn’t have any programming experience. The six participants took to the various combinators surprisingly quickly, the main stumbling block being getting the functions to connect in the right way… Some UI work to do there, and I got some valuable feedback on it.

Once the participants had got the hang of things on headphones, we all switched to speakers and the seven of us played acid techno for an hour or so together, in perfect time sync thanks to netclock. Here’s a mobile phone snippet:

The sound quality doesn’t capture it there, but for me things got really interesting musically, and it was fun walking around the room panning between the seven players…

Text update and source

I’ve updated Text a bit to improve the visual representation of higher order types (you’d probably need to full screen to view):

I won’t be touching this until after the workshop on Saturday.

I’ve also made the source for the visual interface available here under the GPLv3 free license. To get it actually working as above you’d also need to install my tidal library, Jamie Forth’s network sync, my sampler, the nekobee synth, and somehow get it all working together. In short, it’s a bit tricky, I’ll be working on packaging soonish though.

Test run of Text

I’ve been rather busy writing lately, my PhD funding runs out in April, and I hope by then I’ll have finished and will be looking for things to do next.

I have had a bit of time to make Text, a visual language I mentioned earlier, a bit more stable, here’s a test run:

A bit of a struggle, partly due to the small screen area I gave myself for the grab, but also due to some UI design issues I need to sort out before my workshop at Access Space in Sheffield next week, on the 5th February. Access Space is a really nice free media lab, but will turn nasty unless I free the workshop software, so expect a release soon.

In case someone is interested, here’s the linux commandline I use to record a screencast with audio from jackd:


gst-launch-0.10 avimux name=mux \
! filesink location=cast.avi \
ximagesrc name=videosource use-damage=false endx=640 endy=480 \
! video/x-raw-rgb,framerate=10/1 \
! videorate \
! ffmpegcolorspace \
! videoscale method=1 \
! video/x-raw-yuv,width=640,height=480,framerate=10/1 \
! queue \
! mux. \
jackaudiosrc connect=0 name=audiosource \
! audio/x-raw-float,rate=44100,channels=2,depth=16 \
! audioconvert \
! queue \
! mux.

Text

Text is a experimental visual language under development.  Code and docs will appear here at some point, but all I have for now is this video of a proof of concept.

It’s basically Haskell but with syntax based on proximity in 2D space, rather than adjacency.  Type compatible things connect automatically, made possible though Haskell’s strong types and currying.  I implemented the interface in C, using clutter, and ended up implementing a lot of Haskell’s type system.  Whenever something changes it compiles the graph into Haskell code, which gets piped to ghci.  The different colours are the different types.  Stripes are curried function parameters.  Lots more to do, but I think this could be a really useful system for live performance.

New text editor

I stopped using feedback.pl, a self-modifying live code editor written in Perl, some time ago, in favour of editing Haskell in emacs.  It’s about time I made a specialised editor for the pattern stuff I’ve been doing with Haskell.  Actually I have been dreaming of a visual programming language editor for some time, which for the moment I’m just calling `Text’.  Here’s a screen shot of its current state:

I’ve noticed that Visual Programming Languages like Max and PD are still based around text, using e.g. distance as secondary notation, not actually part of the language syntax.  In Text, words that are closest together are automatically connected if their types are compatible.  You can see that I haven’t quite got the ‘closest together’ algorithm nailed yet, I’ve no idea why its decided to parse this as three separate, overlapping statements.  However I have got currying working, which I’m happy with.  At the moment it’s written in C using the clutter library, which I’m finding fun.  Anyway just a quick update, I’ve got a lot more ideas to implement before I know whether this idea has legs or not…