Category: texture
Research products
I’ve been enjoying the idea of “research products” as opposed to “research prototypes”. Prototypes are understood as a partially working thing as a step towards an answer to a design problem. Research products on the other hand are understood as they are, rather than what they might become. Here’s how Odom et al describe it in their 2016 CHI paper “From Research Prototype to Research Product”. Unfortunately this is a closed access ACM paper, but you can find a pdf online, for now at least. Here’s the four features of research products that they highlight:
- Inquiry driven: a research product aims to drive a research inquiry through the making and experience of a design artifact. Research products are designed to ask particular research questions about potential alternative futures. In this way, they embody theoretical stances on a design issue or set of issues.
- Finish: a research product is designed such that the nature of the engagement that people have with it is predicated on what it is as opposed to what it might become. It emphasizes the actuality of the design artifact. This quality of finish is bound to the artifact’s resolution and clarity in terms of its design and subsequent perception in use.
- Fit: the aim of a research product is to be lived-with and experienced in an everyday fashion over time. Under these conditions, the nuanced dimensions of human experience can emerge. In our cases, we leveraged fit to investigate research questions related to human-technology relations, everyday practices, and temporality. Fit requires the artifact to balance the delicate threshold between being neither too familiar nor too strange.
- Independent: a research product operates effectively when it is freely deployable in the field for an extended duration. This means that from technical, material, and design perspectives an artifact can be lived with for a long duration in everyday conditions without the intervention of a researcher.
I’m finding this helpful thinking about my live loom. It’s not intended as a commercially viable product, but it’s also not intended as a step towards one. It’s intended to be a device for exploring computation, without automation and all its forced simplicity. It works very well, every time I use it I’m blown away by the generative complexities of handweaving, and it helps me see computer programming language design afresh, with a beginner’s mind. So it’s inquiry driven, and finished in that it’s ready to embody an area of inquiry and host exploration of that. In terms of fit – well its lasercut body and trailing arduino aligns it with 21st century maker culture, and solenoids align it with 20th century electromechanics, but its fundamental design is that of an ancient warp weighted loom, so it has some fit there although it has a lot to learn from the past in terms of ergonomics.
In terms of ‘independence’ it’s not quite there yet, but is designed with open hardware principles, using easy to source parts and permissive CC licensed designs. The next step is supporting others in replicating the hardware which will happen in the next few months. This is where it gets exciting for me – how will the live loom function as an ‘epistemic tool’ – will the research ideas carry with the loom, or will the replicators ‘misunderstand’ the loom and take it in a new direction? Of course the latter case would be failure in one respect, but I get the impression that designers see such failure as positive, where objects support divergent use..
In any case by thinking about the live loom as a research product, it helps me explain what it’s for. When I show it to people, they often treat it as a work-in-progress towards a fully automated loom, like one driven by the famous Jacquard mechanism. That’s the opposite of what I’m trying to do, as that mechanism is what separates humans from the mathematical basis of weaving as computational interference. As a research product, the live loom foregrounds computational augmentation rather than automation.
Research papers as research products
This leads me to think about research papers as research products too – many will have the experience of publishing a research paper, getting excited when someone has cited it, only to find that they’ve totally misunderstood what you were trying to say, even taking the opposite meaning. What if we treated papers as research products, that we deploy in the world, and then observe what they do? I just read Christopher Alexander’s foreword to Richard Gabriel’s book “Patterns of software”. Alexander is an architect (of buildings), and Gabriel is a computer scientist who has studied Alexander’s work for decades in order to try to develop a similar pattern-based approach in software. What’s interesting is that Alexander seems profoundly disappointed in the book that he’s writing a foreword for, although he’s chooses his words generously he basically asks Gabriel to write a different book, and to learn from his more recent work where he solves all the problems in his older work that Gabriel references. It is amazing that Gabriel would host such a text at the front of his book! Really Richard Gabriel is an amazing computer scientist and thinker, and I think Alexander is being a bit naive in assuming that such a comparatively young field of computer science could solve its core problems by going through his four-volume text on designing physical buildings – these are really very different domains indeed. What is more interesting is that Gabriel gives voice to the person he cites. This goes way beyond peer review to giving his text its own life in the process of being published. I’m looking forward to the rest of the book!
Coding with knots
Being inspired by Quipu, and also Dave and Julian‘s work on visualising and sonifying Quipu data (their “coding with knots” paper is here, will link to preprint soon..), I thought I’d have a go at making some. Quipu seem to be ancient databases, used to archive and communicate information. They were in use by Andean people over hundreds of years. The goal is to encode Tidal patterns in a similar manner, which I think was also Dave’s idea. So far, I think this should work very well.. Some initial (and of course, naive) thoughts about trying to make Quipu are below.
I started with four-ply 100% cotton handknit yarn, which was less fluffy and easier to work with than sheep’s wool. It did cross my mind to try to find Alpaca wool for authenticity, but it was not very easy to find, and I read that cotton was used in the Quipu too. After a bit of knotting things weren’t going too well, due to the stretchiness of the yarn it was difficult to get the knots close together, and it was all a bit fiddly.. Especially for the ‘end knot’ in a number which requires you to pass the yarn through its own loop up to nine times, securing itself in neat spiral. I wondered whether this yarn had the right structure for the task..
Looking closer at the diagrams and photographs of Quipu on-line though, it became clear that the Quipu yarn was ‘doubled’ – folded in half and twisted together to create a loop at one end, which could then be used to attach to another piece of yarn with a ‘cow hitch’. I found I could double small pieces of wool quite easily by hand, by twisting it with the direction of the existing twist until I met resistance, folding the yarn in half, and letting it twist together. Happily, the result was a yarn which was not only easy to hitch (and un-hitch) from another piece, but also was much easier to work with, in terms of tying the knots and getting them to sit in the right place, with a little help from a 20mm crochet hook.
It seemed important to start knotting at the bottom of the doubled yarn, otherwise it would start to unravel back to its original 4-ply. In terms of the Quipu representation of natural numbers (which handily, is base ten), this meant starting by knotting the unit, and then the tens, and so on. Once attached, the threads were easy to move along the thread they were attached to, and un-hitch if necessary.
I’ll need a lot more practice to make the knots more ‘readable’. Yarn is twisted in a particular direction – in this case the ‘Z’ rather than ‘S’ direction, so perhaps the knots have to match in terms of whether they’re right or left handed. There is a suggestion that the direction of twist and handedness of the knot is significant in terms of information storage, I’m a bit sceptical about this, as least they appear to be dependent on one another. For now I’m putting a little bit of twist into the yarn as I tie the knot to make sure the ply doesn’t ‘open up’.
Anyway, I think I have some grasp of how this can work now. The next task is to try to notate Tidal patterns. The ability to hitch threads to the side of others should make knotting together the parse tree of a TidalCycles pattern fairly straightforward. I’d also like to use beads to represent things which aren’t directly numerical, such as sound samples — Hama beads look perfect for this, the right size, available in bulk in a wide range of colours, and cheap! Hopefully I’ll have something to show at my lunchtime talk at the ODI this Friday.
*Update* I did indeed manage to transliterate some Tidal code into knots and beads, in particular taking this by Kindohm:
jux (|*| speed "1.5") $ foldEvery [3,4] (0.25 <~) $ stack [
s "less:2([3 5]/2,8)" # unit "c" # speed "4",
s "less(3,8)" # cut "1" # up "{0 -3 1}%32",
s "{~ [~ cp] ~ less:3/2 ~}%4",
s "[less:1*2 less:1]*2"]
I created this:
I’ll post again with more explanatory detail, but from my notes..
- Knots are numbers, including a slightly awkward attempt at representing rational numbers
- Values which are not directly numerical are represented with beads. Samples are yellow for cp and purple for feel (the sample number is a number, so is a knot). As an exception, single letters of the alphabet are treated as ordinal numbers and so represented with knots, e.g. c = 3.
- Function names are a bead followed by a brown bead. In particular, red brown for jux, pink brown for foldEvery, blue brown for ~>
- Subpatterns and functions-as-parameters are separate strings tied on to the side (currying works out fine)
- Parameter names are a bead followed by a black bead. red black for sound, green black for cut, blue black for up, pink black for speed.
The parse tree comes out rather nicely I think, and is almost a complete representation of the original pattern (I decided to overlook the difference between # and |*| for this first attempt). I’m trying to minimise the use of beads, treating colours as logograms feels like cheating, simply making a text out of an alphabet of colour. The branching structure is what takes it away from one dimensional text though, and into something which is much closer to that of a running computer program (particularly a pure functional one like a Tidal pattern).
The next step is to try to compose a new pattern directly as string, to explore its affordances.. Maybe this could inspire a breakthrough in usability for my Texture interface for Tidal..
PENELOPE
I’ve just realised that I haven’t posted here about my new job. I have left my post as research/teaching fellow in the University of Leeds, and since February 2017 have started work for a Museum, in particular the Research Institute of the Deutsches Museum, an incredible science museum in Munich — although I am still based in Sheffield UK. I’ll be working part time over the next five years on the PENELOPE research project lead by Ellen Harlizius-Klück, following our previous project Weaving Codes, Coding Weaves.
Colourful texture
Texture v.2 is getting interesting now, reminds me of fabric travelling around a loom..
Everything apart from the DSP is implemented in Haskell. The functional approach has worked out particularly well for this visualisation — because musical patterns are represented as functions from time to events (using my Tidal EDSL), it’s trivial to get at future events across the graph of combinators. Still much more to do though.
Texture 2.0 bug exposure
Texture 2.0 (my Haskell based visual live programming language) is working a bit more. It has reached gabber zero – the point at which a programming language is able to support the production of live techno. Also I’ve made some small steps towards getting some of my live visualisation ideas working. Here’s a video which exposes some nice bugs towards the end:
This is an unsupported, very pre-alpha experiment, but if you want to try to get it working, first install Tidal (and if you want sound, the associated “dirt” sampler). Then download the code from here:
https://github.com/yaxu/hstexture
.. and run it with something like runhaskell Main.hs
PhD Thesis: Artist-Programmers and Programming Languages for the Arts
With some minor corrections done, my thesis is finally off to the printers. I’ve made a PDF available, and here’s the abstract:
We consider the artist-programmer, who creates work through its description as source code. The artist-programmer grandstands computer language, giving unique vantage over human-computer interaction in a creative context. We focus on the human in this relationship, noting that humans use an amalgam of language and gesture to express themselves. Accordingly we expose the deep relationship between computer languages and continuous expression, examining how these realms may support one another, and how the artist-programmer may fully engage with both.
Our argument takes us up through layers of representation, starting with symbols, then words, language and notation, to consider the role that these representations may play in human creativity. We form a cross-disciplinary perspective from psychology, computer science, linguistics, human-computer interaction, computational creativity, music technology and the arts.
We develop and demonstrate the potential of this view to inform arts practice, through the practical introduction of software prototypes, artworks, programming languages and improvised performances. In particular, we introduce works which demonstrate the role of perception in symbolic semantics, embed the representation of time in programming language, include visuospatial arrangement in syntax, and embed the activity of programming in the improvisation and experience of art.
Feedback is very welcome!
BibTeX record:
@phdthesis{McLean2011,
title = {{Artist-Programmers} and Programming Languages for the Arts},
author = {McLean, Alex},
month = {October},
year = {2011},
school = {Department of Computing, Goldsmiths, University of London}
}
RIS record:
TY - THES ID - McLean2011 TI - Artist-Programmers and Programming Languages for the Arts PB - Department of Computing, Goldsmiths, University of London AU - McLean, Alex PY - 2011/10/01
Workshop output
The Text live coding workshop went really well, surprisingly well considering it was the first time anyone apart from me had used it and (so I found out after) most of the participants didn’t have any programming experience. The six participants took to the various combinators surprisingly quickly, the main stumbling block being getting the functions to connect in the right way… Some UI work to do there, and I got some valuable feedback on it.
Once the participants had got the hang of things on headphones, we all switched to speakers and the seven of us played acid techno for an hour or so together, in perfect time sync thanks to netclock. Here’s a mobile phone snippet:
The sound quality doesn’t capture it there, but for me things got really interesting musically, and it was fun walking around the room panning between the seven players…
Text update and source
I’ve updated Text a bit to improve the visual representation of higher order types (you’d probably need to full screen to view):
I won’t be touching this until after the workshop on Saturday.
I’ve also made the source for the visual interface available here under the GPLv3 free license. To get it actually working as above you’d also need to install my tidal library, Jamie Forth’s network sync, my sampler, the nekobee synth, and somehow get it all working together. In short, it’s a bit tricky, I’ll be working on packaging soonish though.
Test run of Text
I’ve been rather busy writing lately, my PhD funding runs out in April, and I hope by then I’ll have finished and will be looking for things to do next.
I have had a bit of time to make Text, a visual language I mentioned earlier, a bit more stable, here’s a test run:
A bit of a struggle, partly due to the small screen area I gave myself for the grab, but also due to some UI design issues I need to sort out before my workshop at Access Space in Sheffield next week, on the 5th February. Access Space is a really nice free media lab, but will turn nasty unless I free the workshop software, so expect a release soon.
In case someone is interested, here’s the linux commandline I use to record a screencast with audio from jackd:
gst-launch-0.10 avimux name=mux \
! filesink location=cast.avi \
ximagesrc name=videosource use-damage=false endx=640 endy=480 \
! video/x-raw-rgb,framerate=10/1 \
! videorate \
! ffmpegcolorspace \
! videoscale method=1 \
! video/x-raw-yuv,width=640,height=480,framerate=10/1 \
! queue \
! mux. \
jackaudiosrc connect=0 name=audiosource \
! audio/x-raw-float,rate=44100,channels=2,depth=16 \
! audioconvert \
! queue \
! mux.
Text
Text is a experimental visual language under development. Code and docs will appear here at some point, but all I have for now is this video of a proof of concept.
It’s basically Haskell but with syntax based on proximity in 2D space, rather than adjacency. Type compatible things connect automatically, made possible though Haskell’s strong types and currying. I implemented the interface in C, using clutter, and ended up implementing a lot of Haskell’s type system. Whenever something changes it compiles the graph into Haskell code, which gets piped to ghci. The different colours are the different types. Stripes are curried function parameters. Lots more to do, but I think this could be a really useful system for live performance.