I collaborated with fellow Music Technology MFA Jason Jahnke for my piece AutoLyrics in the Automaton concert I directed. While originally we planned for a multi-faceted combination of various audio and visual projects Jason was working on, time eventually condensed our work to getting the first operational code onto a new instrument he had literally just finished fabricating and assembling hours earlier called the AquaHarp II.
The AquaHarp was a glass-striking instrument that ran code behind the scenes which essentially randomly struck each of its glasses in a very long, slow, loop. The impact is a sharp tap on the center of the glass, and the glass used had a rather short decay period. AquaHarp was shown at the same Digital Arts Expo at CalArts that Noise Floor and TD Skillz came from.
AquaHarp II improves on the earlier iteration in a number of important ways. First, the striking mechanism has been replaced by an acrylic hammer shape from a narrow, sharp piston. The delay period between electrically telling the solenoids to start and stop striking acts somewhat like a MIDI velocity message, as well. Each glass position has a specially-sized mount for stabilization and security. The overall frame is lightweight and sturdy.
I had conceived of several ways to extend the algorithmic composition techniques of AutoLyrics to work with the AquaHarp II, and ultimately they worked perfectly and proved to be just as effective as I would imagine they would be... once we got the AquaHarp II responding to MIDI, which had never actually been done with the AquaHarp.
This became my eleventh-hour project for Automaton; I needed to find a way to get MIDI I/O into a standard Arduino Mega without breadboarding out the pins of a 5-pin (MIDI) DIN jack or using firmware flashes such as HIDuino from my friend and fellow CalArts MTIID MFA alum Dimitri Diakopoulos, which would have a higher learning time than I could afford at the moment.
Based on my flirty acquaintance relationship with the actual MIDI spec, I was thinking about how standard MIDI messages are usually just three binary bytes anyway (message type, status byte 1, status byte 2) - I dunno if it's serial or parallel or whatever over a MIDI cable or USB or wireless MIDI, but it's just three bytes - I should be able to get this into the serial monitor of a running Arduino without problems.
There were many problems to this, conceptually. While trying to use the officially-supported Arduino MIDI library, I realized many problems, such as the enormous mismatch in baud rate between the standard MIDI connection (31250) and most Arduino sketches (9600) would probably mean really asynchronous reception, and in fact things like a lack of a parity bit (which I had just recently been learning about in Advanced Circuit Design when discussing the creation of serial protocols) meant that when I received data, it would just come back as a garbled mess of truncated messages and line breaks.
I had even had some promising success from the other side of the signal chain by using USB/software MIDI-to-serial protocols such as the Hairless MIDI to Serial Bridge and even Serial to MIDI Converter, which is in fact just a wrapped application around Processing's serial library, and so it requires Java to run. (My program director, Ajay Kapur, probably would have insisted that we create something like this already, considering his preference for visualizing the reception of serial data using bar graphs in Processing.) And yet I just could not get coherent bytes to flow into my serial monitor. Just a few hours before the concert, I decided to scrap all my work, delete all this gross middleware and start fresh.
ChucK has - with only the most recent version 220.127.116.11, released to be ready for the new book on audio programming written by Ajay - added new, slightly unstable, mostly totally-undocumented serial I/O libraries. Ajay has already begun teaching a new crop of Interface Design for Music and Media students to begin demoing their ideas for instrument development exclusively in ChucK, rather than going through many other intermediaries such as Max. So, while the new crop of Interface Design students may produce a wealth of examples, this facility is otherwise undescribed in the present literature.
Yet, I knew ChucK would be an appropriate utility knife for this job, since I new it could receive and programmatically parse incoming MIDI events, and then ideally construct new serial events based on that reception - hypothetically, with much less runtime overhead than another DAW or middleware or some other non-"pseudo-interrupt style", event-based execution.
However, my present experience with constructing and transmitting various simple data types for efficicency sake over very simple protocols - bit masking and shifting operations and such - is brand new, so I really wasn't confident to develop my own specification with parity between Arduino and ChucK in time to understand what I was doing. The current (slightly hidden) example code for ChucK serial does come with one example Arduino sketch, as a matter of fact, but what it does (report "bar" whenever ChucK sends it a "hi!") is not very useful.
I knew how to receive and create serial in Arduino, but not do any kind of special parsing such as the .readBytesUntil() function used in the example code, and so this combined with my lack of knowledge about transmitting a specific ASCII char byte ("A", or "B", or "F", etc.) into the hack solution I developed.
Rather than just parsing a serial message's size being greater than 0 and flipping on a "switch," I filled the report with an integer reporting the size of the message until reception of a newline. So, even though "C" and "G" would both return only "1," that doesn't mean I can't have a dual correlation in the mapping C/DD/EEE/FFFF/GGGGG etc. That way I'd see the note itself in ChucK and the scale degree from the Arduino and get distinct sizeof reports from each position.
The result was thrown together in the truest essence of the idea - containing an inefficient case-checking algorithm with holes (no descrimination for note ons-only, just no note offs!) and the entire default example codes for both MIDI input and Serial output, but it worked and it was really exciting.
If you'd like to checkout the codebase (complete with one of my signature ridiculous README files), check the GitHub repository here.
You can faintly hear the debut of the robot running this code receiving input from a Max for Live MIDI channel in my Ableton Live session starting at 4:42 in the room recording here.