I felt that using the wand instrument would be more engaging and instructive if there were a visual component. So using the topics we covered this semester in The Nature of Code, specifically particle systems, vectors, forces and oscillation, I made a set of musical particles that exist on a screen. Each time a musical note is played from the wand, the corresponding particle jumps up into the air on the screen. The ball then falls down to the “ground”, bouncing until it comes to rest in it’s original place until the note is triggered again. This kind of playful interaction makes it more clear to the user the effects of their actions with the wand on the musical notes and adds an element of feedback, heightening engagement.
You can see the signal flow in the picture below. The wand itself is connected to the laptop via Bluetooth MIDI, which is a simplified and straightforward implementation of BLE. Each time the wand plays a musical note, a MIDI note is received in MaxMSP where it is forwarded to both Ableton, where the sound is generated, and to p5.js, where the particle movement is triggered. There are other MIDI specific effects that are contained in MaxMSP which correspond to specialized movement coming from the wand. For instance, when the user swings the wand quickly downwards or upwards, a special sound is played with a corresponding visual response. These interactions are not explicit and are discovered by the user
The project is centered around the Arduino MKRZERO micro-controller board. I connected a BM-A01 Bluetooth MIDI transmitter to the TX pin of the Arduino to receive the notes. The wand’s orientation and movement is tracked by a BNO055 9-axis accelerometer.
I imagine this piece being used to highlight Mergia’s story and musical stylings. Mergia lived under the censorship of the Derg dictator ship in the late 1970’s and made his way to the United States in 1983, settling at Howard University where he recorded Hailu Mergia & His Classical Instrument. The album features simple instrumentation - a Rhodes electric piano, a synthesizer, and a drum machine - which I have recreated somewhat here. The music imparts a sense of joy and playfulness that I sought to capture in the interaction. The sound of the wand is rolling and tumbling, where each note is played in sequence to the one next to it, up and down through the musical scale, which mimics Mergia’s rolling right hand technique, similar to other Ethiopian artists such as Emahoy Guèbrou.