Physical Computing

tangible interaction for a personal project
A demo video of the final concept.

Inspired Sylvan Esso's looping rhythms in her piece "Coffee" (found on soundcloud and NPR), I designed a tangible interaction that invites users to manually layer audio and animations over one another — one audio track per letter of the word "design". To explore the visual and motion components of my concept, I sketched out ideas for how each letter would animate, according to its corresponding audio track.

My initial concept sketches.

Before refining my sketches any further, I decided which audio tracks to use. After spending probably too much time browsing, I decided on 6 simple tracks, one for each letter of the word "design". The first track features a guitar clip in E major. I layered on to that foundation a clap clip, a shaker clip, a sound effect, a bass clip, and a guitar melody in G minor. The clips that have rhythm are all at 110 bpm.

Using Audition, I compiled the clips to get a feel for what they would sound like together. Even though the input would ultimately determine when a clip is played, I wanted to get a sense of a best case scenario (if the input triggers an audio clip directly on the beat).

I used Audition to test how each audio clip might sound while layered onto eachother.

Once I heard the audio files coming together, I was able to draw inspiration from the feeling of the music. I was inspired by one of the album covers that provided some of the clips in particular, and drew my colors from that inspiration.

This album cover (via splice.com) served as my inspiration for my color palette.

Given the sound of the audio clip, I created a motion for the "D" letter that is driven by short "staccato" line segments. At 110 beats per minute, the clip and animation are similarly fast paced. I designed the rest of the letters to similarly complement their respective audio tracks.

A demo video testing the motion design for each letter.

Having refined my motion design, I moved on to prototyping the interaction using my keyboard as inputs.

A prototype of the interaction using keyboard buttons as inputs.

To replace the keyboard inputs with something more reminiscent of playing music, I explored two main concepts: six unique turntables or one old school boombox. Below are some sketches storyboarding each concept.

By sketching these concepts out, I discovered that the simplest solution for individual interaction would be combination of both the turntable and the buttons from the boombox. Instead of directly triggering the animations to play, the buttons would "select" each letter. Spinning an 8-track record (glued to a potentiometer) would then trigger each letter to "play". Using an arduino, I began to prototype how to replace the keyboard inputs with LED buttons.

My initial LED button circuit.

While this circuit and code worked in terms of lighting up the LED inside the button, when I tried to connect multiple buttons using the same circuit, I started to have issues with my input being inconsistent. To fix this issue, I grounded each input using a resistor, in addition to adding a resistor to the ground. After duplicating this setup six times, I ended up with the following circuit. The orange wires are my outputs (pins 8–13) for each button. The yellow wires are my grounds for each button. The green wires are my power sources. The blue wires are my inputs (pins 0–7).

My revised circuit for all six buttons.

Having figured out all of my inputs, I soldered everything using wires in my Arduino kit and connected everything using my breadboard. After figuring out how all of my inputs would fit together, I then designed the turntable which would house them. Below are some of my inspiration images (click for source)s.

While many of my inspiration images had knobs on the front panels of the turntables, I decided to place my buttons on top of the turntable in order to make them easier to push. I also experimented with different levels of instruction for the interaction, ranging from explicit written instructions to simple icons.

Ultimately, I decided to go with minimal instructions, with the buttons below the record. Additionally, while I liked the wood aesthetic of some of my inspiration images, I felt it would clash with the cheap plastic that my buttons were made out out of. After making a foam core model of the turntable, I transitioned to laser cut acrylic. Using black acrylic would also have the desirable effect of highlighting the interactive elements — the buttons and record — as they would be the only white elements on the turntable.

In order to join each side of the turntable together, I used glue and lap joints, as shown by the sketches below. Thank you to Chris Stygar, Josiah Stadelmeier, and the Carnegie Mellon School of Design 3D Lab for helping to conceptualize and laser cut these elements!

The initial sketches and final Illustrator file for the turntable (above) and final assembly, with the Arduino housed via a removable bottom piece (below).

As a final step, I updated my prototype code to use the buttons and potentiometer as inputs. I also had to create a new "selected" state such that only one button's LED would be on at any given time.