Music from the Heart -- Music Suit
(→Music Tones - Electrical Design)
Revision as of 18:11, 16 March 2010
This project attempted to create a natural form of musical expression by connecting sensors to the body. Six tilt switches were attached to the wrist, ankles, and shoulders, each controlling a single pitch from the pentatonic scale. The heart beat was obtained using photoplethysmography on the user's finger, and this signal was used to strike a drum in sync with the heart beat.
- Thomas Peterson (Computer Engineering, 2010)
- James Rein (Biomedical Engineering and Music Cognition, 2010)
- Eric West (Mechanical Engineering, 2011)
INSERT TEAM PICTURE HERE
Although intended as a single, cohesive system that would allow the user to intuitively make music, the project was easily divided into three subsystems: heart rate monitor, drum actuation, and music tones. Below is more explanation about each subsystem.
Heart rate monitor - Concepts
In brainstorming how to translate a heart beat into a drum beat, we decided that the least intrusive method would be the best. Attaching electrodes to the user would simplify the process of identifying a heart beat, but attaching the sensors directly to the skin would be a time-consuming and personally-invasive process. Instead, we decided a finger tip sensor would be much more comfortable and easy to use.
To make a finger tip sensor, we used the concepts of photoplethysmography. Photoplethysmography is typically used in pulse oximeters and finger-tip sensors in commercially-available devices. We originally tried to hack a heart rate monitor that we had purchased, but this proved difficult, so we decided to make our own.
The basic concept of photoplethysmography is that blood reflects a certain amount of IR light, and the blood density in the finger changes as the heart pumps, so the IR reflectivity of the finger changes as the heart beats. Using an IR emitter-detector pair found in the mechatronics lab followed by lots of amplification and filtering, we were able to obtain a decent signal with peaks when the user's heart beat.
Heart rate monitor - Electrical Design
The signal coming directly out of the IR detector was very weak and noisy. To obtain just the information we desired, we ran it through the following filters and amplifiers:
- Passive High Pass Filter - cutoff frequency = INSERT
- Inverting Amplifier - gain = INSERT
- Active Low pass filter - cutoff frequency = INSERT
- Bandpass filter - Frequency range = INSERT
- Comparator - output to the PIC
The complete signal processing circuit is shown below.
INSERT CIRCUIT DIAGRAM
Heart rate monitor - Mechanical Design
The sensor itself was a INSERT COMPONENT NAME HERE that was encased in open-cell foam insulation. We inserted it into the foam by drilling and filing a hole in the foam to the appropriate size, and gluing the sensor in place. A round, finger-shaped groove was filed into the foam and the sensor sat flush with the bottom of this groove. The user's finger was held in the correct position with relatively constant pressure using a latex band around the finger and the foam enclosure.
INSERT PICTURE OF THE HEART SENSOR
Drum actuator - Electrical design
Once the signal from the heart rate sensor is sent to the PIC, the rising edge triggers an interrupt which starts the driving sequence for the motor that strikes the drum. The drive sequence was: drive down (into drum) 100ms, drive up 75ms, wait 150ms. The interrupt does not do the entire driving sequence, it merely flags a variable to start the sequence, which is done in the main loop, as seen in the code below.
An additional interrupt was triggered by the falling edge of the signal; this falling edge interrupt added additional wait time that prevented the PIC from being "fooled" by a noisy falling edge. For instance, if the falling signal had some noise in it, the rising edge interrupt would be triggered again, thus causing the drum to strike on both the rising and falling edges of the heartbeat. We only wanted one strike per heartbeat, so the falling edge interrupt was triggered by the first falling edge, and any noise in the falling edge was ignored because the PIC was told to wait for 300ms. With a maximum delay of 150 + 300 = 450ms from the two interrupts, the fastest heartbeat our drum could play was 133 beats per minute, but we did not expect anyone to come play the instrument after running, so this was plenty high for our purposes.
Initially, we tried skipping the comparator and sending the analog signal directly into one of the PICs analog inputs. Our intention was to use the fast Fourier transform (FFT) library to try to extract the dominant frequency. Unfortunately, we ran into various problems. First, to get the accuracy we wanted for small frequencies, the sample window had to be very long (10 seconds or more), which made the output lag changes in heart rate. Additionally, the FFT was often "messy", which messed up our original algorithm of detecting the peak frequency between .5 and 2.5 hz. The system would sometimes randomly switch between very different frequencies, and although it was right most of the time it was enough to make the drum beating sound bad. We also tried algorithms to find the fundamental frequency using the entire FFT (instead of just the .5-2.5 hz range), but this was still unreliable. We considered FFT peak detection algorithms, but they seemed to complex for the time and computing power we had available. Because of all this, we went with the simple comparator in the end.
The motor for driving the drumstick was driven by an L298 H-bridge. See Driving a high current DC Motor using an H-bridge for more information on how this works. Our particular circuit diagram is shown below:
Drum actuator - Mechanical Design
We wanted to use a real drum for our drum beat, so our challenge was to attach a motor and drumstick to a drum. We chose a small rack tom for its resonant tone that is reminiscent of a heart beat.
To attach the drumstick to the motor shaft, we used a small block of acrylic (.5"x.5"x2"). The drumstick was cut down to 8" in length so that it would require less torque from the motor. In one end of the block, a hole was drilled and the drumstick was sanded to the correct diameter in order to snuggly fit into the hole. To ensure that the drumstick could not slide out of the block, a screw was placed in a hole drilled through both the stick and the block. The other end of the block was attached to the motor shaft using a split-clamp method: A hole the size of the motor shaft was drilled through the block, and then a slit was cut from the end of the block to the hole, allowing a screw to tightly clamp the block around the motor.
INSERT PICTURE OF DRUMSTICK-MOTORSHAFT ATTACHMENT
To attach the motor-stick assembly to the drum, we utilized the tuning screws on the drum. A sheet metal bracket with holes in the appropriate places attached the motor to two of these tuning screws. The bracket was easily made by cutting and bending a piece of GAUGE gauge steel sheet metal to the appropriate size. Ideally, this part would have been made out of one continuous piece of sheet metal, but we could not find a large enough sheet available in the shop, so we used rivets to connect two smaller pieces together.
After testing the drum striking, we decided it was too loud. To dampen the sound, we taped a piece of foam core and a shop rag to the head of the drum where the stick strikes it.
Parts list for drum actuator
- Bargain drumstick, at least .5" in diameter
- Small block of acrylic, .5"x.5"x2" (aluminum would work just as well, this is what was available)
- GAUGE steel sheet metal for bracket
- Screws and nuts
- Foam core, shop rag, tape for dampening
Music Tones - Electrical Design
The music tones were activated by tilt switches and generated using a YMZ284 chip and were output through a 1/8" jack to a standard set of computer speakers.
Each YMZ284 is capable of producing and mixing any combination of three tones. Since we wanted to have six notes from a pentatonic scale, we needed two of these chips and used an summing opamp circuit to combine outputs from each chip. We envisioned a possibility of adding overtones to each note, to make them sound better and more like a real instrument, so we ended up having four YMZ284s attached to our board. When we tried adding an overtone an octave above the fundamental frequency, it ended up sounding harsh and whiny. We decided the tones sounded better with no overtones, so we are only using 2 of the 4 chips on our board.
The YMZ284 communicates with the PIC over an 8-bit databus, plus additional pins for chip select, address, write enable, and reset. All four of our YMZ284s shared all of their pins except the chip select line; each required their own individual chip select line so that the PIC could specify with which chips it wanted to talk. The circuit diagram below shows how the chips were connected to the PIC and the summing circuit that followed.
Tilt switches - Electrical Design
In order to detect when the limbs of the user had moved past a certain critical angle, we chose to use tilt switches. We purchased our tilt switches for $2 each from Electronics Goldmine. We chose to use tilt switches instead of an accelerometer because the switches were much cheaper and we figured they would be easier to work with.
The tilt switches we chose are optically-based with an emitter and detector that are selectively blocked by a small ball. When the critical angle is achieved, the tiny ball rolls up a ramp, allowing the emitter to shine directly on the detector, and current flows. When the angle is neutral, the ramp forces the ball to block the light, making current stop flowing through the phototransistor. The "ramp" is really an inverted cone, so the switch can be activated by a change in angle in any direction away from the neutral position. We purchased three different tilt switches to try, each with a different critical angle - 15, 30, and 45 degrees. The configuration we ended up using was:
- 30 degrees for shoulders and ankles
- 45 degrees for elbows
In hindsight, we would have used all 45-degree switches because they produced the sharpest jump from off to on. The 15-degree switches were practically unusable because they created a large amount of noise as they switched on and off.
We used the tilt switches to provide high-low inputs to the PIC to tell it when to turn on any of the six tones. By attaching these switches to the body such that they are in the neutral position when the body is relaxed, we were able to control the six tones by moving the body in relatively natural ways.
Tilt switches - Mechanical Design
The switches were attached to the user's body using adjustable straps and buckles. The wrist and ankle sensors were simply a loop of webbing with an adjustable buckle attached. The straps could be tightened or loosened to fit any size user. To attach the switch to the straps, we created an enclosure for each switch out of high-density foam (left over scraps from DSGN 307). The switch was jammed into a hole in the foam so that it could not easily be moved. The foam was then taped to the straps using electrical tape.
INSERT PICTURE OF THE WRIST AND ANKLE STRAPS
The shoulder straps were a bit more involved. Two loops with buckles, similar to the wrist and ankle straps, went over the user's shoulder and underneath their armpit. To keep these straps from sliding off the shoulders, a large strap across the user's back connected the two smaller straps together. This large strap was also used for mounting the solder board which gathered the wires from all six tilt switches and sent them to the PIC over one ribbon cable. Using this strap configuration, we found that the user could comfortably and securely wear the sensors.
To attach the switches to the shoulders, we again created a foam encasement. However, electrical tape was not secure enough to hold the foam pieces to the irregularly-shaped shoulder straps. Instead, we riveted a piece of canvas to the webbing straps. We chose to use rivets instead of sewing because the sewing machine could not tightly sew the foam encasement in place. With rivets, we were able to pull the canvas tightly around the foam so that it couldn't move.
INSERT PICTURE OF THE SHOULDER STRAPS
To download the entire C file, click here.
The project turned out to be a mild success overall. The tilt switches worked very well for activating the music tones, but the heart rate monitor was less successful. One of the biggest issues was getting rid of very low frequency drift in the signal that made the peaks sometimes as high as 10V, but other times less 1V. Dealing with this fluctuation in peak size was the most difficult part of signal processing. As mentioned above, we tried FFT in order to establish a steady heart rate, but could not get the resolution we desired, thus we abandoned the idea.
PROBABLY NEED MORE TECHNICAL RESULTS HERE, BUT I'M NOT SURE WHAT TO SAY RIGHT NOW
The best result was that using the device was just plain fun. We each had a great time moving around and hearing the fun sounds that resulted, or if we were feeling ambitious, we would try to play a simple melody like "Mary Had a Little Lamb." It was rather difficult to play anything more complicated than that because it required lots of body control and precise movements.
- tilt switches activated each of the six notes when they were supposed to
- the YMZ284 chips were able to produce all six notes at the same time, allowing cool chords
- tones were in tune and formed fun melodies and chords as the body moved
- drum actuator hit the drum reliably and with good tone
- heart rate monitor usually showed clear peaks on a scope, but was finnicky in practice
Room for improvements
- tilt switches sometimes bounced, so the beginning, or particularly the ends of the notes sounded jagged
- Possible Solution: add a low pass filter to the switch signal so that the signal ramps up and down.
- the shoulder straps sometimes needed adjustments for each individual in order for the sensors to be in the correct position
- Possible Solution: create straps with more constraints, such as a strap across the chest as well as the back, to ensure that the sensors are in the correct positions and orientations.
- the straps could be easier to put on
- Possible Solution: maybe integrate them into a single garment, like a onesie
- the heart rate monitor was not reliable. Low frequency drift made setting a comparison level nearly impossible, especially for more than one specific user.
- Possible Solution: improve FFT method so that desired resolution could be achieved in a short sample period (around 10 seconds). Or implement a complicated peak detection algorithm on the PIC.
JAMES SHOULD ADD SOME KIND OF HEARTWARMING CONCLUSION CUZ HE'S GOOD AT THAT STUFF