About the sensors

Here is a video which explains how the sensors and the software work. This is fairly non-technical, and is designed to give teachers and project leaders an idea of how things work.

Special Needs Music Software Demo from David Stevens on Vimeo.

The custom built software used in the project is basically a four channel sound sampler, which can play back pre-recorded sounds, and record sounds that the children make. There are two ways of playing back the sounds – either playing individual sounds (“segments”) or manually playing a very small loop of sound from anywhere in the stored sound (granulation).

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Playing segments (individual sounds): I’ve recorded myself into the software, simply counting numbers. Moving my hand in fornt of the distance sensor causes individual words to play back.

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Playing grains: Here, instead of playing back the individual segments (in this case, the individual words), by moving my hand back and forth I’m moving the play position across the whole recording.

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Pitched segments: This is the same as the play segments audio, except that the sensor movement also changes the pitch of the playback.

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Pitched grains: And this is the same thing for the granular playback – moving my hand moves the play position and changes the pitch of the sound.

The sounds in the computer are played using a variety of sensors – these are simple devices which measure things such as distance, light level, movement, pressure, bend, temperature and so on. (Everyday examples of the use of sensors include burglar alarms, and the detectors which open doors automatically as you approach.)
After trying out a number of different types of sensors on earlier projects, we have settled on three types as the most useful for this workshop:
Infrared sensors – measure distance from the sensor. Backwards and forwards movements of the whole body, arms, hands or head (which the children generally find hilarious) are used to control playback.
Floor pads – measure pressure when someone stands on them. (They can also be turned upside down and pressed – useful for children who are unable to support their own weight for long when standing). These are most often used to play drum sounds, which means that children can dance to their own drum beats!
Tilt sensors – measure the amount of turn from a neutral position. Hand-held and waved in the air, these sensors have two axes of movement, and so can control two different parameters at once (eg which sound to play, and the playback pitch)
Though not used very often, the software can also be controlled with game controllers such as joysticks and the Wii controller.