IAT320: A3 - Movement "Musical Movement"

•

IAT320: A3 - Movement

MUSICAL MOVEMENT

a musical display of color and movement

CONCEPT

My concept describes the dynamic relationship between music and movement. It is a rich affiliation of two forms of communication, with music being a highly expressive form of communication and can be represented in so many different ways. I chose to represent music and movement on a digital display using a music visualizer with moving bits of color and energetic shapes to visually show the relationship of music and motion.

PROCESS

My final iteration consists of a SHARP 2Y0A21 infrared proximity sensor for the input of movement (detecting and measuring the distance between an object and the sensor) and my laptop screen as the output display. Originally I had wanted to implement a gyro sensor as the analog input to measure tilt instead of distance and would have created a more interactive project; however I faced issues with simply connecting the gyro I had purchased to my Arduino. I could not get any motion readings from the sensor. Therefore, I had to think of a different way to use movement as an input and had to settle with an infrared proximity sensor to measure distance and movement -- a more simple input, but one that still detects movement.

I used Arduino and Processing together to create the output display to visually represent the motion data coming from the sensor. I used the serial communication between the two programs to send the analog data from the sensor in Arduino to Processing. I then used Processing to use that data as the variables to change the color and size of the ellipses being shown on screen.

INTERACTION

I chose to use the shape of a handprint to afford the movement of the audience to place their hand over the sensor and therefore interact with the project. Usually handprints are used to signify the placement of one's hand over it, such as hand scanners for example. I want the audience to see how their hand movement can manipulate the visual display from the music visualizer, hence making them one with the music.

FURTHER IMPLEMENTATION

I would like to eliminate the lag within the code as Processing receives the analog input data from Arduino, since as of now there is heavy lag on the screen when the hand is closer to the sensor. Once that transition is smoother, I believe it can be a fun tool for DJs, clubs and parties for light visual entertainment along with energetic music.

ARDUINO CODE

const int sensorPin = 0; // the pin that the IR sensor is attached to

void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
// initialize the sensorPin as an input:
pinMode(sensorPin, INPUT);

}

void loop()
{

int sensorVal = analogRead(sensorPin)/4; // dividing sensor value by 4 reduces the range to 0 to 255
Serial.write(sensorVal);

//wait 100 milliseconds so we don't drive ourselves crazy
delay(100);
}

PROCESSING CODE

/*
* A Simple Monochromatic sound Visualizer
*
* This sketch plays a sound and sends the average
* values in the audio buffer as integer values to the arduino
* where an LED changes its brightness based on the values sent.
*
* > I changed this code to take in analog sensor data from an IR sensor
* using Arduino and output an ellipse that changes color and size
* based on the serial data coming from the IR sensor.
*
* design and code by:
* Abhik Pal. 10th January 2014, 23:13
* Modified by Graceleen Cruz, 301165632, March 1 2016
* IAT320: Assignment 3 - Movement
*/

// Importing the Minim and Serial libraries

import ddf.minim.*;
import processing.serial.*;

// creating required objects
AudioPlayer song;
Minim minim;
Serial myPort;

// variable to store incoming bytes to be translated into colors
int sensorValue;
int strokeColor;

// define colors
color BLUE = color(0,128,255);
color GREEN = color(0,255,128);
color YELLOW = color(255,255,0);
color ORANGE = color(255,178,102);
color PINK = color(255,102,178);
color RED = color(204,0,0);

// variable to store the value to be sent
int sendVal = 255;

// variable to store whether the song is being played or not
boolean isPlaying = false;

void setup()
{
// setting the size to be the current display size
size(displayWidth, displayHeight);

// setting everything up!
minim = new Minim(this);

// make sure to change the stuff inside the
// "quotes" to mathch the file you want to play
// here I'm playing "song.mp3"
song = minim.loadFile("01 Red Lights.mp3");

// change the COM port to match your Arduino's COM port
// change the baud Rate if you have used someting
// different from 9600 bauds.
//println(Serial.list());
String portName = Serial.list()[2]; // change the 0 to a 1 or 2 etc. to match your port
myPort = new Serial(this, portName, 9600);
}

void serialEvent (Serial myPort) {
// get the byte:
int inByte = myPort.read();
sensorValue = inByte;
}

void draw()
{
// clearing out the screen by creating a rectangle
// having dimentions of the screen size.
fill(20);
noStroke();
rect(0, 0, width, height);

// this loop is used to extract and display visuals
// based on the values present int the audio buffer
for (int i = 0; i < song.bufferSize() - 1; i++)
{
// variables to store the right and left audio values
// i.e. sound being sent to each left and right speaker.
float rightVal = song.right.get(i);
float leftVal = song.left.get(i);

// the next few lines account for the pattern being
// displayed on the screen
pushMatrix();
translate(width/2, height/2);
rotate(radians(map(i, 0, song.bufferSize(), 0, 360)));

stroke(0, 0, 0, 10);
strokeWeight(25);
point(0, 150 + leftVal*200);
point(0, 150 + rightVal*200);

strokeWeight(2);
stroke(220, 220, 20);
point(0, 150 + leftVal*150);
stroke(20, 220, 220);
point(0, 150 + rightVal*150);

// display sensor changes and visualize as an ellipse
if (sensorValue >= 30 && sensorValue <= 60) {
strokeColor = BLUE;
println("Color = Blue");
} else if (sensorValue >= 61 && sensorValue <= 90) {
strokeColor = GREEN;
println("Color = Green");
} else if (sensorValue >= 91 && sensorValue <= 110) {
strokeColor = YELLOW;
println("Color = Yellow");
} else if (sensorValue >= 111 && sensorValue <= 130) {
strokeColor = ORANGE;
println("Color = Orange");
} else if (sensorValue >= 131 && sensorValue <= 150) {
strokeColor = PINK;
println("Color = Pink");
} else if (sensorValue >= 151) {
strokeColor = RED;
println("Color = Red");
}
stroke(strokeColor);
ellipse(0,0,sensorValue,sensorValue);

popMatrix();
}
}

// function to change play/pause the song each time the
// left mouse button is pressed.
void mousePressed()
{
// if the song is already being played
// we pause it and change the value of isPlaying variable
if (isPlaying)
{
song.pause();
isPlaying = false;
}

// and if it is paused we play it and change 'isPlaying' accordingly
// :P pretty simple right?
else
{
song.play();
isPlaying = true;
}
}