This is a self initiated project. This is also a Work in Progress Project.
This is a project about visual, sound and perception.
Right now this project has only begun. So this might be too early a stage for me to write too much about it.
What i'm trying to do is to create a new language with sound scape, a language which can translate the visual world into auditory perception. My aim is to be able to give the visually impaired, a chance at perceiving vision in their own way in order to lead normal lives without feeling that they never knew what it is to be able to see.
I have begun working on algorithms to achieve the same. But its too early to show anything.
Below is a very simple algoithm which detects the position of the brigthest source of light and traces with a sinewave sound in the stereophonic sound space.
If you know how to code in processing, you might wanna compile the code and see it yourself.
import processing.video.*;
import ddf.minim.*;
import ddf.minim.signals.*;
Capture video;
Minim minim;
AudioOutput out;
SineWave square;
void setup(){
size ( 640, 480 ) ;
video = new Capture ( this, width, height ) ;
video.start();
minim = new Minim ( this ) ;
out = minim.getLineOut ( minim.STEREO, 512 ) ;
square = new SineWave ( 500, 1, 44100 ) ;
out.addSignal ( square ) ;
}
void draw(){
if(video.available()){
video.read();
pushMatrix();
scale( -1, 1 );
image( video, 0, 0, -width, height );
popMatrix();
int brightestX = 0 ;
int brightestY = 0 ;
float brightestValue = 0 ;
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++ ) {
for (int x = 0; x < video.width; x++ ) {
int pixelValue = video.pixels [ index ] ;
float pixelBrightness = brightness( pixelValue ) ;
if ( pixelBrightness > brightestValue ) {
brightestValue = pixelBrightness ;
brightestY = y ;
brightestX = x ;
}
index ++ ;
}
}
//fill ( 255, 204, 0, 128 ) ;
noFill();
stroke(0,150,255);
ellipse ( width-brightestX, brightestY, 200, 200 ) ;
float poss = (float)(( (width-brightestX) - 320.0) / 320.0 ) ;
square.setPan ( poss ) ;
//square.setAmp ( (float)brightestY ) ;
}
}
void stop()
{
out.close();
minim.stop();
super.stop();
}
Here is a screen capture of a small variation on the code.
