Interactive Projection Installation

•

INTERACTIVE PROJECTION INSTALLATION

Spicing up an evening stroll or night ride home: an interactive projection installation surprises passers-by of the Wilminkplein with a projection that they can control by their movement. Wirelab invested in the hardware to create such an interactive system, consisting of an IR-camera, a PC, and a professional projector. At first, an open-source framework was used to perform the object detection. It performed sub-optimal and quickly became outdated and deprecated. The task for me was very clear: create a piece of software that gets our systems up and running again and gives them a performance boost.

The result is a flexible system that can recognize and track multiple custom trained objects and can be connected to any camera input and video output. The system has been designed to be as flexible as possible to be suitable for many different environments. The performance has increased significantly. Not only is the new software more accurate, but it's also possible now to detect different objects and track them over time. Thanks to this it is possible to create more interesting and complex interactions. The example below can recognize and differentiate between pedestrians, cyclists, and scooters and uses their movement as input for a fluid simulation.

The heart of the system is a convolutional neural network model that is trained on custom objects and communicates their location in the projection plane to an application that uses these locations to create an image. The system comes with a set of utility tools. These make it easy to collect and label images of target objects for training a new model, handle situations in which the projection plane differs from the video input plane, and automatically powers the beamer on and off at a given time (for example sunset/sunrise).

The example below is one specific example of the system functioning in its totality. The projection application is based on an existing solution, showing the flexibility of the object tracking software that was built.