The Swordguide is a interactive application that uses the Kinect-Camera to simulate you wielding a virtual sword on your computer-screen! Because of the Kinect's 3D-Camera, it can detect your hand's movements and teach you some basic sword moves with 5 different swords!
In the following video, I will show you how to operate the programm and how to test your virtual sword skills.
In the following video, I will show you how to operate the programm and how to test your virtual sword skills.
Unfortunately, the Kinect has a rather low resolution. I was bound to that hardware because of the 3D technology.
All of the 3D-Work is done in Maya. Five very different swords were modelled from scratch: A sword, a katana, a piratesword, a florett and a persian shamshir. They also each received basic textures to display a believable image. The textures remained basic, because otherwise, the application would have performance problems and the Kinect-Camera resolution isn't that good anyways, as you just saw in the video.
So how does the Kinect look into 3D-Space?
It works with an Infrared-sensor, which detects, ho far something is distant to the camera. In the basic stages of the development, I implemented a real-time video output as a pointcloud, which means that I disregarded any colors to just get video of points, and how far away they are.
It works with an Infrared-sensor, which detects, ho far something is distant to the camera. In the basic stages of the development, I implemented a real-time video output as a pointcloud, which means that I disregarded any colors to just get video of points, and how far away they are.
Of course all of this seems optional for a sword simulator on a 2D-screen, but having 3D-data is really useful to make it seem more realistic. As you have seen in the video, the virtual swords scale and turn in response to your movement in 3D-space.
The next picture shows a pointcloud, which also looks for patterns in the depth-axis to detect a person.
The coding itself is done in Processing. It's a great language for designers, because of its easy nature and direct access to visualisation. The reason why I used it for this project is because of the library SimpleOpenNI, which lets me extract all the data the kinect camera provides and use them for my purposes: to detect a hand and give it a sword.
The "Guide"-part of the application lies in a few curved lines and tracking of movements. Basically, a transparent line is seen on screen, which you can move along with your virtual sword. If you move correctly, the line will flash green for a short amount of time as a confirmation, that you did it right. There are two strikes, one offensive and one defensive for each sword.
The line's stroke varies in style, depending on the sword you are training with. The following picture shows the offensive strike of the claymore.
The line's stroke varies in style, depending on the sword you are training with. The following picture shows the offensive strike of the claymore.
