Experiment for Kinect

In this experiment I am trying to use Hand Gesture to play the audio. By using KinectInputModule to track human body.

First create KinectInputModule class which is also a subclass of Unity’s BaseInputModule.

Then by using Process() method to track hovering, pressing actions.

Next which is coding.

For handling hover, I used BaseInputModule’s HandlePointerExitAndEnter method.

For processing press we are going to use a really basic solution which is just tracking our handstate.

This is how to make a Kinect hand cursor. My future plan that I want to use gestures as buttons to control the button.


Demo_Final Project Proposal

I do a demo video for the final project which can make my proposal more clearly.

Demo Video

This video is a demo video for my final project. What I want to do is people can play music through their bodies and gestures to play music. They can try to match the poses in screen to play the music notes then after finish all poses, people can listen the whole song which finished by themselves.

Below is my schedule for next term:

I will try to do more deep research on how to combine audio sensory experience with visual to get an immersive experience and also do more future research on how to use gestures as button to play the audio. Since my project need have a lot of audio, I will try to finish the audio on first 5 weeks, and also start to do graphic design on week5. 3D person which is also an important part in my project, I will spend a lot of time on it. And try to start unity development on week 8 and continue doing in term 4.

Experience for Rewild Our Planet

Rewild Our Planet which is an immersive and interactive exhibition in ArtScience Museum. The technology they used is AR. Users can hold the phone which given by staff to experience the exhibition well. It will take you to the last wilderness landscapes on Earth. By taking you out of this world, REWILD Our Planet brought us on an inward journey to discover what it takes to protect the natural habitats that we love.

“Rewild Our Planet”的图片搜索结果

Nature is moving away from us at an unprecedented rate. Animal and plant species are disappearing at an alarming rate. Our forests, oceans and fresh water are all at stake. Nature is vital to our survival and future. Our homes, our health, the food we eat, the air we breathe, the water we drink — our lives and everything we care about, without nature, would not exist at all.

The project, produced by immersive technology studio PHORIA and in partnership with WWF, Google, Netflix and the museum of art and science, is provided by Silverback Productions. The aims of this exhibition is to inspire us to save our planet.

An experience for Alice in Wonderland

Yesterday I had an experience in ArtScience Museum for an exhibition called Alice in Wonderland.

The most impressive of this exhibition is an installation called Tea Cup Party which designed by Mad Hatter.

In this space users can sit around a table, and a lot of cups and plates are the table. In order to create an immersive experience for users, the designer used a lot of lights and audios for it.

It make me know the importance of immersive experience. And for the immersive experience, audio is also plays an important role.

Ongoing Research for audio sensory experience

Since hearing is an important channel after vision in five senses. It is an essential medium for people to obtain external information and audio sensory experience.

Basically, the sound can create a different mood and atmosphere, it directly connects to people’s emotion. The sound can affect people in many ways, such as physiology and psychology. The visual and auditory systems every now and again cooperate to encourage the recognizable proof and confinement of articles and occasions in the outer world. Experience assumes a basic role in building up and keeping up consistent visual–auditory affiliations, so the distinctive sensory prompts related with focuses on that can be both seen and heard are incorporated properly. For boost area, visual information is ordinarily progressively exact and solid and gives a reference to aligning the view of auditory space.

In physiology, sound can easily affect the human body because of sound propagates in the water faster than in air, and there have 70% of water in the human body. Physiologically, sound affects us in very powerful ways that hearing is the main sense of alarm for the human to begin a process. Moreover, adding the sound detail or strengthening a particular rate of response, or increasing the volume of specific sound to improve the capability of the human’s brain to locate the sound. Secondly, in a psychological way through music to change people’s emotions. Such as listening to the favorite song to trigger happiness. Besides music, there are many natural sounds also can assist to change people’s mood.

An Experience for Marina Barrage

Today we went to Marina Barrage to have a field trip. The main topic of this exhibition is about climate change. This exhibition has a lot of interactive installation for introduction.

The first installation which is by moving the ring which is a screen to show different photos, but the background are just the normal photos. People can through interacting with it to see and read more.

The second installation which is an AR installation. The white cubes behind which are markers. People can customize it to create their own buildings and also can turn to different angles.

The third installation that users can through closer to the fans to activate it. When users put their hands closers, all fans will turn on.

Case Study

I do research some cases that to inspire me by using which interactive technology to do my final project. Since my topic is about human sensory experience and I wanted to do a project about music, so I will focus more on audio sensory experience.

Kinect Piano
Interactive floor piano

These two project which are more like an air piano, which can play piano by using Kinect, inspired me that can use different ways to experience music.

Air piano by using Kinect

This project make me feel that using Kinect is a suitable way for me to do my final project. This project is through buttons to play the musical notes which is similar to my project and also make me feel it is feasible.