School of Science and Technology 科技學院
Computing Programmes 電腦學系

A Virtual Reality Game for Exercise with Motion Capture and Body Sensors

HUI Hon Kit, LEUNG Yiu Yeung and NG Ka Leung

ProgrammeBachelor of Science with Honours in Web Technologies
SupervisorDr. Jeff Tang
AreasVirtual Reality and Augmented Reality Applications
Year of Completion2016
Award ReceivedChampion, Digitoy Student Project Contest 2016



We aim at developing a sports game that helps people to release stress by doing more exercise. With the combination of the new technology(Virtual Reality, abbreviated as VR), we focus on how to create a new way for people to do sports, without consideration of the weather, budget, the size of venue and others‛ company. In reference to the success of a very hot society “Superhuman Sports”, extending human abilities or sensation would make people be more motivated and excited to do sports.

To achieve our aim, the main objective of the project is to develop a monster combat sports game which can calculate and analyse for the calories burned after users have played the game. The project is defined by a number of objectives:

  • To attract and motivate players to do workout
  • To train the reaction of players and let them have an excellent workout
  • Develop a safe checking system

Background and Methodology

Our application aims to encourage more people to exercise. Our gameplay will divided into three parts, the user inputs, game environment, and the game environment safety control. And our main aims is, help the people to lose their weights in a reality and funny way.

In the mobile application, the system flow chart is shown below:

The following shows the flow charts of device connections and notification function:

As we have mentioned, our application is to encourage users to do exercise. One of the realisation is to calculate, record and show the data after players have played our game. Therefore we need to capture and process the posture or motion of the user, such as jumping, crouching or even attacking or punching. For counting the speed that the user give and calculate the response time, we opt for the tool- Xbox One Kinect(Kinect). Kinect can help us to monitor the user actions.

The Kinect sensor can detect more than 20 parts of the body. If you put the 3D model into the game engine using Kinect Software Development Kit(Kinect SDK), the 3D model will move in accordance to the user actions at the same time.

In the game, users are supposed to attack or dodge the enemies. So, we will use the collision feature of Unity to handle the Interaction. In Unity, we can use some scripts to move the position of characters or to destroy the objects.

Our application is to attract more people to do exercise and have fun. At the same time, we also want users to feel more realistic, which motivates us to use the Virtual Reality(VR) technology, the oculus VR development kit 2, to create a vivid environment inside our application.

Since the game needs to collect and measure the electrical transmission of muscle activities, we chose to use the MyoWare. However we later discovered that the MyoWare cannot transmit the data to the PC directly. As a result, we decided to build a Wireless Biofeedback Controller. It consists of the Muscle Sensor – MyoWare TM , Arduino Board and the Bluetooth module. They can handle muscle activity measurement, communication between bluetooth and the sensor, and wireless transmission respectively.

During the gameplay. Users need to make an attack to the monsters when the counters above each monster have reached “1”, such as punching or kicking. They can see their results (including numbers of kills, Current Combo, and Max Combo) on the screen at the same time. The blue bar above the results is the “Skill Bar”. This skill bar is charged by the number of kills of the monsters. Users can determine the use of the ultimate skill when the skill bar is fully charged.


Our survey were conducted among 20 respondents. Almost half of the respondents(45%) aged from 18-24 years old, which makes our survey results representative of the current conditions of adolescents. From the results, 20% of interviewees do exercise less than one time per week and 25% of them, which is one-fourth of the interviewees do exercise less than 10 minutes each time. These results both reveal that many people do not have enough exercise each week. This can be foreseen that they may have high tendency to have health issues. Moreover, for the attractiveness of our game, more than 75% of respondents agree that the game is fun and feel tired after playing the game, and more than 70% of respondents agree that they may play the game regularly and the game does encourage them to do exercise. These results also symbolize that we are successful in motivating people to do exercise by our game, which is contributed by its fun and interest.

Conclusion and Future Development

There are some suggestions for the game features. We will consider these suggestions for further improvements of our game. For example, we may increase the current song list, add more different types of game and add the 2-player mode.

Copyright Hui Hon Kit, Leung Yiu Yeung, Ng Ka Leung and Jeff Tang 2016

Jonathan Chiu
Marketing Director
3DP Technology Limited

Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.

Krutz Cheuk
Biomedical Engineer
Hong Kong Sanatorium & Hospital

After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.

Hugo Leung
Software and Hardware Engineer
Innovation Team Company Limited

Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.

The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.

Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.

Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.

Course CodeTitleCredits
 COMP S321FAdvanced Database and Data Warehousing5
 COMP S333FAdvanced Programming and AI Algorithms5
 COMP S351FSoftware Project Management5
 COMP S362FConcurrent and Network Programming5
 COMP S363FDistributed Systems and Parallel Computing5
 COMP S382FData Mining and Analytics5
 COMP S390FCreative Programming for Games5
 COMP S492FMachine Learning5
 ELEC S305FComputer Networking5
 ELEC S348FIOT Security5
 ELEC S371FDigital Forensics5
 ELEC S431FBlockchain Technologies5
 ELEC S425FComputer and Network Security5
 Course CodeTitleCredits
 ELEC S201FBasic Electronics5
 IT S290FHuman Computer Interaction & User Experience Design5
 STAT S251FStatistical Data Analysis5
 Course CodeTitleCredits
 COMPS333FAdvanced Programming and AI Algorithms5
 COMPS362FConcurrent and Network Programming5
 COMPS363FDistributed Systems and Parallel Computing5
 COMPS380FWeb Applications: Design and Development5
 COMPS381FServer-side Technologies and Cloud Computing5
 COMPS382FData Mining and Analytics5
 COMPS390FCreative Programming for Games5
 COMPS413FApplication Design and Development for Mobile Devices5
 COMPS492FMachine Learning5
 ELECS305FComputer Networking5
 ELECS363FAdvanced Computer Design5
 ELECS425FComputer and Network Security5