School of Science and Technology 科技學院
Computing Programmes 電腦學系

‘Watch’ Out! Elderly Safety Alert System

Wong Yiu Hong, Li Kin Lun and Poon Ka Ho

ProgrammeBachelor of Science with Honours in Web Technologies
SupervisorDr Jeff Tang
AreasInnovative Systems
Year of Completion2015

Objectives

This project is aiming to provide an integrated, smart platform that can able to study the behavior of elderly residents. We decided to develop an Android application with Elderly safety alert system. The final product we expected the system could determine whether the elderly fell down or not and notify the guardian when the elderly need assistance.

The concept of our system is to provide a promising and cost-effective way of improving safety alert for the elderly residents and the disabled in a non-obtrusive way. Therefore we will implement a personal safety system that keeps an eye on elderly residents, monitoring their behaviors and serves as an alert system to notify users if they are injured or incapacitated. A brief line of message will be send to the guardian’s station or smart device which indicates the activity or abnormal incident about the resident. Guardian could also monitor from their monitor to acknowledge the condition from the web camera of the system that place in the location. For the rest of the project proposes a handy, simplicity approach to achieve home care, react and response to variable tasks performed both end user and elderly residents.

Background and Methodology

The whole system can be divided to three components, which is the Data Collection Side, Server Side and Message Receiver Side.

Here using three kind of devices, which is Android wear, Android phone/tablet and webcam or Kinect. Android Wear Easier for elderly to accept modern technologies, as they usually think it requires knowledge of controlling any smart devices. Therefore, Wrist watches are the perfect and easiest approach to collect the data application need and act as a safety alert system to the elderly. To ease the difficulty in hardware production, we decided to develop on the Android Wear manufactured by Motorola during the project.

Android wear (Moto 360) which is an Android powered wearable device and required to connecting with an android phone via Bluetooth. According to the data from Motorola the specification are listed below:

Moto 360 is assumed wearing on the elderly left wrist, while elderly got movement; our application will start to collect motion data from accelerometer and doing motion analysis at the same time.

Android Phone/Tablet

Currently most android wear does not have internet connection ability, required to connect an android device as the bridge of the internet connection. Also, the system need to response to the server at real time. Due to this problem, we use the android mobile device to connect the TCP Socket Server for returning the Elderly status in JSON format to the Message Receiver Side via the mobile network or Wi-Fi network.

Webcam / Kinect

Since the system will include the feature of real time home view for the care taker in the Message Receiver Side. When the care taker received the elderly fallen notification, then he/she can take a look on the elderly real time image to check the elderly current condition. Currently, we use Kinect as a webcam only. In the future, the Kinect may consider as the device that for motion detection. The situation of the connection between PC and webcam are illustrated at middle of the figure xx that surrounding with red and yellow rectangle.

PC (Server Side)

PC is used for connecting the Kinect represent as a bridge of the TCP Socket connection to returning the real time image in JSON format to the Message Receiver Side via the mobile network or Wi-Fi network.

Main Server

The main server is reserved for receiving the data from Data Collection and Message Receiver and return the suitable data to Message Receiver Side, which is used for push notification for the Message Receiver Side built by the TCP Socket Server with Multithreading that can accept multiple connection written in python. After receiving the data from Data Collection Side, the server will check the data a consistent or not and do decision making, return the data to the right algorithms.

The application collects data from elderly’s daily activities to detect their fall signal. Since we required to develop the application for collects data from elderly’s daily activities to detect their fall signal, we have chosen Moto 360 as our data input source. Moto360 is running on Android wear operation system, which provides different kind of API for developer to develop third party application for wearable devices.

The following table summarizes the motion sensors that are available on the Android platform [2] For the case of our application, we only use the hardware-based sensor, Accelerometer to apply on our application. (The sensor that highlighted in green in following table)

The classifier requires input motion data on every second, it consumes the battery quite a lot. As most of the Android Wearable device does not come with large capacity of battery, the application consumes around 1% of battery in every 2 minutes. Moreover, it is not feasible to slow down the application to decelerate the use of battery, as the application could not analysis the motion data in real-time manner. After a few trial and error and look up in API manual, we discovered that there were 4 kinds of sampling rate in SensorManager section, the related information are listed below:

To detect a short period of fall signal, a multilayer perceptron neural network is a commonly used ANN structure was selected as the most powerful classification algorithm for precise classification. The input layer consists of 28 neurons which represent the number of features that have been extracted on each sliding window. The proposed neural network was trained for 20 times using back-propagation algorithm as the learning technique. The training process is finding the optimal weight between each neuron.

The classifier multiply with each input feature with the weight trained by back-propagation through the hidden layer, and determine whether the output result is below or above the threshold. The output of the neural network has 3 possible states, “Fall”, “Normal” and “Potential Fall”.

To minimize the situation of outputting “Potential Fall”, Dynamic Time Warping (DTW) is implemented to recognize a motion before the neural network detects a fall signal. The purpose of DTW was to minimize the occurrence of “Potential Fall” situation, while DTW could recognize whether the elderly is performing a certain activity. Therefore, the result of the classifier will be more promising.

The alignment of measurements by DTW for measuring the distance between two sequences and the DTW obtains a mapping between the sequences. The black squares denote the optimum warping path.

Evaluation

In this project, we have tested our application for 200 hundred times. We have invited 4 persons to simulate the situation with our application. The test included 2 different situations, which are performing daily activities and fall simulation. The fall simulation includes free fall, forward fall, backward fall, fall to left and fall to right. These 2 test cases were also separate performed with and without the dynamic time warping process. There were total 40 tests performed. The test was aimed to determine whether the wearable device is able to detect fall signal from the subject and the result of the wearable device handling different fall simulation. Classification result is considered to be positive if the classifier properly recognizes the fall signal and to be negative if it does not. Therefore, there are four cases: 1. True positive (TP): A fall occurs, the wearable device detects it. 2. True negative (TN): Non-fall occurs, the wearable device does not detect a fall. 3. False positive (FP): Non-fall occurs, the wearable device detects a fall. 4. False negative (FN): Fall occurs, the wearable device does not detect a fall. The test result shows that the application has an 8% error rate overall. After integrated with DTW, the error rate decreases 1%, which is an improvement. The application successfully detects fall signal from the subject. However, the occurrence of False Positive cases were caused by changing posture of wrist, when the subject moved the wrist and stopped for the next posture, the application consider it as being unconscious after a sudden movement. The results of adding DTW process did not improved the accuracy of detection. However, it helps reduce the false alarm of the application.
Neural NetworkNeural Net + DTW
TRUEFALSETRUEFALSE
Case 1 Positive103103
Negative7070
Case 2 Positive100100
Negative100100
Case 3 Positive102103
Negative8070
Case 4 Positive6160
Negative94104
Case 5 Positive5272
Negative8583
Neural NetworkNeural Net + DTW
Type I errorType II errorType I errorType II error
Case 115%0%15%0%
Case 20%0%0%0%
Case 310%0%15%0%
Case 45%20%0%20%
Case 510%25%10%15%
Mean error rate17%15%
Total error rate8.50%7.50%
Besides testing of the application, we have also conducted a survey about user experience of our application. There are total 14 interviewees accepted our survey. (9 Male & 5 Female). The results of our survey are listed below: It helps me be more effective on take care the elderly.
  • 50% Strongly Agree
  • 50% Agree
The elderly does not require any knowledge in the use of electronics.
  • 36% Strongly Agree
  • 50% Agree
  • 14% Disagree
The elderly would love to use this product.
  • 36% Strongly agree
  • 50% Agree
  • 14% Disagree
The application will not broke down when I mis-clicked.
  • 36% Strongly Agree
  • 57% Agree
  • 7% Disagree
The product is easy to use.
  • 57% Strong Agree
  • 43% Agree
The product fit my needs.
  • 50% Strongly Agree
  • 43% Agree
  • 7% Disagree
I am satisfied with the product.
  • 50% Strongly Agree
  • 50% Agree
I want this product.
  • 50% Strongly Agree
  • 43% Agree
  • 7% Disagree

Conclusion and Future Development

Overall the classifier is quite successful, since it fulfills the needs of detecting the elderly fall signal using their motion data. In most of the case, it could able to alert users when there is fall pattern in the motion data. However, there are still have space to improve a lot. The further explanation will describes improvement in the later section.

Most users satisfied with the design of the application, especially the simplicity of the application, it could easily to carry and operate with both elderly and family members. Also, most of them agreed that the application did not requires a lot of extra equipment to achieve the benefits of ensuring elderly’s safety.

Although the application meets the requirement of users, there are still have few drawbacks can be improved. First, DTW process can detect more different activity by increasing the window size. Currently the DTW process only input fixed number of data-points as a single input, which limit the variety of motion, as some movement may need larger window size to stores. Moreover, adding more templates to increase the accuracy of motion detection. As the template amount increases, the DTW process could handle more motion data from person with different characteristics such as walking pattern, body joint movement.

Copyright Wong Yiu Hong, Li Kin Lun, Poon Ka Ho and Jeff Tang 2015

Jonathan Chiu
Marketing Director
3DP Technology Limited

Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.

Krutz Cheuk
Biomedical Engineer
Hong Kong Sanatorium & Hospital

After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.

Hugo Leung
Software and Hardware Engineer
Innovation Team Company Limited

Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.

The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.

Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.

Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.

Course CodeTitleCredits
 COMP S321FAdvanced Database and Data Warehousing5
 COMP S333FAdvanced Programming and AI Algorithms5
 COMP S351FSoftware Project Management5
 COMP S362FConcurrent and Network Programming5
 COMP S363FDistributed Systems and Parallel Computing5
 COMP S382FData Mining and Analytics5
 COMP S390FCreative Programming for Games5
 COMP S492FMachine Learning5
 ELEC S305FComputer Networking5
 ELEC S348FIOT Security5
 ELEC S371FDigital Forensics5
 ELEC S431FBlockchain Technologies5
 ELEC S425FComputer and Network Security5
 Course CodeTitleCredits
 ELEC S201FBasic Electronics5
 IT S290FHuman Computer Interaction & User Experience Design5
 STAT S251FStatistical Data Analysis5
 Course CodeTitleCredits
 COMPS333FAdvanced Programming and AI Algorithms5
 COMPS362FConcurrent and Network Programming5
 COMPS363FDistributed Systems and Parallel Computing5
 COMPS380FWeb Applications: Design and Development5
 COMPS381FServer-side Technologies and Cloud Computing5
 COMPS382FData Mining and Analytics5
 COMPS390FCreative Programming for Games5
 COMPS413FApplication Design and Development for Mobile Devices5
 COMPS492FMachine Learning5
 ELECS305FComputer Networking5
 ELECS363FAdvanced Computer Design5
 ELECS425FComputer and Network Security5