School of Science and Technology 科技學院
Computing Programmes 電腦學系

ClassS: A Mobile Application for Facilitating Teaching and Learning in Smart Classroom

Hong Wing CHAN, Gordon Tsz Kwan CHAN, Samson Ming Chun CHUK

ProgrammeBachelor of Computing with Honours in Internet Technology
SupervisorProf. Vanessa NG
AreasInnovative Systems
Year of Completion2019

Objectives

The project aims at facilitating teaching and learning in a smart learning environment by developing a roll-call system and an educational mobile application, enhancing the reliability of attendance list.

The traditional method of making roll-call by teacher is time-consuming and has an impact on the use of lesson time. Moreover, attendance taken by students may disturb students’ learning, and the attendance list may not be credible. Besides, teachers’ workloads will be increased for the administrative tasks to be done after class.

With the help of our roll-call system, attendance can be taken once student have arrived at classroom for generating an accurate attendance list. Also, students can use the mobile application to participate in class activities more. This can increase student class participation and create more chances of interactions between teachers and students.

In the name of facilitating a smart campus, the proposed educational application’s objectives are sixfold:

  • Analyse data to find students’ facial patterns.
  • Design and develop a sensor to detect student’s identity with vision.
  • Design and develop a learning algorithm for facial recognition usage.
  • Design a communication platform for more in-class interaction.
  • Develop a prototyping smart educational application based on the above.
  • Evaluate new application against current capacity.

Video Demonstration

Background and Methodology

Techniques and technologies used

Facial Recognition

Facial Recognition is a technology to find out human facial characteristics and shapes in an image or a video and recognize the identity of a person.

Figure 1: Facial Recognition Sample

Positioning

Positioning is used to calculate the position of an object in Space. Global Positioning System (GPS) is a famous positioning system to apply outdoor positioning.

Figure 2: Global Positioning System (GPS)

Wi-Fi positioning

Wi-Fi positioning system is a geolocation system to find out the locations of devices by connecting to nearby Wi-Fi hotspots and other wireless access points.

Figure 3: Indoor Wi-Fi Positioning

System architecture

Roll-Call System

Create facial data set

During the registration process, students are required to take 3 available photos.

Roll-call by facial recognition

The surveillance camera can keep detecting faces, and the system will send the image to Face Recognition API for analyzing. Students’ ID will then be returned, and the attendance record will be updated.

Roll-call by Wi-Fi positioning

The roll-call request will be sent if the mobile is connected to campus Wi-Fi, then the server will update the attendance record.

Smart Interacting System

Students can complete lab exercises prepared by teachers. Teachers can manage attendance, post exercises, and provide evaluations and comments in distributed exercises.

Wish Tree

Students can submit questions or opinions to teachers for further explanation or improvement.

System Design and Implementation

Create facial data set

During the registration process, students are required to take 3 available photos. After that, the application will detect the number of facial features in the photos.

Figure 5: The procedure of creating facial data set

Roll-call by facial recognition

The surveillance camera can keep detecting faces, and after cropping the image, the system will send the image to Microsoft Cognitive Services Face API for analyzing. Students’ ID will then be returned, and the attendance record in database will be updated.

Figure 6: The procedure of roll-call by facial recognition

Raspberry Pi is used as the surveillance camera. Raspberry Pi will process captured photos and send them to the server. In order to implement facial recognition technology, the combination between different extension modules and the Raspberry Pi computer is needed.

Figure 7: Raspberry Pi camera

When students first login, they need to take 3 valid photos by in-app function or select from photo library to create a facial data set. The application will authenticate the photos whether they belong to the same person.

Figure 8: Login page

Figure 9: Creating facial data set

After logging in, it will display students’ locations and their subjects. With the help of facial recognition system, it processes the automatic roll-call and then releases confirmation notification. The attendance list is visible in the teacher’s account which lights up the present students’ icons after authentication.

Figure 10: Student’s Home page

Figure 11: Face detection in the classroom

Figure 12: Teacher’s attendance list page

When students press “subject”, related exercises will be listed, including the comment and the status of the exercises. Students can select the target exercise for completion. When finished, the status of the exercise will be updated as “done”.

Figure 13: Student’s Exercises List page

Figure 14: Exercises page

Evaluation

In user evaluations on our roll-call system, all respondents were satisfied with our roll-call system on its efficiency comparing to using attendance list. They believed our roll-call system could avoid proxy attendance and greatly reduced management workload.

Figure 15: Usefulness on Roll-call System

Some of the respondents felt that our interactive system could provide more chances for them to communicate with lecturers or students. Most of the respondents agreed with the facilitation of communication between teachers and students.

Figure 16: Usefulness on Interactive System

Conclusion and Future Development

In the future, a Student-Side Model Training System will be designed to enhance the accuracy of face recognition.

Also, a Raspberry Pi camera setting plan should be made. It includes the consideration of the positions of multi-Raspberry Pi cameras to fit different sizes of lecture rooms.

Moreover, apart from the common short questions in the system, more detailed lab tests for professional subjects through interactive-multimedia, like 3D Biological Organs, can be included in the system.

Jonathan Chiu
Marketing Director
3DP Technology Limited

Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.

Krutz Cheuk
Biomedical Engineer
Hong Kong Sanatorium & Hospital

After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.

Hugo Leung
Software and Hardware Engineer
Innovation Team Company Limited

Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.

The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.

Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.

Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.

Course CodeTitleCredits
 COMP S321FAdvanced Database and Data Warehousing5
 COMP S333FAdvanced Programming and AI Algorithms5
 COMP S351FSoftware Project Management5
 COMP S362FConcurrent and Network Programming5
 COMP S363FDistributed Systems and Parallel Computing5
 COMP S382FData Mining and Analytics5
 COMP S390FCreative Programming for Games5
 COMP S492FMachine Learning5
 ELEC S305FComputer Networking5
 ELEC S348FIOT Security5
 ELEC S371FDigital Forensics5
 ELEC S431FBlockchain Technologies5
 ELEC S425FComputer and Network Security5
 Course CodeTitleCredits
 ELEC S201FBasic Electronics5
 IT S290FHuman Computer Interaction & User Experience Design5
 STAT S251FStatistical Data Analysis5
 Course CodeTitleCredits
 COMPS333FAdvanced Programming and AI Algorithms5
 COMPS362FConcurrent and Network Programming5
 COMPS363FDistributed Systems and Parallel Computing5
 COMPS380FWeb Applications: Design and Development5
 COMPS381FServer-side Technologies and Cloud Computing5
 COMPS382FData Mining and Analytics5
 COMPS390FCreative Programming for Games5
 COMPS413FApplication Design and Development for Mobile Devices5
 COMPS492FMachine Learning5
 ELECS305FComputer Networking5
 ELECS363FAdvanced Computer Design5
 ELECS425FComputer and Network Security5