School of Science and Technology 科技學院
Computing Programmes 電腦學系
| Programme | Bachelor of Computing with Honours in Internet Technology Bachelor of Science with Honours in Computer Science |
| Supervisor | Dr. Jeff Au Yeung Siu Kei |
| Areas | Intelligent Applications |
| Year of Completion | 2024 |
This project aims to enhance the accessibility of Cantonese content while providing convenience to both the content creator and audience by creating a platform for automating transcription and Cantonese-to-written Chinese translation from video input.
Our objective is to develop a Cantonese-to-Chinese auto-subtitling tool, which specifically provides written Chinese subtitles for Cantonese movies and videos. The application will be a web application and the application can be mainly divided into transcription and translation.
Demonstration Video
Presentation Video
The technologies we used for this project were included here, many of these technologies are third-party Python libraries.
Figure 1: Use case diagram
Single-page website design used for clarity and ease of use.
Caters to both standard and experienced users:
Advanced settings have default values:
Uploaded video file is the main data source:
System data flow is represented with a supporting diagram.
Figure 2: User flow diagram representation of our system.
Uploaded video is the primary data source.
Temporarily stored and deleted after processing and user disconnection.
Data flow process:
Figure 3: Data flow diagram representation of the system.
Training methodology
Dataset:
Training Process:
Model Evaluation:
Objective: Assess overall application performance (transcription + translation).
Method:
User Evaluation
Objective: Evaluate user experience.
Method:
Experiment
Objective: Assess machine translation performance.
Model Used: Fine-tuned BART-Cantonese model.
Method:
Processing Time Evaluation:
The system’s total processing time is tested using a range of Cantonese videos, both short (10 minutes). Two key outputs are timed:
User Evaluation:
Selected users—primarily native Mandarin speakers or Cantonese content creators—test the application. Their feedback is gathered via a questionnaire focusing on:
Translation Model Evaluation:
The BART-Cantonese model, fine-tuned with the team's dataset, is tested on subtitled videos. Evaluation includes:
Together, these assessments ensure the application is thoroughly tested for speed, usability, and translation accuracy.
The application is structured around five key functions, providing an intuitive user experience from video input to post-processing:
The design emphasizes ease of use and customization, supporting a seamless subtitling workflow for users.
System Processing Time
The team benchmarked the total time required to generate subtitles and preview videos across different setups:
Translation Model Performance
Three translation models were evaluated: Ayaka, indiejoseph, and fnlp, with results split between high and low similarity sample types.
No single model excelled in both; models trained longer (like Ayaka) handled low similarity (localized/slang-heavy) content better, while others fared better with high similarity (formal) content.
When compared to Azure, the final Ayaka model:
Ultimately, Ayaka was adopted as the final model due to its superior handling of informal and localized content, fulfilling the project's objective of high-quality Cantonese-to-written-Chinese subtitling.
The project successfully implemented a Retrieval-Augmented Generation (RAG) system powered by LLaMA-2 and enhanced with Semantic Textual Similarity (STS) to provide accurate, context-aware answers within the singing domain. Key achievements included:
Limitations
Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.
After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.
Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.
The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.
Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.
Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.
| Course Code | Title | Credits | |
|---|---|---|---|
| COMP S321F | Advanced Database and Data Warehousing | 5 | |
| COMP S333F | Advanced Programming and AI Algorithms | 5 | |
| COMP S351F | Software Project Management | 5 | |
| COMP S362F | Concurrent and Network Programming | 5 | |
| COMP S363F | Distributed Systems and Parallel Computing | 5 | |
| COMP S382F | Data Mining and Analytics | 5 | |
| COMP S390F | Creative Programming for Games | 5 | |
| COMP S492F | Machine Learning | 5 | |
| ELEC S305F | Computer Networking | 5 | |
| ELEC S348F | IOT Security | 5 | |
| ELEC S371F | Digital Forensics | 5 | |
| ELEC S431F | Blockchain Technologies | 5 | |
| ELEC S425F | Computer and Network Security | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| ELEC S201F | Basic Electronics | 5 | |
| IT S290F | Human Computer Interaction & User Experience Design | 5 | |
| STAT S251F | Statistical Data Analysis | 5 |
| Course Code | Title | Credits | |
|---|---|---|---|
| COMPS333F | Advanced Programming and AI Algorithms | 5 | |
| COMPS362F | Concurrent and Network Programming | 5 | |
| COMPS363F | Distributed Systems and Parallel Computing | 5 | |
| COMPS380F | Web Applications: Design and Development | 5 | |
| COMPS381F | Server-side Technologies and Cloud Computing | 5 | |
| COMPS382F | Data Mining and Analytics | 5 | |
| COMPS390F | Creative Programming for Games | 5 | |
| COMPS413F | Application Design and Development for Mobile Devices | 5 | |
| COMPS492F | Machine Learning | 5 | |
| ELECS305F | Computer Networking | 5 | |
| ELECS363F | Advanced Computer Design | 5 | |
| ELECS425F | Computer and Network Security | 5 |
