School of Science and Technology 科技學院
Computing Programmes 電腦學系

YOLO-Enhanced Care Label Reader for Clothing Care and Maintenance 

Tse Long Fung Ivan, Chan Hau Wing Harriet, Cheng Tsz Tsun Issac, Cheng Wai Kit Jacky

ProgrammeBachelor of Computing with Honours in Internet Technology

Bachelor of Science with Honours in Computer Science
SupervisorDr. Roy Li
AreasIntelligent Applications
Year of Completion2024

Objectives

Project Aim 

The project aims to address the negative impacts of textile waste by providing a proactive solution that extends the lifespan of clothing items. To achieve this, an Android application incorporating a care label reader and a smart wardrobe feature will be developed. 

Project Objectives 

High-Precision Care Label Recognition 

  • Train a model using a dataset of 10,000+ images (≥100 per class) based on the GINETEX standard. 
  • Target >80% mAP50 and >90% real-world accuracy in recognizing garment care labels. 

Model Accuracy & Reliability Enhancement 

  • Evaluate model with 100 real-world tests. 
  • Apply refinements to maintain or exceed 90% real-use accuracy.

Result Categorization Algorithm 

  • Develop logic to group care label outputs. 

Suggest laundry instructions (e.g., what can be washed together) to help users care for garments and extend clothing lifespan. 

Clothing Registration Interface 

  • Design a user-friendly input system where users scan care labels and manually enter details like color, type, or season. 
  • Allows users to build a personalized clothing inventory. 

Inventory Management System 

  • Create tools for users to manage and view their wardrobe digitally, improving outfit planning and item care. 

User Experience Optimization 

  • Collect prototype feedback to refine design, usability, and satisfaction. 

Apply findings to align the app with user expectations and behaviours 

Videos

Demonstration Video

Presentation Video

Methodologies and Technologies used

Frontend:

  • Built using React Native to deliver a responsive and user-friendly mobile interface. 

Backend:

  • Combines a Node.js HTTP server with a Python script to deploy the trained model. Hosted on AWS EC2, ensuring scalability and stable performance. 

Database:

  • Using MongoDB, a flexible NoSQL database, to manage user accounts and garment registration data efficiently and with minimal setup complexity. 

AI Model:

  • Implements a YOLOv8-based multilabel classification model trained via Ultralytics’ SDK and fine-tuned with a custom dataset for care label recognition. 

Hardware:

  • An Android phone is used for real-device testing to assess compatibility, performance, and user interaction. 
  • Local Development: Visual Studio Code is the main IDE, leveraging port forwarding and Expo Go to simplify cross-platform mobile development and streamline testing. 
Key Goals: 
  • Enhance the precision and relevance of AI-generated answers for vocal training. 
  • Support a wide range of users—from beginners to professional singers. 
  • Ensure scalability across sub-domains like classical singing, modern styles, and vocal health. 

The chapter introduces the architecture and technical methods supporting this solution, designed to handle singing-related queries more effectively through advanced natural language understanding. 

System Design 

Figure 1: High-Level System Design Diagram

Figure 2: Component Diagram showcasing the architecture of the application

Figure 3: Dataflow Diagram representing the flow of information of the application 

Figure 4: Use-Case Diagram illustrating the functionality of the application

Sign In 

Navigation & Input: 

  • User clicks the account icon → redirected to Sign-In page. 
  • Inputs email and password, then clicks “Sign In”. 

Validation Check: 

  • If fields are invalid or incomplete → show “Missing field” toast. 
  • If valid → send login API request with {email, password} to backend. 

Backend Processing: 

  • Server checks for the email in the database. 
  • If not found → respond with error, show toast: “No account found”.
  • If found → compare hashed input password with stored hash. 

Authentication Outcome: 

  • If hashes match → return success with user ID and username. 
  • If hashes don't match → return failure, show toast: “Login failed”. 

Final Response: 

  • Frontend shows toast: “Login Success”. 

Redirects user to User Account page. 

Figure 5: Activity diagram of Sign In page 

Image Preview  

Retrieve Image URI:  

  • After a photo is taken in the Camera activity, the image URI is passed to the Preview activity. 

User Confirmation: 

  • If the user wants to retake, the app navigates back to the Camera activity. 
  • If the user confirms, the image is: 
  • Encoded to Base64 
  • Sent to the backend for processing (e.g., label recognition or storage) 

Post-Processing: 

  • Once backend handling is complete, the frontend navigates to the Add Garment activity to continue the registration process with the processed image data

Figure 6: Activity diagram of Image Preview page

Data Retrieval

  • On entry, the page loads a list of registered garments from MongoDB, fetched via a backend API. 

Item Selection: 

  • Each garment is displayed with a checkbox. 
  • Users select the laundry items they wish to process. 
  • Item Confirmation Modal: 
  • Upon clicking an icon, a Modal pops up displaying: Care label, Color, Image of selected items 

Options: 

  • Next → proceeds to washing suggestions 
  • Close → returns to main page 
  • Washing Suggestion Modal: 
  • Selected item details are sent to the backend 
  • Washing Suggestion Algorithm. 
  • A second Modal shows tailored washing instructions. 

Options: 

  • Back → returns to confirmation Modal 
  • Close → exits to main page 
  • Navigation Handling: 
  • If the user switches to another feature, the Laundry Care page is closed, and they are redirected accordingly. 

Figure 7: Activity diagram of Laundry Care page

Module Design and Hierarchy 

Figure 8: Hierarchy of frontend functions 

Results (Prototype System Design) 

Sign in and Sign-Up Page  

Wardrobe Page 

Figure 9: UI design of Sign in and Sign-Up page

Figure 10: UI design of Sign in and Sign-Up page

Main UI 

Garment Register Page

Laundry Care Page 

Figure 11: UI design of main page

Figure 12: UI design of Garment Register page

Figure 13: UI design of Laundry Care page 

Camera and Image Preview Page

Label Scanning Camera Page 

Figure 14: UI design of Camera and Image Preview page

Figure 15: UI design of Label Scanning Camera page

Conclusion

The project successfully implemented a Retrieval-Augmented Generation (RAG) system powered by LLaMA-2 and enhanced with Semantic Textual Similarity (STS) to provide accurate, context-aware answers within the singing domain. Key achievements included: 

  • Building a specialized knowledge base. 
  • Fine-tuning LLaMA-2 for domain relevance. 
  • Enabling efficient content retrieval and generating high-quality responses. 
  • Establishing and meeting robust evaluation benchmarks. 

Limitations 

  • The knowledge base, while rich, lacked full coverage for rare or complex singing topics. 
  • The system was unable to handle multi-turn dialogue, limiting conversational continuity. 
  • Use of top-1 context retrieval restricted the model’s response depth. 

Future Development

  • Shift to a top-k retrieval approach to synthesize insights from multiple sources. 
  • Expand the knowledge base to include wider and deeper content across all singing subfields. 
  • Improve context management for smoother, ongoing conversations. 
Jonathan Chiu
Marketing Director
3DP Technology Limited

Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.

Krutz Cheuk
Biomedical Engineer
Hong Kong Sanatorium & Hospital

After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.

Hugo Leung
Software and Hardware Engineer
Innovation Team Company Limited

Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.

The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.

Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.

Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.

Course Code Title Credits
  COMP S321F Advanced Database and Data Warehousing 5
  COMP S333F Advanced Programming and AI Algorithms 5
  COMP S351F Software Project Management 5
  COMP S362F Concurrent and Network Programming 5
  COMP S363F Distributed Systems and Parallel Computing 5
  COMP S382F Data Mining and Analytics 5
  COMP S390F Creative Programming for Games 5
  COMP S492F Machine Learning 5
  ELEC S305F Computer Networking 5
  ELEC S348F IOT Security 5
  ELEC S371F Digital Forensics 5
  ELEC S431F Blockchain Technologies 5
  ELEC S425F Computer and Network Security 5
 Course CodeTitleCredits
 ELEC S201FBasic Electronics5
 IT S290FHuman Computer Interaction & User Experience Design5
 STAT S251FStatistical Data Analysis5
 Course CodeTitleCredits
 COMPS333FAdvanced Programming and AI Algorithms5
 COMPS362FConcurrent and Network Programming5
 COMPS363FDistributed Systems and Parallel Computing5
 COMPS380FWeb Applications: Design and Development5
 COMPS381FServer-side Technologies and Cloud Computing5
 COMPS382FData Mining and Analytics5
 COMPS390FCreative Programming for Games5
 COMPS413FApplication Design and Development for Mobile Devices5
 COMPS492FMachine Learning5
 ELECS305FComputer Networking5
 ELECS363FAdvanced Computer Design5
 ELECS425FComputer and Network Security5