School of Arts and Social Sciences
From the poetry of the Tang dynasty to the modernist fiction of Eileen Chang, Chinese literature has the power to transport readers to past times and unfamiliar places. While the texts themselves are unchanging, every generation engages with writing from the past in a different way, informed by their own reality. The challenge for those teaching Chinese literature now is how to make it accessible and engaging for the current generation of students whose reality is saturated in digital technology and media.
For one team in the Hong Kong Metropolitan University (HKMU) School of Arts and Social Sciences, the answer was to embrace digital pedagogy, and use immersive, interactive digital installations to enhance their students' ability to step into a writer's world. In this way, as students engage with texts that can be challenging for a contemporary reader to understand, alongside their own reading, they can experience digital renderings of scenes from the stories, and interact with them using mixed, augmented, and virtual reality technology. In combination with artificial intelligence-powered Chinese language writing assessment tools, these immersive experiences are also helping students hone their own writing skills.
This approach became possible with the 2023 opening of the Digital Art Laboratory, hosted by the Department of Creative Arts. The 1,700 sq. ft. Space is equipped with 360-degree and 270-degree cave automatic virtual environment (known as CAVE) systems, using multiple projectors to create immersive scenes.
The launch of the lab coincided with the award of a HK$3 million grant for a three year research project, fully supported by the Sin Wai Kin Foundation. This funding was then matched by the Hong Kong Government's University Grants Committee under its Research Matching Grant Scheme, an endorsement of the project's potential impact. It was led by Rebecca Leung Mo Ling Associate Dean (Administration and Development) cum Associate Professor of the School of Arts and Social Sciences.
The aims of the project, Teaching and Learning in Digital Humanities: A Study of Enhancing the Experiences of Chinese Literature and Culture through Interactive Installations for Immersive Space, were twofold: to both enhance learning for the participating students. and to create the opportunity to assess the impact of virtual immersive experiences on teaching and learning.
Leung was supported by a strong team of four co-investigators who each brought their own skillsets. Department of Creative Arts assistant professors Walter Wong Shu Kei and Vincent Mak Shing Fung did the heavy lifting on the animation, arts strategies, character design, modelling, visual effects of the scene creation. Lee Lok Man, an assistant professor in the School of Arts and Social Sciences, brought his knowledge of Chinese literature and deep teaching experience, and the computer science aspect was led by William Lai Chi Fu, assistant professor in the Department of Creative Arts. Despite this strong team, the whole project would not have been possible with the students and alumni from Department of Creative Arts who participated in project production, including modelling, animation and calligraphy.
The team's source material came from four literary masterpieces and two dynastic periods: over 100 poems by Tao Yuanming on his reclusive, rural life; Shen Congwen's 1929 short novel Xiao Xiao (蕭蕭); Mu Shiying's 1941 short novel Five People in the Nightclub (夜總會裡的五個人); Eileen Chang's 1943 wartime fiction Love in a Fallen City (傾城之戀); and scenes from a Song dynasty city and daily life in the Han dynasty.
Donning mixed reality headsets, students enter Mu Shiying's nocturnal world, visit the five characters at their respective tables, seemingly close enough to touch, and hear their tales of woe, immersed in the authentic mood of a Shanghai nightclub. With a gaming-style console in hand, students stroll through Tao Yuanming's bucolic retreat, using modern technology to complete gamified tasks that connect them with Tao's love of birds, flowers and the gentle rhythms of farming life.
From the launch of the first experience in late 2024 to early 2026, almost 400 HKMU students, studying Chinese literature as part of their course requirements or simply as an academic enrichment option, have engaged with the six immersive experiences. Lee has seen a change in the way his Chinese literature students respond to the texts.
“I like to teach in a traditional way but I also work with technology experts, and can insert some new things here. When I was only teaching in the traditional way, even when I taught some very good poems, the students were not impressed by anything, they showed no reaction. Using these immersive experiences, we get a very good reaction in class, they give feedback, they show some emotional inspiration.”
The immersive experiences can help literature scholars go deeper in their understanding of the text, but they can also encourage people who might otherwise avoid literature entirely to start reading. The project relied on Lai's computer graphics skills. As someone who assiduously avoided Chinese literature and the classics when he was at school, and is an avowed “new media guy”, Lai was in a good position to understand how to engage students who might otherwise not give literature a second glance. He says it's been fun to see how words are being translated into different visual media.
“I enjoy movies, I enjoy gaming. I watch anime. I like comics. I’m an outsider here. My training has been in computer science throughout my entire education. You’re never going to get someone like me to go into a bookshop and pick up Chinese literature,” he says. In the pre-digital age, many students tried a literature class but dropped the class after a couple of weeks, but using digital experiences can bring them into the literary fold. “We have to use some new teaching and learning strategies to let them easily enter the world of the literature and culture, and see how beautiful Chinese literature is.”
The immersive experiences rely on the technical skills of computer scientists, but the content is meticulously validated by subject experts. Each of the experiences cleaves faithfully to the texts, and they are historically accurate in every domain, from lighting to sound and dress of the characters. The design process turned out to be highly iterative, with a lot of back and forth with subject experts, and was quite labour intensive too, especially early on when the artificial intelligence (AI) tools were less powerful than they are now. The Shanghai nightclub scene, for example, took three research assistants a year to create.
The goal of the project is not to showcase state of the art technology, but to demonstrate how to pick the right technology for the task at hand. “HKMU is an applied sciences university, not a pure sciences university, and how to apply what technology is important,” says Lai. “In applying, we need to go through very critical design thinking to pick the right solution to solve the right problem.”
Lee is quick to point out that AI and immersive experiences can't do the work for the students. Rather, they are a bridge between the students and their own understanding of the text.: “We have to define what humans can do, and what the tools such as AI can do. You can easily use AI to analyze a poem in one minute, but the most important thing when reading Chinese literature is that it's personal. It’s about the emotion. It’s about how you yourself to interact with characters. Tools like digital, immersive experiences serve as a kind of inspiration, but finally, you have to get back to the text, that is the most important thing.”
This philosophy speaks to the impact of the research that Leung and her team are doing, which is to demonstrate that digital tools are not supposed to make people lazy; rather, they can stimulate new ideas, inspirations and a new level of learning. “Like every generation, we use our contemporary hearts to interpret the classics,” says Lee.
The project has also created a framework for collaboration among experts in the fields of the arts, education, technology, animation, visual effects, AI, entertainment, and game design, and this framework can be adopted by many other fields in the arts and humanities, not just literature. Part of this framework is inviting experts to review the authenticity of the scenes. In this project, that expert critique and endorsement confirms the project's strong impact. Apart from building the framework, the project is also evaluating the effectiveness of immersive vs. traditional methods.
“We have conducted user studies whereby we invited more than 200 HKMU students to participate in different kinds of immersive experience,” explains Lee. “We have invited experts in the field of Chinese literature and also some secondary school teachers to come over and try the immersive experiences out and give comments. Often the feedback from teachers was: when can you design some for my school literature curriculum?”
As insights emerge from the team's research, they have been able to share it at numerous academic conferences, and in journal publications on individual literary figures such as Tao Yuanming and on the teaching concepts behind the immersive installations.
The original research project ended in 2026, but a second round of funding has enabled the same work to continue under a new research project. This should see more students than ever walking out of the Digital Art Lab inspired to read Chinese literature, and in the process help HKMU can establish a teaching brand known for its embrace of innovative education practices.
