School of Science and Technology 科技學院
Computing Programmes 電腦學系

A Robotic Chinese Chess(Xiangqi) Master

NG Pui Lam

ProgrammeBachelor of Computing with Honours in Computing
SupervisorDr. Li Tak Sing
AreasArtifical Intelligence and Robotics
Year of Completion2012


The aim of this project is to let elderly people playing Chinese Chess with autonomous robot, so that they can still enjoy the fun of the game when they find no people as their opponent.

The project mainly focuses on building an autonomous robot. To achieve this aim, I have defined a number of objectives as follows:

  • The design and implementation of mechanical arm. The mechanical arm receives a signal from processor and moves chess token from one grid to another on chessboard.
  • The design and implementation of a chessboard sensor. The sensor has the ability to recognize the status of chessboard. After the player finishing his turn, it detects the movement of the chess and sends data to processor for manipulation.
  • The design and implementation of an intelligent chess-playing system. The system is responsible for finding out the most beneficial choice base on the current chesses positions on a chessboard.
  • An evaluation implementation. That evaluates weather the project let elderly people play the chess in a concrete way and enjoy the fun in playing chess with the robot.

Background and Methodology

Chinese Chess game is a very popular strategic game in Chinese history. It’s origin can be traced from BC. As we seen, many retired elder love to get together in garden. What they most do are exercising and playing Chinese Chess boardgame. It is one the most favorite activities among elderly people. In Hong Kong, elderly people, especially those with illness, usually live in a boring life. They may live alone or stay alone for long time a day. It is hard to find someone as their opponent for playing Chinese chess boardgame. Although there are many software games allowing people to beat computer, most of the elderly people feel unfamiliar with using it and hard to enjoy the fun. As we seen, a proper user interface is significant to the users themselves. In short, many elder neither carry out Chinese chess game because of loneliness and they cannot benefit from the general software chess game. Therefore, we deliver an autonomous chess playing robot to answer their need.

There is a camera shooting at conventional chessboard. PC accesses the image and works out the status by extracting information from photos. A program can understand the choice of player and play software chess game. After the software chess game has made its choice, the program can recognize its turn by screen capture and instruct the robot to move the target chess token on the conventional chessboard. As a result, the robot can play against player with intelligence. The methodology is shown below:

The following describes the main steps in recognizing the status on both chessboards through the camera for the chess robot:

  • Determining the size and location of the chessboard, in which each corner of the chessboard is marked with a black dot. In RGB model, black color has a very low RGB values. Hence, the black points can be searched out by this characteristic. The size of chessboard can be calculated from the x, y position of them.
  • Calculating each relative position on chessboard, since the Chinese Chessboard is in rectangle shape. It is beneficial to finding out every chess position of it. The four black point locations can act as the reference point in calculating the other relative chess position.
  • Finding out the pixel pattern of each chess position, after working out all the relative chess positions, we can analyze the pixel pattern around each of them. Green chess tokens give a lot of green color pixel with relatively high G value in RGB. Red chess tokens are characterized with relatively high R value. While majority of white color implies no chess token there.

Chinese chess tokens appear in short Cylinder shape which is not beneficial to holding it. So, magnet is adopted in my method. A magnet is inserted in each of the chess tokens. Moreover, there is a wheel, in which one side attached with a magnet, at the end of the mechanical arm. Once the robot is instructed to hold up a chess token, the wheel rotates the magnetic side toward the surface of it. By the exiting of magnetic field, they attach together. In addition, there is a piece of plastic in the middle of them for avoiding the direct in touch. After moving to the desired position, the wheel turns to the non-magnetic side. So that, the chess token is released with the magnetic force disappearing. The following shows an illustration of picking up and down a chess:

The system consists of three main components, they are Moving arm robot, Recognition System and Communication Interface. Moving arm robot is to achieve the task of moving chess token on conventional chessboard. Recognition System allow the system understand the status of both side chessboards. Communication Interface acts as the middle media to synchronize both sides. The system overview is shown below:


In building the Chinese Chess Robot, I prefer LEGO Mindstorms robotic kit as the hardware platform. It allows developers to enjoy the advantages of fast prototyping, flexibility in the robot and easy integration of new hardware. Additionally, it qualified with Bluetooth module, numerous sensors and sufficient output devices (motor, LCD screen, loudspeaker etc.).

NXG is the standard development tool of NXT Brick. For building a large and complex project, we should take the capability and nature of development kits in consideration. Among the various third-party development kits, leJOS is my software developing platform with using Java as the language. The kits provide a set of Java library and the API of leJOS itself. They give all round support to the software development especially the Bluetooth connection.

The Moving Arm Robot consists of 2 NXT bricks, mNXT and sNXT. mNXT connected with 3 motors and 1 touch sensor. sNXT controls 2 motors and 1 touch sensor. mNXT plays the role of master between them. It receives instruction from PC and orders sNXT directly. It controls the mechanical part for picking up and putting down chess tokens. It also determines the robot movement across chessboard. sNXT is the slave of mNXT. It allows the robot moving along chessboard according to the order from mNXT. The overall infrastructure is shown below:

The PC tells mNXT which chess is moved, while mNXT informs sNXT how long the distance to move. Therefore, communications are necessary among them. Bluetooth is used as the method. It involves initiator and receiver in setting up the connection. Initiator requests the connection actively, while receiver is waiting for the request. There are two connections, PC and mNXT as well as mNXT and sNXT. The former is completed before the latter. PC acts as initiator which finds out mNXT and requests the connection. On the other side, mNXT plays the role of initiator and sNXT acts as receiver. It takes 5 seconds in total.

Position is represented by x and y values. Such that each position is represented by two integers and PC sends four integers mNXT each time. When mNXT receives those data, it works with sNXT. After finishing the work successfully, mNXT informs PC the completion by sending “done”.

sNXT is responsible for moving the robot along y-axis. mNXT sends a integer to sNXT. It rotates the motor according to the integer. When sNXT finishes it’s movement, “done” is sent to inform mNXT. Then, mNXT performs picking up or putting down chess token. The following shows the key communication interface:

There is a camera to shoot at the conventional chessboard. There are six references for locating its position. An algorithm finds out all position of chessboard and determines the existence of red or green chess tokens. This recognition method is only able to distinguish red and green chess token. For withstanding the geometrical error, the calculation is capable to handle certain level of distortion. The following shows an illustration:

There is a 1.3 million pixel camera connected with PC in USB. With the help of JMF, we can access the image from camera. The resolution of the image is 640 x 360 pixels. This is the highest resolution of using JMF. The conventional chessboard is slightly different. There are six reference points marked at the four corners, middle top and middle bottom. Each grid length and width is 40mm and 35mm respectively. There are 9 x 8 grids. So the length and the width of the chessboard are 36cm and 28cm. The following shows an illustration:

Before starting the game, we need to help the system recognize the position of chessboard in photo. We capture a photo from camera and use mouse cursor to tag the six reference points manually. Each reference point is composed by 3 x 3 pixels. Each pixel contains a special RGB value with R = 2, G = 3, B = 4. There is an algorithm which looks for a pixel with this value and surrounded by this value pixels. As a result, the system finds out the location of those six reference points. The following shows an illustration:

The communication interface has to recognize the status of both conventional and software chessboard. In our method, we extract information from photo. The following shows an illustration:

We make comparison between the system chess array and input photo. The value of (5, 7) is 0 in input photo while it is 2 in system side. This means the (5, 7) chess token is moved. In the Input photo chess array, (5, 2) is 2 while that of system chess array is 0. We can acknowledge the chess (5, 7) is moved to (5, 6) with this method. The following shows an illustration:


To increase accuracy, as the chessboard is made from a 9 x 8 grid with each grid is 4cm x 3.5cm. Motors rotate 82 degrees and 111 degrees for each grid along and across chessboard respectively. However, after several times of movement, position error may happen. There are two methods to keep the balance and correct position of the robot.

The frist method is by moving tracks, there are two tracks on both sides. The wheels of the robot move along them to limit the movement. The following shows an illustration:

The second method is by reseting position. Robot carries out position reset every three turns. It keeps moving backward slowly. Two touch sensors are on both left and right sides. When one side reaches at an obstacle, the touch sensor is pressed and the corresponding side motor stops its movement. The following shows an illustration:

Geometrical error is one of the biggest hindrances. This error results in the distortion of conventional chessboard in the photo. Based on our experiments, we found that six reference points can perform better than four. An example of the photo distortion is shown below:

In the player turns, there are many unwanted disturbance such as waving hands in front of camera, any invalid move breaking the game rules can cause the system suffer. Therefore, it should ignore those invalid input photo by checking them. Those problems mainly happened in player turn. There are several checking for input photo from camera. The details are described in the following:

  • Two Same Input Photos: In player turn, player has to move chess token on chessboard. Their hands are taken into photo inevitably. This usually affects the recognition result. This problem can be filtered by checking the result of two input photos. One is taken before another by one second. When they are same, the checking is passed.
  • Total Number Checking: It checks whether the number of chess tokens reasonable or not. The number of red chess tokens should remain same while that of green chess tokens should not be equal or one less than the original.
  • Handle Invalid Move: Since recognition system can only distinguish green and red chess token, it cannot really check out whether the status of the chess token can act this move or not. However, the pc software chess game is equipped with this ability. It is able to do the checking and gives out no respond if the moving is not valid. The communication interface still controls the mouse cursor to move the target chess according to the input photo. If the software chessboard keeps unchanged, we can acknowledge this move is invalid and request the player make the other attempt.

Conclusion and Future Development

The Robot System overall achieve success and I believe this can deliver good experience to elder. However, there are rooms of improvement. With the use of OCR, the system can be much more reliable. The PC system can be replaced by Android base device. This can become simpler.

In the future, robot will engage in every part of our life and I hope this project can make contributions to robot development.

Copyright Ng Pui Lam and Li Tak Sing 2012

Jonathan Chiu
Marketing Director
3DP Technology Limited

Jonathan handles all external affairs include business development, patents write up and public relations. He is frequently interviewed by media and is considered a pioneer in 3D printing products.

Krutz Cheuk
Biomedical Engineer
Hong Kong Sanatorium & Hospital

After graduating from OUHK, Krutz obtained an M.Sc. in Engineering Management from CityU. He is now completing his second master degree, M.Sc. in Biomedical Engineering, at CUHK. Krutz has a wide range of working experience. He has been with Siemens, VTech, and PCCW.

Hugo Leung
Software and Hardware Engineer
Innovation Team Company Limited

Hugo Leung Wai-yin, who graduated from his four-year programme in 2015, won the Best Paper Award for his ‘intelligent pill-dispenser’ design at the Institute of Electrical and Electronics Engineering’s International Conference on Consumer Electronics – China 2015.

The pill-dispenser alerts patients via sound and LED flashes to pre-set dosage and time intervals. Unlike units currently on the market, Hugo’s design connects to any mobile phone globally. In explaining how it works, he said: ‘There are three layers in the portable pillbox. The lowest level is a controller with various devices which can be connected to mobile phones in remote locations. Patients are alerted by a sound alarm and flashes. Should they fail to follow their prescribed regime, data can be sent via SMS to relatives and friends for follow up.’ The pill-dispenser has four medicine slots, plus a back-up with a LED alert, topped by a 500ml water bottle. It took Hugo three months of research and coding to complete his design, but he feels it was worth all his time and effort.

Hugo’s public examination results were disappointing and he was at a loss about his future before enrolling at the OUHK, which he now realizes was a major turning point in his life. He is grateful for the OUHK’s learning environment, its industry links and the positive guidance and encouragement from his teachers. The University is now exploring the commercial potential of his design with a pharmaceutical company. He hopes that this will benefit the elderly and chronically ill, as well as the society at large.

Soon after completing his studies, Hugo joined an automation technology company as an assistant engineer. He is responsible for the design and development of automation devices. The target is to minimize human labor and increase the quality of products. He is developing products which are used in various sections, including healthcare, manufacturing and consumer electronics.

Course CodeTitleCredits
 COMP S321FAdvanced Database and Data Warehousing5
 COMP S333FAdvanced Programming and AI Algorithms5
 COMP S351FSoftware Project Management5
 COMP S362FConcurrent and Network Programming5
 COMP S363FDistributed Systems and Parallel Computing5
 COMP S382FData Mining and Analytics5
 COMP S390FCreative Programming for Games5
 COMP S492FMachine Learning5
 ELEC S305FComputer Networking5
 ELEC S348FIOT Security5
 ELEC S371FDigital Forensics5
 ELEC S431FBlockchain Technologies5
 ELEC S425FComputer and Network Security5
 Course CodeTitleCredits
 ELEC S201FBasic Electronics5
 IT S290FHuman Computer Interaction & User Experience Design5
 STAT S251FStatistical Data Analysis5
 Course CodeTitleCredits
 COMPS333FAdvanced Programming and AI Algorithms5
 COMPS362FConcurrent and Network Programming5
 COMPS363FDistributed Systems and Parallel Computing5
 COMPS380FWeb Applications: Design and Development5
 COMPS381FServer-side Technologies and Cloud Computing5
 COMPS382FData Mining and Analytics5
 COMPS390FCreative Programming for Games5
 COMPS413FApplication Design and Development for Mobile Devices5
 COMPS492FMachine Learning5
 ELECS305FComputer Networking5
 ELECS363FAdvanced Computer Design5
 ELECS425FComputer and Network Security5