Mme DOAN Thi Huong Giang, doctorante de l’Institut MICA, a brillamment soutenu sa thèse à Hanoi le 11 juillet 2017, et a ainsi obtenu le titre de Docteur en Sciences, spécialité Control Engineering and Automation

 

Title: Dynamic hand gesture recognition using RGB-D images for Human - Machine Interaction.

 

Thesis supervisor: Dr. VU Hai
Thesis co-supervisor: Dr. TRAN Thi Thanh Hai

 

Jury:
Prof. PHAM Thi Ngoc Yen (Chairman, SEE-HUST)
Assoc. Prof. DAO Mong Lam (Reviewer, Military Science and Technology Institute)
Prof. PHAN Xuan Minh (Reviewer, SoICT-HUST)
Assoc. Prof. LE Thi Lan (Secretary, MICA-HUST)
Assoc. Prof LE Thanh Ha (Commissioner, VNU-Hanoi)
Assoc. Prof. PHAM Van Cuong (Commissionner, Posts and Telecommunication Institute of Technology)
Dr. VU Hai (Supervisor, MICA-HUST)

 

Motivation
Home-automation products have been widely used in smart homes thanks to recent advances in intelligent computing, smart devices, and new communication protocols. In term of automating ability, most of advanced technologies are focusing on either saving energy or facilitating the control via an user-interface. To maximize user ability, a Human Computer Interaction method allows end-users easily using and naturally performing the conventional operations. Motivated by such advantages, this thesis purses an unified solution to deploy a complete hand gesture-based control system for home appliances. A natural and friendly way is deployed which aims to replace the conventional remote controller.

A complete gesture-based controlling application requires both robustness as well as low computational time. However, these requirements meet many technical challenges such as huge computational time and complexity of hand movements. A set of suggestive hand gestures is pursued. There is an argument that the characteristics of the hand gestures are important cues in contexts of deploying a complete hand gesture-based system. On the other hand, recent new and low-cost depth sensors have been widely applied in HCI. Which opens new opportunities for addressing the critical issues of the gesture recognition schemes. This thesis attempts to use the benefits of the Kinect sensor which provides both RGB and depth features. Utilizing such valuable features offers an efficient and robust solution for addressing the challenges.


Objectives
•    An unique dynamic hand gestures set is deffined to convey commands for almost home appliances. Datasets are captured to evaluate usability with the designed datasets and experimental evaluation of the proposed solutions.
•    A real-time system for spotting dynamic hand gestures includes: The solutions for hand detection and segmentation from RGB-D images. Then, dynamic gestures are efficiently spotted from a continuous segmented continuous hands sequence.
•    A dynamic hand gesture recognition method involves the hand gestures representation and phase synchronization.
•    A full framework to control home appliances using dynamic hand gestures without requiring GUI interface. Then, a full hand gesture-based system is setup under indoor scenarios of the smart-room.


Contributions
•    Contribution 1: A dynamic hand gesture dataset to conduct the commands of the home appliances is designed. The proposed gestures are suitable to deploy gesture-based systems for a smart room environments. The gesture set conveys useful and supportive characteristics for deploying a robust hand gesture recognition system.
•    Contribution 2: An efficient user-guide scheme is proposed to use the heuristic parameters-based with a trade-off between a real-time system and independence end-user system. Which helps to obtain both a real-time hand detection and good performance of hand segmentation. Then, an efficient spotting method is proposed that uses these segmented hand.
•    Contribution 3: An efficient manifold-based representation is proposed for dynamic hand gestures which combined spatial-temporal features produce gesture representation. By using some most significant dimensions from the non-linear reduced space, the spatial features are extracted for dynamic hand gesture representations, and trajectories of hand movements are extracted.
•    Contribution 4: A new fully control development is deployed to control light and fan in home automation by using dynamic hand gesture recognition system.