KKH Dedicated Robot-Human Speech Dialogue

BSc./MSc. Student Assignment Robot Rose

Robot ROSE will work fully autonomously, but in future implementations must be able to listen to both voiced care staff commands and hospital client questions. For either cases, a speech recognition not only needs to be implemented, but also a proper dialogue tree established related to relevant nurse commands and client questions.

First, a standard (open source) speech input module must be implemented on robot ROSE. Second an indexation of nursing commands and client questions needs to be established such that a dialogue tree can be created. Subsequent tests with nurses and client need to be performed to obtain user-feedback and opinion.

Project output:
  • Indexation of required robot speech input commands from nurses
  • Indexation of frequent hospital client questions
  • Implementation of an (open source) speech recognition module
  • Setup of a dialogue tree for both nurses and clients
  • Test of dialogue tree in KKH environment to obtain user feedback and opinions
Required:
  • Excitement about new technologies, eager to work on state-of-the-art technologies;
  • Independence, research appetite, enthusiastic, self-critical;
  • Sense of responsibility, duty fulfillment;
  • Eager to work in an international environment, team-player, collaborate with multiple institutions and companies, determined to dedicate time in the project, able to set priorities;
  • Basic understanding of math and programming (C++, Python, Matlab);
  • Familiarity with the Robot Operating System (ROS).
Nice to have:
  • Basic understanding of machine and deep learning, neural networks, optimization techniques (gradient descent, mini-batch gradient descent);
  • Basic understanding and/or experience with computer vision techniques (object recognition, feature detection, image classification etc.);
  • Experience with deep learning frameworks (Caffe, CMTK, Tensorflow, Pytorch, Theano, etc.) would be highly appreciated;
  • Hardware and software skills (experience with NVIDIA Cuda, Jetson, etc.).