Hardware and Software Based Methods for Safe Human-Robot Interaction with Variable Impedance Robots

Faculty Development Competitive Research Grant Program 2018-2020

Despite recent advances, robots are still outperformed by humans in tasks requiring dexterity, safety and efficiency. Human-like performance would shift industrial robotics to a paradigm, where robots and humans can work together collaboratively and safely without the need of protective barriers. The potential of achieving human level performance inspired researchers to design anthropomorphic robots [1]. After initial focus on the kinematic configuration of the robots, the research focused on their dynamic behavior in situations such as collisions, interaction with humans, objects and the environment. One approach is the light weight robots with integrated torque sensors and active torque control [2] (See Fig. 1b). In spite of the presence of successful and already commercialized examples, these robots have inherent limitations [3]. Firstly, the actuators in the joints are not decoupled from the links, and compliant behavior is achieved by active control: therefore, impacts lasting a very short time might exceed the torque limits of the gearbox and actuators, and presumably damage the system. Secondly, on the contrary to the human musculoskeletal system, these robots do not have the ability to store energy induced in their link-side structure, which limits their velocity and dynamic force in tasks such as throwing, jumping and running.
Variable impedance actuated (VIA) robots [4-11] have recently emerged to cope with the challenges faced by the actively torque controlled light weight robots. VIA robots (See Fig. 1c) are usually intrinsically compliant, and elastic elements in the joints store energy. This way, the robots can decrease energy consumption in repetitive tasks, increase maximum force and velocity capabilities, and absorb impacts. Moreover, the variable impedance behavior of the robot can protect both the robot joints and the human. However, these benefits are also accompanied by certain disadvantages: the design gets complicated due to the elastic elements and their additional actuators, and the absolute position accuracy and mechanical bandwidth are reduced due to the higher elasticity. The parameter identification, task planning, control and design of VIA robots are also challenging problems. Due to nonlinear dynamics, actuation constraints and high number of control inputs, classical controllers cannot utilize the full potential of VIA robots. In order to exploit the natural dynamics, the impedance of the actuators should be optimally modulated during operation. Preliminary controllers based on this idea were used to decrease energy consumption, to improve human-robot interaction safety, and to increase performance in explosive movement tasks [12-16].
A generalized task planning and closed-loop control framework for VIA robots was introduced by the PI and his co-researchers [17]. In this framework, tasks are defined as constrained nonlinear optimization problems and solved using quadratic programming (QP). In the closed-loop control part, nonlinear model predictive control (NMPC) is utilized to control the VIA robot. NMPC solves a finite-horizon optimal control problem (FHOCP) on-line at each sampling instant. FHOCP determines a sequence of control moves within a given prediction horizon (typically, much shorter than the whole task length), by utilizing the available model of the system, to minimize a cost function and ensure that the evolution of the system variables be inside the imposed boundaries. The whole determined sequence is not directly used (as in classical open-loop optimal control): instead, only the first control move is applied to the system, and the FHOCP is solved again at the next sampling instant. It was shown that this control approach for VIA robots is robust to parameter uncertainties and external disturbances [17]. It also provided superior performance in explosive motions and trajectory tracking [18-19]. The advances introduced by the PI’s research group allowed integration of additional elements to the VIA robots. Specifically, a reaction wheel was integrated to a planar VIA robot and it was shown the performance can be improved by applying reactive torques as needed [20-22].
Industry 4.0 is a new trend in automation aiming to create smart factories of the future. One of the objectives of this cyber-physical systems paradigm is safe physical HRI in collaborative settings such that the strength, speed, repeatability and accuracy of the robots can be combined with the human skills, intelligence and versatility. Multiple research groups are engaged in this research. For instance, Safe and Autonomous Physical Human-Aware Robot Interaction (SAPHARI) project was a large scale European Union research project involving partners such as German Space Agency Institute of Robotics and Mechatronics, Italian Institute of Technology, Technical University of Munich, KUKA, University of Pisa and University of Bremen. This project focused on different aspects of safe HRI in industrial settings such as compliant actuation, real-time perception, human-aware task planning, reactive action generation [23-25]. Another large scale project, ROBO-PARTNER, investigated seamless human-robot cooperation for intelligent, flexible and safe assembly operation (mainly for the automotive industry). New task planning schemes for HRI was introduced by these researchers. They were also in pioneers in using AR/VR interfaces for reducing assembly errors and operator fatigue. One should note that these projects did not study safety of VIA robots.
There are two fundamental problems regarding physical HRI with VIA robots. Firstly, VIA robots are safe for humans when joints have high compliance and low speed. High compliance decreases the robustness to uncertainties and external disturbances and also increases position tracking error due to low bandwidth. Secondly, a human does not have a way of understanding whether a VIA robot is safe to be near or not, i.e. the stiffness and velocity information of different parts of the VIA robot for a short time horizon are not known to the human worker. If a VIA robot would achieve robust and precise position tracking during compliant operation and if the safety state of the robot can be broadcasted to the human worker, the safety of physical HRI in industrial settings would be significantly improved. To the best of the investigators’ knowledge, no other research group worked on these problems yet. In this project, we intend to use novel solutions based on reaction wheel and AR technologies to tackle the first and second problems, respectively.
After being phrased as the technology of the future for a long time, AR is becoming a hot research topic thanks to the introduction of capable AR devices such as Microsoft Hololens, Meta AR and Magic Leap. Besides multiple application areas in medicine, defense, entertainment and gaming, AR is also emerging as an important tool for industrial automation and smart manufacturing (See Fig. 2). Recently, AR was used in human-robot collaborative tasks for providing online assembly instructions to the worker [26]. This work also aimed at increasing the “safety feeling” of the worker by visualizing data coming from a robot’s controller and by displaying visual alerts to increase their awareness for a potentially hazardous situation. It should be noted that these robots were position controlled classical industrial manipulators not VIA robots. In this project, we want to relay the safety information of the VIA (based on a safety criterion such as Head Injury Criterion [27]) to the worker using an AR headset. We will achieve this by visualizing the trajectory of different robot segments for the present task with different colors denoting various safety levels. We call this visualization as the “safety aura”.
Reaction wheels have been used for spacecraft attitude control and stabilization since 1960-s [28]. By applying torques to a fast turning disk, they can apply reaction torques with a high bandwidth. Due to their applicability in zero gravity conditions, they were used also for space robotics applications [29-30]. Specifically, effects of reaction moments on satellite-based manipulators and different approaches for base platform stabilization were analyzed in [31]. Closed-loop velocity control as opposed to current control of reaction wheels in satellite-based robot arms was discussed in [32]. However, their use in mainstream robotics was limited. Recently, ETH Zurich researchers introduced a test platform “Cubli”: a cube that can jump up, walk and balance itself using three reaction wheels [33]. Investigators were the first ones to integrate reaction wheels to VIA robots and improve their performance [20-21]. In this project, we will integrate reaction wheels to the VIA robots such that high bandwidth actuation can be achieved even if the VIA robot is run in a highly compliant (and intrinsically safe) mode. We will also exploit the high bandwidth of the system to realize quick reflect mechanism to act during impacts.
Intellectual property on the safety of physical HRI of VIA systems using reaction wheels and AR is also non-existent. Therefore, the investigators see a great opportunity for intellectual property generation in this area. Considering that VIA is emerging the next robot design paradigm, the know-how and intellectual property, which will be generated within this project, has also a great potential to leave a long-lasting imprint to the future of physical HRI which is pursued by all major robotics companies in the world.
With the latest push for digitization in Kazakhstan and industrial modernization based on state-of-the-art tools and methods, AR is also a vital technology for Kazakhstan. One of the side benefits of this project will be training of multiple young Kazakhstani researchers in this disruptive field. Spillover effects of know generated in VIA robots, reaction wheels and AR might also beneficial for companies such as Kazakhstan Engineering which is trying to intensify its research and development efforts.
Finally, the European Parliament resolution of 16 February 2017 “Civil Law Rules on Robotics” is the first resolution of European Parliament in robotics. This resolution is deeply concerned with the safety of HRI and urges robot designers to “consider and respect people’s physical wellbeing, safety, health and rights.” Considering European Union runs the largest research and innovation program in civilian robotics in the world (SPARC with 2.8 billion EUR funding), investigators expect the significance and relevance of physical HRI research (especially for VIA robots) to substantially increase in the next decade.