Human workers need to handle heavy objects and materials (e.g., assembly parts, cartons, bulk materials in bags) in various industries such as construction, manufacturing and assembly, mining, transport and logistics, rescue and disaster operations, military operations and timber/forestry. However, manual manipulation is tedious and it reduces efficiency and causes musculoskeletal disorders, back pains and injuries in workers [1]. On the contrary, autonomous manipulation is usually inflexible (or less flexible) and less adaptable [2]. By flexible manipulation, we here mean the manipulation method that can be easily modified and reprogrammed to respond to altered circumstances or conditions, which is adaptable, adjustable and versatile [3]. By flexible manipulation, we do not mean manipulating an object that is itself flexible or deformable [4]. To achieve flexible manipulation, we propose that human-in-the-loop collaborative automation system such as the power-assist robotic system (PARS) can be comfortably used for object manipulation, where the combination of mechanical strength of a robot and intelligence of a human can make the human–robot system superior to a robot or an individual human [5]. PARSs can provide various advantages, e.g., (1) power assistance through sharing power and reducing haptically perceived heaviness, (2) flexibility in positioning and ease in motion control through direct human–machine haptic interface and haptic information sharing, (3) naturalness and intuitiveness as human intent is reflected through human input to the system, etc. These advantages can foster high precision, efficiency, robustness and human-friendliness in object manipulation [5, 6].
Reviews on state-of-the-art PARSs for industrial object manipulation show the early-stage works of Kazerooni that introduced the fundamental principles of information and power sharing of a PARS for load manipulation [5]. After that, a significant number of PARSs have been proposed for handling objects, e.g., [6,7,8,9,10,11,12,13,14,15,16,17]. In [6], Niinuma et al. proposed a power-assisted overhead crane for object manipulation and also compared its performance to that of conventional automated manipulation. In [7], Doi et al. proposed a pneumatically actuated hand crane-type PARS for object manipulation. Hara introduced a switching mechanism between automatic transfer and power-assist controls for horizontal manipulation of object [8]. Yagi et al. proposed a control method for a pneumatically actuated upper arm PARS for agricultural load manipulation [9]. Dimeas et al. introduced admittance neuro-control of a PARS for lifting objects [10]. Hara and Sankai demonstrated a “Hybrid Assistive Limb-HAL” prototype to assist humans in carrying heavy loads [11]. In [12, 13], it was shown how admittance controls were varied to adjust situations while handling large loads with power-assist. Various types of industrial assistive devices (IADs) for load manipulation were introduced and analyzed in [14]. In [15], Olivier et al. presented “Cobomanip”—an IAD for load manipulation in industries, and so forth. A few PARSs for object manipulation are already in practical applications in industries such as the “Power Loader Light-PLL” [16] and Cobot [17]. The above reviews show that power assisting automation technologies for manipulation of heavy objects achieved significant advancements. However, PARSs for object handling still have a few fundamental limitations or challenges as follows that require close attention:
Mismatch between visual and haptic perceptions
A human user perceives reduced heaviness while manipulating an object with power-assist [5]. The user feed-forwardly estimates the manipulative force (load force, grip force) to manipulate (e.g., lift) the object with the PARS depending on visually perceived weight of the object [18]. Here, the load force reflects human’s intent in manipulation that influences the motion [5]. The haptically perceived weight is smaller than the visually perceived weight [5], and thus, the applied load force estimated by the user based on the visually perceived weight is incorrect (larger than the load force actually required to lift the object to the desired position successfully) that results in harmful motion (acceleration), poor safety and lack of stability [5, 19]. As a consequence, human–robot interaction (HRI) and overall performance in manipulation become unsatisfactory that also reduce user’s trust in the robot [20, 21]. Furthermore, cognitive workload and fatigue may increase if the user undergoes a careful visual check of the prospective weights before handling objects with power-assist to realize the difference between visually and haptically perceived weights.
Gravity compensation in robot dynamics can be an approach to solve the aforementioned problem [6, 10, 14]. However, zero gravity removes haptic feelings and restricts naturalness in direct kinesthetic co-manipulation of objects [22]. As an alternative approach, the gravity can be partly compensated by using a virtual mass in the dynamics [8, 12, 13]. In this approach, the mass value needs to be estimated in such a way that it provides expected haptic feelings in the user. However, basis of estimation of mass value for partial compensation of gravity has not been justified yet, which does not help achieve expected HRI and performance [19, 22]. Another alternative approach may be the use of a tentative feed-forward model of the load force as a user input to the PARS with a notion that the model may be adjusted if the user gains experiences [7, 8]. However, effectiveness of such notion has not been justified properly. Model-based predictive controllers (e.g., a model predictive controller—MPC) may also be used to generate the predicted input force based on an optimization scheme to provide predicted output (acceleration) [23]. Constant torque/force method [24] may also be used to provide constant or nearly constant output force/torque. However, the load force depends on object gravity in power-assist dynamics and the optimum input force provided by MPC or the constant force may not produce optimum haptic feelings in user. In fact, estimation of load force for manipulation with power-assist is a cognitive phenomenon that depends on user’s visual perception of object weight [5, 19], and hence, the input load force cannot be estimated by any computational model perfectly. Instead, the effects of excess in load force can be counterbalanced if an active compliance control method is proposed reflecting/mimicking user’s cognition in power-assist dynamics [19, 25]. Here, by cognition we mean human operator’s mental action or process of acquiring knowledge and understanding about the objects and environment through thoughts, experiences and senses [26]. Cognition can convey the similar meanings as perception, discernment, apprehension, learning, understanding, comprehension, insight, etc. Cognition can mean weight perception, which is the perception, recognition and discrimination of the heaviness of a lifted object [19]. It may be a combination of visual perception and haptic perception [27]. However, such cognitive or weight perceptual approaches integrating human thoughts, perception and capabilities are not observed with the state-of-the-art control strategies of PARSs except a few preliminary initiatives [19, 25, 28].
Selection of appropriate control strategies
Selection of control strategies for manipulating objects with power-assist is very challenging [29]. Large inertia, friction and dynamic effects are expected while manipulating heavy objects, which can be compensated and positional accuracy can be provided by admittance controls [12]. Admittance parameters (e.g., virtual mass, damping and stiffness) can affect HRI and manipulation performance. For example, for large admittance parameters, large load force is required to move the object and the user feels more heaviness that may cause fatigue. The movement may also be slow due to low acceleration. However, it may be possible to achieve precise (e.g., smooth, fine) manipulation. On the contrary, low admittance parameters may need less human force to accelerate the object that may result in low fatigue, but precision in manipulation may reduce due to the reason that the robot is more reactive. These are the disadvantages of fixed admittance control that indicate the necessity of variable admittance control [13]. In [13], a variable admittance control strategy was proposed where a virtual mass varied to adjust acceleration and precision in power-assisted manipulation. However, the effects of excessive acceleration generating from user’s error in the programming of load force due to difference in perception between visual and haptic weights were not mitigated. Furthermore, changes in virtual mass (the mass value used in the dynamics) change acceleration [19], but it may also alter haptic perceptions [5]. Consequently, HRI and manipulation performance may be affected adversely [19]. Hence, a novel variable admittance control strategy seems to be necessary to modulate the kinematics (acceleration) and haptic perceptions differently in power-assisted manipulation to achieve better HRI and performance. However, such novel strategy has not been proposed and validated yet properly [19].
Comprehensive evaluation scheme
A comprehensive evaluation scheme is necessary for PARSs for object manipulation, which can be used to optimize HRI and co-manipulation performance. Not only robotics parameters, but also HRI and manipulation performance need to be optimized to achieve human-friendliness in collaborative manipulation. Objective evaluation is emphasized; however, there are some HRI and performance criteria that can neither be measured objectively nor be ignored. Hence, subjective evaluation also needs to be considered as complementary to objective evaluation. HRI criteria should address both physical HRI (pHRI) and cognitive HRI (cHRI), and performance criteria should include the key performance indicators (KPIs) of power-assisted manipulation in actual industrial applications. The state-of-the-art literature shows a few detached initiatives for evaluation of PARSs. For example, only precision, stability and efficiency were evaluated in [6], and user comfort was evaluated in [9]. Safety in PARS was provided through mechanical design [12, 13, 30], but a safety evaluation and analysis method was not proposed. In [31], Sylla et al. focused on ergonomic criteria ignoring other HRI and performance criteria. Maurice et al. [32] addressed simulation-based musculoskeletal risks in power-assisted manipulation only. A complete hazard analysis and risk assessment for power-assisted manipulation is especially important due to the potential risk of unexpected motion [19]. Different standards and guidance such as ISO/TS 15066, ISO 10218–1 and ISO 10218-2 have been proposed to ensure safety of collaborative robotics [33,34,35,36,37]. However, initiatives to conduct risk analysis and ensure safety following this guidance for power-assisted manipulation have not been taken yet. As a result, it is still uncertain about simultaneous attainment of user-friendliness and high performance in power-assisted manipulation.
Naturalness and intuitiveness in manipulation
The principle advantage of a PARS is that a user manipulates a heavy object with the PARS, but he/she feels lightweight [5]. For heavy and large objects manipulated with a PARS, the user cannot grasp the entire object using a power grip properly, and thus, the HRI and performance experienced for power-assisted manipulation of heavy objects may not reflect user’s naturalness and intuitiveness. However, appropriate methodology to achieve naturalness and intuitiveness in HRI and performance for power-assisted manipulation has not been proposed and validated yet.
Being motivated by the above limitations/challenges of the state-of-the-art PARSs for object manipulation, we summarized the specific problems of the state-of-the-art PARSs and proposed appropriate solutions as given in Fig. 1. According to the proposed solutions in Fig. 1, the purpose of this article is to develop a human-friendly PARS for industrial heavy object manipulation exploiting human cognition-based variable admittance control as a means of active compliance, develop a comprehensive scheme to evaluate the system, and ensure naturalness and intuitiveness in power-assisted manipulation. The core innovations as we attempt to bring are: (1) proposing appropriate control strategies for PARSs for object manipulation, (2) illustrating a method to include user’s weight perception in power-assist system dynamics and control, (3) proposing a comprehensive evaluation scheme including an HRI optimization method for power-assisted manipulation, and (4) proposing a method to achieve naturalness and intuitiveness in power-assisted manipulation.
To bring the core innovations as above, we adopt two main objectives for this article and use two steps to address the objectives:
-
i.
In the first step (second section to seventh section), we investigate a method to include weight perception in the dynamics and derive weight perception-based fixed admittance control for the PARS. We then determine a comprehensive evaluation scheme including risk assessment and determine optimum HRI and performance for fixed admittance control using a local optimization scheme for a set of hard constraints for lifting lightweight objects with power-assist.
-
ii.
In the second step (eight section to tenth section), we investigate a variable admittance control strategy based on weight perception, kinematics and kinetics features to add variable compliance to the PARS for improving HRI and performance so that optimum HRI and performance can be achieved for a set of soft constraints, and also risk in object manipulation is reduced. We evaluate the variable admittance control strategy for lifting lightweight objects. We then validate the optimization and control approaches for vertical lifting of heavy objects using a multi-DOF PARS. We calibrate the HRI and performance for heavy object manipulation through comparing the evaluation results for lightweight object manipulation with that for heavy object manipulation.
We then discuss how the findings can be used to develop power-assist devices to manipulate heavy materials in actual industrial environments. Note that as a preliminary initiative, we here consider vertical lifting only as it is common in industries, humans feel heaviness more in vertical lifting, and it needs more power assistance. We also explain how the proposed approach can be augmented to 6-DOF dexterous manipulation. The presented 1-DOF design seems to be simple, but we think that such design may be sufficient to address the objectives, and achieve the core innovations.