The hybrid control system described above was used to perform closed-loop actuation of a single microrobot, grasp planning of multiple microrobots for a micromanipulation task, closed-loop actuation of multiple microrobots, and open-loop micromanipulation of a star-shaped SU-8 microstructure.
Closed-loop actuation of a single microrobot
One of the benefits of the closed-loop control of OFB microrobots is the ability to accurately update the position of the microrobot using data from the image-processing algorithm. Here, we have demonstrated actuation of one microrobot along a preset zigzag path using the hybrid control system (Fig. 3). The microrobot was actuated from its initial location to 5 waypoints (waypoints 2–6 in Fig. 3) using automated open-loop control sequences. Each point-to-point actuation (1–2, 2–3, 3–4, 4–5, and 5–6; see Fig. 3a) consisted of a series of 30 smaller actuation segments, at a rate of 2 Hz. The actuation velocity between the waypoints varied.
First, the microrobot was actuated from position 1–2 using open-loop control, a distance of 1048 μm (Fig. 3a, b). At location 2 (Fig. 3b), the feedback block captured an image of the workspace, detected the physical bubble location within the workspace, and compared it with the intended destination set in the LabVIEW user interface. The microrobot was then moved to a new location (2′ in Fig. 3c) to correct for the positioning error. Similarly, the microrobot position was determined by the feedback block at each waypoint (3–6 in Fig. 3d, f, h, j), compared with the preset destination in LabVIEW actuation block, and then moved to minimize the difference between the preset destination and the actual position (3′, 4′, 5′, 6′ in Fig. 3e, g, i, k). The position error calculated by the feedback block at 2′, 3′, 4′, 5′, 6′ in Fig. 3 was 5.25, 14.1, 5.1, 16.5, and 2.8 μm, respectively. The position error was then reduced to approximately 1 µm after the microrobot locations were updated using the hybrid closed-loop control system.
Grasp planning
Path planning refers to determining a collision-free path for a moving object among obstacles [18]. In this work, a grasp-planning algorithm determines the geometry and location of a microrobot and a micro-object payload. The output of the grasp-planning algorithm is sequences of microrobot locations that form a trajectory from its initial position to its goal position, which is in reference to the payload. There were no obstacles present in the workspace when caging the payload, but the algorithm is capable of determining collision-free path about obstacles. (Additional file 1: Figure S4). The grasp-planning algorithm was used to create a cage of four OFB microrobots around a star-shaped SU-8 microstructure, and transport the object to another location. The star-shaped microstructure consisted of four arms, each 59 μm in length, and a hollow circular center with an inner diameter of 62 μm. The width of the wall around the circular center and the width of the arms were approximately 66 μm. The thickness of the SU-8 was 50 μm, and the structure had an approximate mass of 2.35 μg.
Initially, four OFB microrobots were generated at random locations around the micro-object (Fig. 4a) by momentarily increasing the optical power in each spot using the actuation block (Fig. 2). The feedback block then detects the location and size of the microrobots and the structure. The grasp-planning algorithm uses the location of the structure within the workspace to calculate the caging positions (1′, 2′, 3′, and 4′ in Fig. 4a) at user-defined equidistant locations around the micro-object. The grasp-planning algorithm allows the adjustment of the caging locations based on visual feedback and the shape of the object. In this experiment, the caging formation was rotated clockwise (Fig. 4a–c) to allow a better grasp of the micro-object. The final caging configuration (Fig. 4d) puts the microrobots in positions that will allow them to grasp in between the arms of the micro-object when the caging formation is contracted.
Hybrid closed-loop actuation of multiple microrobots
The caging formation (1′, 2′, 3′, and 4′ in Fig. 4d) calculated by the grasp-planning algorithm of the feedback block sets the destination location for the individual microrobots. These positions were saved in a.mat file and subsequently loaded into the MathScript module of the LabVIEW actuation block. However, the caging locations require a transformation to match the coordinate system of the actuation block. Once the transformed final destinations of the microrobots are serially loaded into the MathScript module from the.mat file, the destination of each microrobot is mapped to the corresponding locations on the LabVIEW user interface. In this experiment, the microrobots numbered 1, 2, 3, and 4 were assigned the caging locations 1′, 2′, 3′, and 4′ (Fig. 4d) as their destinations in the actuation block.
Figure 5 shows the open-loop actuation of four microrobots from their initial positions to the caging locations. Microrobots 1, 2, 3, and 4 were simultaneously actuated at velocities of 19, 29.6, 44.1, and 31.83 μm/s, respectively, using the actuation block (Fig. 5a–c). Here, the simultaneous actuation of multiple microrobots with different speeds demonstrates a capability of the microrobot control system: parallel, uncoupled movement of microrobots along trajectories that vary in direction and distance traveled during the same actuation time. The microrobot actuation took 15 s (Additional file 2: Video S1). Figure 5d shows the path of each microrobot from its initial position to the caging location. Here, the simultaneous actuation of multiple microrobots with different speeds demonstrates a capability of the microrobot control system: parallel, uncoupled movement of microrobots along trajectories that vary in direction and distance traveled during the same actuation time.
Figure 6 shows the OFB microrobots at the caging positions. The image-processing algorithm determined the locations of the microrobots and compared them to the caging locations calculated by the grasp-planning algorithm. Figure 6a shows the locations of Microrobots 1, 2, 3, and 4 as determined by the image-processing algorithm (white dotted circles), and the desired caging locations set by the grasp-planning algorithm (red circles). The feedback block calculates the error between the actual location of the microrobot and the desired caging position, and calculates the new microrobot destination to minimize the error. Microrobots 1, 2, 3, and 4 in Fig. 6a were 27, 24.7, 14.28, and 21.5 μm away from their desired caging positions, respectively. This information is passed to the MathScript node of the actuation block, which moves the microrobots to the new destinations (Fig. 6b). The microrobots were actuated to their new positions at speeds of 5.4, 4.94, 2.86, and 4.3 μm/s for Microrobots 1, 2, 3, and 4, respectively (Fig. 6b).
After updating the location using open-loop actuation, the physical location of the microrobot within the workspace was determined by the image-processing algorithm of the feedback loop, as shown in Fig. 6c. The red circles in Fig. 6c are the caging position set by the grasp-planning algorithm at the beginning of the actuation. In Fig. 6c, the Microrobots at 1′, 2′, 3′, and 4′ were 8, 12, 5, and 19 μm away from their desired locations, corresponding to a reduction in the position error of approximately 50%.
Micromanipulation
The hybrid closed-loop vision-assisted control system allowed an accurate placement of a caging formation, as described above. Figure 7a shows the open-loop actuation of a microrobot matrix approaching a micro-object to grasp it for manipulation. The matrix of microrobots was manually controlled by user input to the actuation block. The object was grasped by contracting the microrobot formation at an average speed of 7.6 μm/s (Fig. 7b and Additional file 3: Video S2). After grasping, the microrobot formation attempted to transport the micro-object in the positive x-direction. However, the micro-object was stuck to the floor of the fluid chamber, resulting in the dislocation of the microrobots from their actuation patterns (Fig. 7c). This phenomenon is more obvious for Microrobots 2 and 3, marked with red arrows in Fig. 7c, as they were moved in the positive x-direction and left the micro-object behind.
To free the micro-object from the surface, the formation of the microrobots was rotated to create a torque on the object while maintaining a firm grasp (Fig. 7d). The twisting of the object helped to overcome its stiction, and the micro-object could then be transported. The micro-object was transported along various trajectories (Fig. 7e) and at various speeds up to approximately 90 μm/s (Additional file 4: Video S3). A graph of the planned trajectory and completed trajectory during the micro-object manipulation is included in Additional file 1: Figure S5. Upon completion of the micromanipulation, the micro-object was released by expanding the microrobot formation (Fig. 7f).