Cell preparation
Neuroblastoma cells were employed as the sample cells for the identification and patch clamp recording. The neuroblastoma cells were cultured in Dulbecco's modified eagle medium containing nutrient mixture F-12 (Gibco, Thermo Fisher Scientific Inc., Waltham, MA, USA) with 10% fetal bovine serum (Gibco, USA) and 1% penicillin-streptomycin-neomycin antibiotic mixture (Gibco, USA) and maintained at 37°C in 5% CO2 humidified air. Stock cultures were passaged at 1:6 weekly and fed twice weekly. The cells were subcultured with the cell density of 100,000 in a culture flask. During the process of subculture, the cells were detached from the culture flask by the treatment of 1 ml trypsin-EDTA (Gibco, USA). Around 5,000 cells were then counted and seeded onto the cover-slip coated with Poly-D-lysine hydrobromide (Sigma-Aldrich, St. Louis, MO, USA) and cultured for 6 days before patch clamp recording. These cells were induced to differentiate by the treatment with 10 μM retinoic acid. All the experiments were conducted at 37°C.
Experimental setup and control strategy
The illustration of the experimental setup for implementing the patch clamp recording based on the visual-based identification is shown in Figure 2. The cells and pipette were first observed under an inverted microscope (Olympus, Shinjuku-ku, Japan). The microscope was equipped with four different objectives with the magnification of × 4, ×10, ×20, and × 40. The live video and digital real-time image were then captured using a CCD camera. The resolution of the image was 1,280 × 768 pixels. Based on the image processing and identification method as described in Figure 2, the coordinates of the cell and the pipette can be obtained from the image, and the movement of the manipulator was then calculated.
To achieve the automated patch clamp, the patch pipette was mounted to a computer-controllable micromanipulator (Sutter MP-285; Sutter Instrument Company, Novato, CA, USA) connected to a controller (Sutter MPC-200, USA). The micromanipulator was controlled to move along the x-, y-, or z-axis with the range from 0 to 25,000 μm. The micromanipulator was then moved to the desired position based on the command received from the controller. The pipette was connected to the manipulator via a pipette holder, and the position of pipette tip was then controlled by the manipulator. The amplifier was used to perform the patch clamp recording. The signal ground of the patch clamp amplifier was connected to the extracellular buffer in the dish. The patch clamp amplifier applied a voltage pulse to the cell and the corresponding current between the electrode inside the pipette and the signal ground was then recorded. By monitoring the current, the condition of the pipette and the cell membrane was determined.
The automatic control strategy can be divided into four steps. In the first step, a digital image containing cells and pipette is captured by the CCD camera, and the system transforms the digital image to the grayscale image. In the second step, the dilation, erosion, and threshold operation are applied to the grayscale image, and the binary image is then automatically obtained. In the third step, the boundary of the cell is identified, and the center of the cell is calculated from the binary image. Besides, the tip of the pipette is also displayed and identified in the binary image. Finally, the coordinate of the center of the cell and the tip of the pipette are obtained. The movement from the tip of the pipette to the center of the cell is calculated based on the updated coordinates, and the command is sent to the controller for moving the manipulator and the pipette tip to the center of the cell. The movement of the pipette tip can be adjusted based on the updated coordinates of the cell and the pipette.
Identification of cells
To perform the detection of the cell and pipette for patch clamp, the digital image of the sample was first transformed to a grayscale image. In the grayscale image, every pixel was represented by an intensity value ranging from 0 to 255. The image was then processed by applying two basic morphological operations, namely dilation and erosion [25]. The concept of the dilation operation was to increase the area of bright regions in the image as the erosion was to increase the area of dark regions in the image. In order to perform the appropriate morphological operation to the image, a structural element, a matrix consisting of only 0 and 1 values, was defined and used. As shown in Figure 3a, a 3 × 3 cross shape structural element was applied to a 5 × 5 binary image, and each pixel value in the element represented the intensity. When the dilation was applied, the maximal pixel value overlapped by the structural element was found to replace the pixel value at the center of the structural element. Therefore, the resulting bright region was increased after performing the dilation as shown in Figure 3b. When the erosion was applied, the minimal pixel value overlapped by the structural element was found, and the pixel value at the center of the structural element was then replaced by this minimal pixel value. Therefore, the resulting bright region was decreased after performing the erosion as shown in Figure 3c.
The process flow of applying the morphological operation to the cell image is shown in Figure 4. As shown in the grayscale image of the cell in Figure 4a, the cell exhibited higher intensity in the region of the cell membrane as compared with that was displayed in other areas of the cell. In order to allow the system to detect and obtain the cell outline, the dilation followed by erosion was applied to the image as shown in the output image of the cells in Figure 4c, the cell boundary with higher intensity than the background was generated.
After performing the dilation and erosion of the cell image, the cells were further segmented from the background by applying threshold operations to the image. For a grayscale image with intensity value ranging from 0 to 255, pixels with an intensity value that is higher than the threshold were changed to the maximum intensity value, i.e., 255, while pixels with the intensity value equal or lower than the threshold were changed to 0. This operating process can be represented by the following equation:
(1)
where dst(x, y) represents the intensity value of each pixel in the output image, src(x, y) represents the intensity value of each pixel in the input image, and threshold is the threshold value.
After the threshold operation, a binary image was obtained in which the white areas indicated the cell features and the black areas indicated the background as shown in Figure 5a. In the next step, the black areas inside the white area were filled with white color as shown in Figure 5b. The cells were then identified by detecting the white areas.
After identifying the cells, the center of the cell was calculated by using the following equations:
(2)
(3)
where C
x
is the x-coordinate of the center of the cell, C
y
is the y-coordinate of the center of the cell. Finally, the membrane and the center of the cells were identified as shown in Figure 5c.
Segmentation of neighboring cells by using distance transform and watershed transformation
Further methods were used to segment the cell from neighboring cells when the cells were too close to each other. As shown in the grayscale image of two adjacent cells (Figure 6a), the two cells were considered as one blob after the morphological operation (dilation followed by erosion (Figure 6b,c), and therefore, the center of the cells was found at the wrong position as shown in Figure 6d.
Two algorithms, distance transform and watershed transformation, were used to further process the image obtained from the morphological operation and segment the connected cells accordingly. The distance transform was first applied to the binary image, and the basic model for applying the distance transform is shown in Figure 7. Consider the binary image as shown in Figure 7a, the foreground was extracted, and distance transform was applied from the boundary of the foreground region in the binary image. Each pixel in the foreground regions in the binary image was replaced by a pixel with gray value that was determined from the computed distance between that pixel to the closest boundary in its foreground region [26] (Figure 7b). Based on this method, the binary image of the cells after distance transform was obtained as shown in Figure 6e.
Threshold operations were then applied to the distance transform image. After the distance transform image was obtained (shown in Figure 6e), another binary image can be generated by the threshold operations. The pixels having intensity value equal or lower than the threshold were changed to 0. Therefore, the area of the cells in the binary image generated by the threshold operations (shown in Figure 6f) was smaller than the area of the cells in the distance transform image (shown in Figure 6e). These smaller foreground regions represent the markers of the cells as shown in Figure 6f. Moreover, after the binary image was generated by the threshold operations, markers can be generated from the different blobs in the binary image. After obtaining the markers of the cells and the background as well as the gradient image of the cells as shown in Figure 6f, the watershed transformation [27] was used, and the markers were colored with different gray levels, i.e. white color for background, lighter gray color for the upper cell, and darker gray color for the bottom cell. In the watershed transformation, the lighter gray maker spreads over all pixels of the upper cell while the darker gray marker spreads over all pixels of the bottom cell. When different markers merged together, they can be recognized from different sources, and thus watershed lines were drawn to separate them as shown in Figure 6g. The cells were finally segmented and identified as shown in Figure 6h.
Identification of pipettes
The method for detection of pipette was developed when the pipette and cells appeared in the same image. The pipette was first calibrated to achieve two conditions for the identification. Firstly, the pipette should only be moved along the x-direction. Secondly, the length of the pipette on the x-direction in the image should be longer than one-third of the image. The grayscale image of the pipette and the cells is shown in Figure 8a. As similar to the method for identification of cells, the dilation followed by erosion was used. The threshold operation was then used to obtain the binary image. After the binary image was obtained as shown in Figure 8c, all blobs of white pixels that were not connected to the left and right edge of the image were filled with black pixels as shown in Figure 8d. After this operation, the cells which are connected to the edge still appeared in the image. Afterwards, the blobs were filled by black pixels except the block that has the longest distance in the x-direction as shown in Figure 8e. As from the prior calibration, the length of the pipette is longer than the length of the cell. The situation of some neighboring cells was also avoided by the operation of distance transform. The boundary of the pipette was identified, and the position of the tip of the pipette was determined.