Online teleoperation of writing manipulator through graphics processing unit based accelerated stereo vision

Remotely operated robots are of great use in unstructured environments where their autonomous counterparts are challenged. These robots are commonly operated using knobs, joysticks or wearable motion sensors. These modes of operation either require a skilled operator or hinder the natural movement. Therefore, contactless motion trackers such as stereo vision, structured light and time of flight systems were invented to allow natural control with minimum training. Stereo vision systems have the advantage over other systems of being cheaper while having scalable accuracy and range. These benefits however are challenged by the high computational requirements of the algorithms used. In this thesis, a framework is developed to enable the use of stereo vision in realtime teleoperation of a manipulator robot for the task of writing. The proposed algorithms aim to accelerate the processes of distortion removal and rectification using precomputed combined maps. Furthermore, the algorithms avoid computationally intensive correspondence problem by matching only the points of interest of a labeled pen to compute its position and orientation. These algorithms are then parallelized using Compute Unified Device Architecture CUDA C language to run on Graphics Processing Unit GPU for hardware acceleration. Like most sensors stereo camera readings are susceptible to noise due to lighting conditions and detection errors. Most techniques used in filtering noise such as the simplest moving average or polynomial spline fitting either eliminates some original important data along with the noise or are not optimized for realtime filtering. Thus, Kalman filter which is popular in tracking applications is proposed. Kalman filter is applied to readings along with the Savitzky-Golay and Moving Average filters. The performance of filtering methods is compared in term of processing speed and Root Mean Square Error (RMSE) with ground truth data collected from a high accuracy digitizing tablet. The proposed technique has demonstrated a significant improvement by reducing processing time and having low RMSE that is nearly equivalent to Savitzky-Golay filter and lower than Moving Average filter. Furthermore, it does not require a window that contains future data samples. Cameras capture images at constant frame rates coupled with hand movement speed variations, this can result in the trajectory points generated have varying movement distance and exceeding the robot speed constraints. Furthermore, most manipulator robots in use are designed with the application in mind being pick and place. Therefore, they follow a speed profile between the pick and place points that begins with acceleration, cruise and finally deceleration. When sending a stream of points to robot controller the speed profile results in the robot having a jittery behavior. To deal with these issues, adaptive sampling algorithm is applied to the data points to adhere to the manipulators speed constraints. Then the response of the internal low-level PID controller was examined and gains were tuned to reduce the rise time to meet real-time requirements. To find the joint angles from the position and orientation of end effector the inverse kinematics equations of the arm are computed beforehand. Then joint angles are streamed at 0.02 seconds interval to ensure that the end effector glides smoothly along the drawn trajectory. The performance of the whole system is evaluated by taking the end effector pose readings from internal sensors through Real-Time Data Exchange (RTDE) port and comparing those against the data from the digitizing tablet. The results demonstrated that the movement path is traced by the manipulator while maintaining minimum error of 0.95 to 1.81mm and minimum delay compared to other implementations. The positional error is reduced by 52% compared to the best-known implementation with captured frames being processed in real-time.

Text FADI IMAD - IR.pdf Download (1MB)

Actions (login required)

View Item