a. The final goal of this research is to make a robotic arm to segment the aim object from the background with the Point Cloud Library(PCL), then the robotic arm can do the Invers Kinematic(IK) motion planning by MoveIt in ROS. With the joint states and joint PWM we can communicate with Arduino through Serial Peripheral Interface(SPI), in order to control the robotic arm servos to grab the aim object. Fig 1. Shows the system architecture of this project.
The system architecture
b. Set up the robotic arm and download the required software for the project. Read researches related to the robotics and come up with your own robotics ideas. The require software and program are Ubuntu, Arduino, CH341 chipset driver, ROS Kinetic, Python and SPI.
a. Install required system and software
i.Ubuntu 16.04: Unix-like operating system, frequent choice of robotics community and scalable.
ii.Arduino IDE: Used for communication between computer and robotic arm.
iii.Ch341 chipset: Get computer recognize Arduino Nano
iv.Ros Kinetic: The robot operating system
b. Set up the hardware for robotic arm
i.Connect Arduino Nano to the robotic arm PCB using jumper wires
Pinout of the PCB socket that connects to Arduino Nano board
Pinout of the Arduino Nano board
ii.Connect the Adafruit breakout board to Arduino Nano board through SPI using jumper wires
Main motion planning and computer vision program will be written in ROS, So we need to set up a communication method for the computer to talk with the Arduino. Therefore, need to use Adafruit breakout board to split the port into SPI pins.
Pinout for the SPI header on Arduino Nano board
c. Run the program to control the robotic arm
i.Write and upload the Arduino program to control l the position of the servo motors. There are 6 servo motors in the robotic arm and they are connected to pin 2-7. Use l the position of the servo motors and the range for the PWM Microseconds is 500-2500 which means the different angle for the servo.
ii.Write python code to send PWM commands from your computer to Arduino using SPI
Need to set up Arduino to be Slave and my computer to be Master which means the Arduino listen to my computer through SPI communication.
iii.Learn ROS and be familiar with it
d. MoveIt Setup
i.With robotic arm's end-effector to some position or pose in 3D space, MoveIt can help us figure out what joint positions should be applied to the robot. Which means we use MoveIt to solve IK.
ii.Setup MoveIt and familiar with using MoveIt in ROS
iii.Since URDF contains links and joints connected together to describe the layout and kinematics of a robot. Write a URDF file for robotic arm which contains the geometry and architecture of robotic arm.
iv.Use MoveIt to setup robotic arm
v.Write the python code to drive robotic arm in rviz.
e. Connect MoveIt and SPI
i.Create a workspace and src folder for the research
ii.Git clone moveit_tutorials package since motion_planning package is built upon this package.
iii.Put all my package (robotic arm configuration package, motion planning package, arm control package)into the new workspace
iv.catkin_make the workspace
v.rosrun the code we should see the simulation for robotic arm move to specific location.
Robotic arm move to specific location
f. Calibrate from joint states to joint PWM
i.Mapping 6 points and find the related slop k and b for each joint. Which
satisfied the equation y = k + b
ii.Data and results
Table 1. Mapping from joint states to joint PWM
g. Camera setup
i.Install Intel Realsense SDK and Realsense ROS Wrapper
ii.Install rgbd launch package in order to launch the realsense camera node.
iii.Run camera together with motion planning and broadcast camera position relative to robotic arm
Broadcast camera position relative to robotic arm
h. PCL in ROS
ii.Write program in C++ to develop the application
iii.Convert the ROS pointcloud message into PCL pointcloud XYZRGB
iv.Use a Color-based region growing segmentation algorithm to segment the pointcloud based on colors.
v.Put the segmented individual point clouds into a pointcloud vector/array and display only the target point cloud, which is the point cloud that will be broadcast.
vi.Convert the target point cloud back to ROS's PointCloud message type and publish it so that we can view it from rviz.
Object captured by the camera
Color base segmentation (the orange region is our target object)
The final segmentation for the object
3.Problem and Improvement
a. For the segmentation part the background cannot contain too many different colors since we are doing the color base segmentation.
b. When calibrating from joint states to joint PWM, the k and b value for one of my robotic arm servo is not accurate which is the servo 2. And this servo use for the base of the whole robotic arm. So we have to redo the servo 2 and get more accurate values otherwise, the arm will not able to move to the desire position.
c. The robotic arm can only recognize the central of the certain object but the robotic arm only have two finger for grip. So this will be a problem if the object have a huge size.
It is a great honor to have the interview opportunity from graduate school. The interview went pretty well. I start with my brief introduction which includes my academic and education background. The first interview question that I had is the group project or research that impressed you most. I talked about the economical and practical prosthetic hand project that I joined in my sophomore year. The main purpose for this project is to designed an affordable prosthetic hand for people who loss their hand in a war or an accident and the hand should satisfy their requirements of daily life. The professor asked me about how we solve the problem of connection of muscle and the actual controlling of the hand. I replied that our project using the voice control device to control the hand and the device was already provided by professor and we mainly deal with the design of holding and rotating and came up with a simple prototype of the prosthetic hand. The professor follow up with the question about my future plan. I talked about my graduate school plan. I will apply for graduate school in the area of integrated circuit after I graduated. In order to have a deep understanding of my interesting area. Then the next question asked why I choose the electrical engineering. I mentioned my parents are both mechanical engineering and what they are doing really contribute to the society. So I want to be an engineer and it was not until I took a circuit course in my sophomore year that I finally found out electrical engineering is what I want to peruse in the future. The final question is how much I known about this university research program. I talked about the research paper that I read before the interview and discussed my own understanding of the research . The professor responded about the recent project that he was doing right now and the purpose of this research. We end up with the discussion about how to improve myself in electrical engineering area and the interviewer gave his useful suggestion.
Through this interview I have more clear goal for my future career. There always much more for me to learn and study in laboratory and research groups. So I really hope to be part of university research group and contribute my knowledge to the program.