3D Vision Bin Picking: A Technical Guide to Picking Loose Parts with a Robot

A container full of jumbled parts

Parts arrive from stamping, forging or the supplier in a metal container, piled up in no particular order. Someone has to pick them up one by one and place them in the machine, on a conveyor or in a tray. It is a simple task for a person, but a very demanding one for a robot: it needs to know exactly where each part is, how it is oriented and where it can grip it without colliding.

In the article on machine vision in robots we introduced bin picking as one of the most widespread applications of 3D vision. Here we go into the technical detail: which parts are good candidates, what drives cycle time and what should be validated before planning a project.

The four phases of each cycle

• Capture. A 3D camera, usually mounted above the container, generates a point cloud: millions of coordinates describing the surface of everything in its field of view.

• Recognition. The software compares that point cloud with the part model, taken from its CAD or a learned sample, and identifies the visible parts along with their position and orientation.

• Grasp selection. Among all the detected parts, it chooses the one that can be picked most safely, based on the defined grasp points and whether it can be reached without colliding with the container walls or other parts.

• Planning and execution. It calculates a collision-free path, the robot picks the part and places it at its destination.

Specialist manufacturers such as Mech-Mind offer industrial 3D cameras and vision software designed for this type of application, compatible with the main robot brands, both industrial and collaborative.

Which parts are good candidates

Not all parts are equally easy. These are the factors with the greatest influence:

Geometry

Rigid parts with recognizable shapes and clear gripping surfaces, such as shafts, flanges, forgings or castings, usually give good results. Very flat parts that overlap, small parts in large quantities or parts that hook into each other, such as springs or hooks, are much more demanding.

Surface and material

Very shiny or reflective surfaces, such as machined aluminium or polished stainless steel, produce reflections that make 3D capture difficult. Matt black or transparent parts are also challenging. Today's cameras have improved a lot in this respect, but it is always worth testing with real parts.

Gripper type

The gripper must be able to fit between the parts and the container walls. Magnetic grippers work well with steel parts; vacuum grippers with smooth, flat surfaces; mechanical grippers with narrow fingers with parts that have holes or features to hold on to. Gripper design is often as decisive as the software.

Placement accuracy and cycle time

Placement accuracy. When picking a part from the pile, the robot does not always grip it at the same point or with the same orientation. If the destination requires precise placement, such as a lathe chuck, an intermediate re-gripping station or a second vision check is usually needed before loading the machine.

Cycle time. The cycle includes capture, processing, path planning and movement. In certain situations the system finds no accessible part and needs a new capture or a slight movement of the container. The cycle must be sized with these situations included, not just the ideal case. A common technique is to capture the next image while the robot is placing the previous part, so that vision adds no time to the cycle.

The challenge of emptying the container completely

Picking the first parts from a full container is relatively easy. The challenge lies in the last ones: parts in the corners, against the walls or in positions the gripper cannot get into. A system that empties almost all of the container and leaves the rest to the operator may not be acceptable in a cell meant to run unattended.

Container geometry, camera position and gripper design are the variables with the greatest influence here. In some cases, tilting the container or using containers with lower walls noticeably improves emptying.

Cobot or industrial robot for bin picking

Bin picking is not tied to one type of robot. With light parts and moderate throughput, a cobot simplifies the installation and allows operators to work nearby. With heavy parts or very short cycles, an industrial robot offers more speed and payload. What matters is that the camera and software integrate properly with the controller of the chosen robot.

Where it is used

• Feeding CNC machines from containers of raw parts, as a complement to CNC machine tending with a cobot.

• Feeding presses, assembly lines or welding stations.

• Unloading castings or forgings.

• Kitting components in intralogistics.

• Sorting mixed parts from the same box.

Why test before committing

Bin picking is probably the robotics application where the gap between a demonstration with ideal parts and the reality of the plant is most noticeable. Oily parts, deformed containers, variable lighting or batches with small differences can change the result considerably. That is why it is advisable to run a proof of concept with your own parts and your own containers.

At Rebots we use Mech-Mind 3D cameras combined with any type of robotic arm for bin picking, quality control and mixed-case (de)palletizing. On the 3D camera bin picking solution page you can see the general approach. If you want to know whether your parts are feasible, send us some photos of the container and the part and we will assess it with you.