Knowledge sharing will be the new direction for the future explosion of robots

Many of the tasks that humans would like robots to do, such as packing items in warehouses, helping bedridden patients, or assisting soldiers on the front lines, cannot be completed because robots cannot recognize and handle common objects.

We can usually fold our own laundry or pick up a water bottle because we've already gone through "big data collection" in childhood, says Stefanie Tellex, a professor of computer science at Brown University. If robots are to perform the same daily tasks, they need access to large amounts of data about how to grasp and manipulate objects. But where does that data come from? Usually, it takes painstaking programming to generate it. Ideally, though, robots could learn from each other.

That’s the theory behind Thales’s Million Object Challenge, which aims to have research robots around the world learn how to find and handle simple objects, from bowls to bananas, while uploading data to the cloud and allowing other robots to analyze and use the information.

Stephanie Telles and Baxter Robot

Thales' lab in Providence, Rhode Island, is like a super-fun kindergarten. On the day I visited, Baxter, an industrial robot made by Rethink Robotics, was standing among a large number of oversized building blocks, scanning a small brush. It moved its right arm back and forth toward the object opposite, using a camera on its body to take multiple pictures and infrared sensors to measure the depth. Then, it used its hands (grippers) to try to grasp the brush from different angles. Once it lifted the object in front of it, it began to shake the object to make sure it was firmly grasped. If it can complete these movements, it means that the robot has learned how to pick up an object.

The robot can often pick up a different object in each gripper, working around the clock. Telles and her graduate student, John Oberlin, have collected data on about 200 items and have begun sharing it, starting with things like children’s shoes and moving on to plastic boats, rubber ducks, cooking utensils like garlic presses, and a duckbill cup that once belonged to her 3-year-old son. Other scientists can also contribute data from their robots, and together Telles hopes they can build a library of information about how the robots handle a million different objects. Eventually, “the robots will be able to recognize a pen in front of them on a crowded shelf and pick it up,” Telles says.

This kind of project is possible because many research robots can be programmed using the same standard framework, ROS (Read Only Memory). Once a machine has learned a given task, it can pass data to other robots, which can upload feedback that can further refine the original data. Thales believes that this information to identify and grasp any given object can be compressed into 5-10MB of data, which is about the same memory size as a song in your music library.

Whenever the robot determines the best way to grasp an object, it archives the data in a format that other robots can use.

Thales was an early partner in the Robobrain project, which demonstrated how one robot could learn from the experience of other robots. Her partner, Ashutosh Saxena, then taught the PR2 robot at Cornell to lift a small cup and place it in a designated spot on a table. Then, at Brown, Thales downloaded the information from the cloud and used it to train her Baxter (which is constructed differently from the PR2 robot) to perform the same task in a different environment.

Right now, such progress may seem incremental, but in the next five to 10 years, we will see “an explosion in the capabilities of robots,” said Saxena, now CEO of a startup called Brain of Things. As more researchers contribute and refine knowledge in the cloud, Saxena believes, “robots should be able to easily access all the information they need.”