Robots are learning how to do our chores

Encoding such detailed instructions through fixed rules and predefined mathematical formulas is highly time-consuming. Researchers at TU Wien adopted an alternative approach: a human demonstrates the desired task multiple times using a specially equipped sponge to clean the sink's edge. By observing these demonstrations, the robot learns the cleaning process and can adapt this knowledge to objects of varying shapes. This research has since been presented at IROS 2024 in Abu Dhabi.

Surface treatments

Cleaning is just one example of surface treatment, a category that includes other critical industrial processes such as sanding, polishing, painting, and adhesive application. These tasks share similar technical challenges.

“Capturing the geometric shape of a washbasin with cameras is relatively straightforward,” explained Prof Andreas Kugi from the Automation and Control Institute at TU Wien. “But that’s not the key challenge. The real difficulty lies in teaching the robot: What kind of movement is needed for each part of the surface? How fast should it move? What angle is required? And how much force should be applied?”

Humans acquire such skills through practice and observation. Christian Hartl-Nesic, head of the Industrial Robotics group within Kugi’s team, compared it to a workshop scenario. An apprentice might have someone looking over their shoulder, advising them to press harder on a specific edge. So, the team aimed to enable the robot to learn in a way that mirrors this human approach.

Utilising demos

To facilitate this process, a specialised cleaning tool was developed: a sponge equipped with force sensors and tracking markers. Humans used this tool to clean a sink's front edge repeatedly. “We collect an extensive dataset from just a few demonstrations, which is then processed to teach the robot what effective cleaning entails,” explained Christian Hartl-Nesic.

The learning process relied on a novel data processing strategy devised by the TU Wien research team, combining several established machine learning techniques. Initially, the measurement data was statistically analysed. This information was then used to train a neural network to understand predefined movement patterns, referred to as "motion primitives." Following this, the robot arm was programmed to execute these motions optimally, enabling it to clean the surface.

The resulting algorithm allowed the robot to clean not only the entire sink but also other objects with complex surfaces. Remarkably, this capability was achieved after being trained only on cleaning a single edge of the sink. "The robot learns that you have to hold the sponge differently depending on the shape of the surface, that you have to apply a different amount of force on a tightly curved area than on a flat surface," explained Christoph Unger, a PhD student from the Industrial Robotics group.

Learning together

The technology demonstrated can be applied across various industries, including sanding wooden pieces in joineries, repairing and polishing vehicle paint, or welding sheet metal components in metalworking. Future iterations could feature robots mounted on mobile platforms, enabling them to serve as versatile assistants anywhere within a workshop.

Another potential advantage is the ability for these robots to exchange knowledge. “Imagine multiple workshops employing self-learning robots for tasks like sanding or painting surfaces,” explained Andreas Kugi. “Each robot could independently gather experience using local data while sharing the parameters it has learned with others.” This concept, called federated learning, allows robots to share fundamental principles without compromising private data, such as the specific geometry of individual workpieces, thereby improving the overall capabilities of all robots involved.

Extensive testing at TU Wien has confirmed the sink-cleaning robot’s flexibility. The technology has also drawn international acclaim. At IROS 2024, held from 14th to 18th October, a conference that attracted over 3,500 scientific paper submissions, TU Wien’s research received the "Best Application Paper Award," recognising it as one of the year’s most notable innovations.