Posted in | News | Machining Robotics

Robotic Manipulation of Deformable Objects

Having the ability to manipulate and work with deformable objects would mark itself as a milestone achievement and leap forward in the robotics industry as it would help address some of the open problems currently circulating in the field.

Image Credit: Lin, X., et al., (2022) DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools. International Conference on Learning Representation, [online] Available at: https://openreview.net/pdf?id=Kef8cKdHWpP

Working with deformable objects and materials presents a series of challenges as there are many research questions around deformables which include estimating and modeling an object’s shape and working out the properties of an object, including plasticity and elasticity.

Now, a team of researchers from MIT, Carnegie Mellon University and the University of California at San Diego has developed a new two-stage learning system known as DiffSkill that acts as a framework for a robotic manipulation system of deformable objects.

Firstly, a “teacher” algorithm solves any steps that a robot must make to complete a given task. Then, it feeds this to a “student” machine-learning model for training that can then apply the abstract ideas to learn new skills in order to complete each step of the task.

DiffSkill

A robotic arm working with a deformable such as dough can present a multitude of obstacles as the dough consistently shape-shifts as it is being worked, which can be difficult to render with an equation.

As working with dough generally requires a sequence of methods using various tools, asking a robot to learn a manipulation task with a number of steps — with various choices — can be difficult since learning tends to be a process of trial and error.

However, using DiffSkill’s teacher/student two-stage learning system could mean that a robotic arm learns how to complete all of the stages in a given sequence.

This method is closer to how we as humans plan our actions. When a human does a long-horizon task, we are not writing down all the details. We have a higher-level planner that roughly tells us what the stages are and some of the intermediate goals we need to achieve along the way, and then we execute them.

Yunzhu Li, Author of the Paper Presenting DiffSkill and Graduate Student in the Computer Science and Artificial Intelligence Laboratory (CSAIL)

In this teacher/student framework, the teacher component is an algorithm that is able to solve short-horizon tasks known as trajectory optimization framework that can calculate the differentiable physics of an object. Thereby, the “teacher” is able to learn and plot how the deformable object, in this instance, dough, must move at each stage of the process.

DiffSkill’s “student” component then emulates the teacher’s actions to link the different skills required to achieve the tasks’ objectives using two camera images, one that presents the dough in its current state and another showing the dough at the end of the task.

The neural network then develops a plan which determines how the various skills should be employed to reach the goal. Consequently, the system devises a series of short-horizon trajectories for each necessary skill and transmits the commands directly to the robotic arm and relevant tools.

Deformable Object Manipulation Strategies

The researchers ran a series of simulations to test and experiment with the DiffSkill technique in three different dough manipulation strategies. In one of the tasks, the robot armed with a spatula lifts dough onto a cutting board then takes a rolling pin to flatten it, while in another task, the robot is able to use a gripper to collect the dough and place it on a spatula, and transfers it to a cutting board.

In the final task, the robot was able to cut a piece of dough in half using a knife and apply the gripper to move the two pieces into two different locations. Furthermore, the researchers found that DiffSkill was able to simulate these tasks at a much better success rate than other trial and error-based reinforcement models.

DiffSkill was the only method able to conduct each three dough manipulation tasks successfully.

Surprisingly, the researchers also discovered that once up to speed, the “student” neural network had the ability to outperform the “teacher” algorithm in deformable object manipulation strategies.

Our framework provides a novel way for robots to acquire new skills. These skills can then be chained to solve more complex tasks which are beyond the capability of previous robot systems.

Xingyu Lin, Lead Author and Ph.D. student at the Robotics Institute, CMU

The next phase of the project involves using 3D data such as point-cloud representation rather than images to make the system more reliable and accurate. The long-term aim is that DiffSkill can be used across a wide range of robotic applications engaging with soft deformable objects, which includes working with textiles.

References and Further Reading

Lin, X., et al., (2022) DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools. International Conference on Learning Representation, [online] Available at: https://openreview.net/pdf?id=Kef8cKdHWpP

Zewe, A., (2022) Solving the challenges of robotic pizza-making. [online] MIT News | Massachusetts Institute of Technology. Available at: https://news.mit.edu/2022/robotic-deformable-object-0331

Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.

David J. Cross

Written by

David J. Cross

David is an academic researcher and interdisciplinary artist. David's current research explores how science and technology, particularly the internet and artificial intelligence, can be put into practice to influence a new shift towards utopianism and the reemergent theory of the commons.

Citations

Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Cross, David. (2022, April 04). Robotic Manipulation of Deformable Objects. AZoRobotics. Retrieved on June 29, 2022 from https://www.azorobotics.com/News.aspx?newsID=12865.

  • MLA

    Cross, David. "Robotic Manipulation of Deformable Objects". AZoRobotics. 29 June 2022. <https://www.azorobotics.com/News.aspx?newsID=12865>.

  • Chicago

    Cross, David. "Robotic Manipulation of Deformable Objects". AZoRobotics. https://www.azorobotics.com/News.aspx?newsID=12865. (accessed June 29, 2022).

  • Harvard

    Cross, David. 2022. Robotic Manipulation of Deformable Objects. AZoRobotics, viewed 29 June 2022, https://www.azorobotics.com/News.aspx?newsID=12865.

Tell Us What You Think

Do you have a review, update or anything you would like to add to this news story?

Leave your feedback
Your comment type
Submit