Having the ability to manipulate and work with deformable objects would mark itself as a milestone achievement and leap forward in the robotics industry as it would help address some of the open problems currently circulating in the field.
Image Credit: Lin, X., et al., (2022) DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools. International Conference on Learning Representation, [online] Available at: https://openreview.net/pdf?id=Kef8cKdHWpP
Working with deformable objects and materials presents a series of challenges as there are many research questions around deformables which include estimating and modeling an object’s shape and working out the properties of an object, including plasticity and elasticity.
Now, a team of researchers from MIT, Carnegie Mellon University and the University of California at San Diego has developed a new two-stage learning system known as DiffSkill that acts as a framework for a robotic manipulation system of deformable objects.
Firstly, a “teacher” algorithm solves any steps that a robot must make to complete a given task. Then, it feeds this to a “student” machine-learning model for training that can then apply the abstract ideas to learn new skills in order to complete each step of the task.
DiffSkill
A robotic arm working with a deformable such as dough can present a multitude of obstacles as the dough consistently shape-shifts as it is being worked, which can be difficult to render with an equation.
As working with dough generally requires a sequence of methods using various tools, asking a robot to learn a manipulation task with a number of steps — with various choices — can be difficult since learning tends to be a process of trial and error.
However, using DiffSkill’s teacher/student two-stage learning system could mean that a robotic arm learns how to complete all of the stages in a given sequence.
This method is closer to how we as humans plan our actions. When a human does a long-horizon task, we are not writing down all the details. We have a higher-level planner that roughly tells us what the stages are and some of the intermediate goals we need to achieve along the way, and then we execute them.
Yunzhu Li, Author of the Paper Presenting DiffSkill and Graduate Student in the Computer Science and Artificial Intelligence Laboratory (CSAIL)
In this teacher/student framework, the teacher component is an algorithm that is able to solve short-horizon tasks known as trajectory optimization framework that can calculate the differentiable physics of an object. Thereby, the “teacher” is able to learn and plot how the deformable object, in this instance, dough, must move at each stage of the process.
DiffSkill’s “student” component then emulates the teacher’s actions to link the different skills required to achieve the tasks’ objectives using two camera images, one that presents the dough in its current state and another showing the dough at the end of the task.
The neural network then develops a plan which determines how the various skills should be employed to reach the goal. Consequently, the system devises a series of short-horizon trajectories for each necessary skill and transmits the commands directly to the robotic arm and relevant tools.
Deformable Object Manipulation Strategies
The researchers ran a series of simulations to test and experiment with the DiffSkill technique in three different dough manipulation strategies. In one of the tasks, the robot armed with a spatula lifts dough onto a cutting board then takes a rolling pin to flatten it, while in another task, the robot is able to use a gripper to collect the dough and place it on a spatula, and transfers it to a cutting board.
In the final task, the robot was able to cut a piece of dough in half using a knife and apply the gripper to move the two pieces into two different locations. Furthermore, the researchers found that DiffSkill was able to simulate these tasks at a much better success rate than other trial and error-based reinforcement models.
DiffSkill was the only method able to conduct each three dough manipulation tasks successfully.
Surprisingly, the researchers also discovered that once up to speed, the “student” neural network had the ability to outperform the “teacher” algorithm in deformable object manipulation strategies.
Our framework provides a novel way for robots to acquire new skills. These skills can then be chained to solve more complex tasks which are beyond the capability of previous robot systems.
Xingyu Lin, Lead Author and Ph.D. student at the Robotics Institute, CMU
The next phase of the project involves using 3D data such as point-cloud representation rather than images to make the system more reliable and accurate. The long-term aim is that DiffSkill can be used across a wide range of robotic applications engaging with soft deformable objects, which includes working with textiles.
References and Further Reading
Lin, X., et al., (2022) DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools. International Conference on Learning Representation, [online] Available at: https://openreview.net/pdf?id=Kef8cKdHWpP
Zewe, A., (2022) Solving the challenges of robotic pizza-making. [online] MIT News | Massachusetts Institute of Technology. Available at: https://news.mit.edu/2022/robotic-deformable-object-0331
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.