When communication lines are open, separate agents like drones or robots can work together to collaborate and finish a task.
Huy and his colleagues tested their algorithms using simulated games like StarCraft, a popular computer game. Image Credit: University of Illinois Urbana-Champaign
However, what if they are not fitted with the correct hardware or the signals are blocked, making communication impossible? Scientists from the University of Illinois Urbana-Champaign attempted to overcome this difficult challenge.
They came up with a technique to train several agents to work collectively by making use of multi-agent reinforcement learning, a kind of artificial intelligence.
It’s easier when agents can talk to each other. But we wanted to do this in a way that's decentralized, meaning that they don't talk to each other. We also focused on situations where it's not obvious what the different roles or jobs for the agents should be.
Huy Tran, Aerospace Engineer, University of Illinois Urbana-Champaign
Tran stated that this scenario is highly complicated and a harder issue since it is not clear what one agent must do versus another agent.
Tran states, “The interesting question is how do we learn to accomplish a task together over time.”
Tran and his collaborators utilized machine learning to resolve this issue by making a utility function that tells the agent when it is doing something beneficial or good for the group.
With team goals, it's hard to know who contributed to the win. We developed a machine learning technique that allows us to identify when an individual agent contributes to the global team objective.
Huy Tran, Aerospace Engineer, University of Illinois Urbana-Champaign
Tran added, “If you look at it in terms of sports, one soccer player may score, but we also want to know about actions by other teammates that led to the goal, like assists. It’s hard to understand these delayed effects.”
Also, the algorithms that were developed by the scientists can determine when an agent or robot is doing something that is not considered the aim. “It’s not so much the robot chose to do something wrong, just something that isn’t useful to the end goal,” they added.
They tested their algorithms using simulated games like Capture the Flag and StarCraft, a popular computer game.
StarCraft can be a little bit more unpredictable—we were excited to see our method work well in this environment too.
Huy Tran, Aerospace Engineer, University of Illinois Urbana-Champaign
Tran stated this kind of algorithm could be applicable to several real-life situations, like military surveillance, robots working jointly in a warehouse, autonomous vehicles coordinating deliveries, traffic signal control, or regulating an electric power grid.
Seung Hyun Kim did the majority of the theory behind the concept when he was an undergraduate student studying mechanical engineering, with Neale Van Stralen, an aerospace student, assisting with the implementation.
Tran and Girish Chowdhary advised both students. The work was presented recently to the AI community at the Autonomous Agents and Multi-Agent Systems peer-reviewed conference.
Training robots to play Capture the Flag
Video Credit: University of Illinois Urbana-Champaign.