"AI football player" is born, capable of dribbling, stealing, passing and shooting

Football is a globally popular ball sport that emphasizes physical confrontation and teamwork.

In real life, no matter whether people play football or not, whether they play well or not, whenever they see (hear) the word "football", they will chat about it vaguely.

Indeed, it is not uncommon for football to have so much news.

However, if an artificial intelligence (AI) learns to play football, and plays it well , would that be a relatively rare thing? After all, current AI sometimes can't even answer questions like "Was Lincoln the president of the United States 500 years ago?"

On September 1, a research team from the British artificial intelligence company DeepMind introduced for the first time an "AI football player" that can complete a variety of actions such as dribbling, physical confrontation, and then make accurate shots.

Figure | "AI players" are playing a 2V2 game (GIF source: Science Robotics)

Although the rules are occasionally broken, such as there are no set pieces such as corner kicks, penalties and free kicks in the game, the research team believes that this research has pushed artificial systems forward towards human-level sports intelligence.

The related research paper, titled “From motor control to team play in simulated humanoid football”, has been published in the scientific journal Science Robotics.

How did AI learn to play football?

As we all know, a standard football game involves two teams, each with 11 players, including 10 players and 1 goalkeeper, competing and attacking each other on the field.

Except for the goalkeeper who can touch the ball with his hands in his own penalty area, other players can only touch the ball with parts of the body other than their hands (except for throw-in). The ultimate goal of the game is to kick the football into the opponent's goal as much as possible.

Therefore, whether in real-life matches or online games, football is a sport that tests individual skills and requires tacit cooperation among team members . It is not easy to complete an excellent pass or an accurate shot, and "it requires solving many problems that both humans and animals will face."

According to the paper, although the DeepMind team simplified the rules of the game in this study and limited the number of players on both teams to 2-3, the results showed that AI players can learn skills such as kicking, dribbling and shooting without prior knowledge of football rules , and complete 2V2 and 3V3 football matches in simulated games.

"They achieve integrated control in an environment where motor skills and high-level goal-directed behaviors are tightly integrated," wrote Siqi Liu, co-first author and co-corresponding author of the paper, and his team.

Figure | "AI player" completes a series of actions such as dribbling, stealing and passing (graphic production source: Science Robotics)

As shown above, in a 2V2 match, the Red Team players completed a series of actions including running, pressing and passing. The whole action was very natural and smooth, and full of oppression.

So, how did a “football novice” transform into an “excellent football player”?

The reason lies in the three-step machine learning framework proposed by the research team.

First , the AI ​​player needs to learn to walk by watching human sports videos, because the initial AI player has no idea what to do on the football field.

Figure | Before training (GIF source: Science Robotics)

Then , after knowing what it is doing, the AI ​​player can learn to play football with the help of reinforcement learning algorithms.

Figure | After 3 days of training (GIF source: Science Robotics)

Finally , the AI ​​player went a step further and learned how to work in a team and perform more difficult motion control based on another reinforcement learning algorithm.

Figure | After 50 days of training (GIF source: Science Robotics)

It can be seen that when the AI ​​player in training continues to receive environmental rewards and gives correct feedback, its football playing level will improve.

From a "football novice" to an "excellent football player", AI has shown off again.

But it’s still not enough

In his classic remarks describing the foundations of cognitive science and AI, Turing Award winner Allen Newell argued that human behavior spans multiple levels of organization, from millisecond muscle twitches to hundreds of milliseconds of cognitive decision-making to long-term goal-directed behavior.

Higher-level behaviors are often associated with more complex interactions with the external environment and other entities . How to exhibit intelligent behaviors at multiple spatial and temporal scales is one of the long-term challenges facing physical entity artificial intelligence.

This research by the DeepMind team, through AI technologies such as reinforcement learning algorithms, has given AI players flexible motion control and multi-agent collaboration capabilities. It is an excellent example of how to learn multi-scale integrated decision-making in a multi-agent environment.

Figure | Confrontation and pushing (GIF source: Science Robotics)

Of course, this study also has certain limitations.

For example, due to the relatively small number of AI players participating in the game, there is still a certain lack of complexity compared to the real world , and it does not mean that AI can play more complex football games. If the number of AI players is increased in future work, more interesting team strategies may be generated.

In addition, due to the short time and fast pace, AI players are not given the ability to adjust the rhythm, maintain energy, decide the formation or replace players who are more defensive or offensive.

In addition, the research team also stated that their method is not suitable for learning directly on robot hardware, and the research results will not be quickly transferred from the simulated world to the real world.

However, the research results do demonstrate the potential of learning-based methods for generating complex movement strategies and also address one of the main challenges of simulation-to-real transfer that arises in simulation training.

At the end of the paper, the research team wrote, "How to achieve multi-scale motion intelligence of similar complexity on agile robot hardware will be one of the exciting research directions."

In the future, AI will be more powerful than you can imagine. What do you think?

References:

www.science.org/doi/10.1126/scirobotics.abo0235