IBM has created an artificial intelligence software system that can do play-by-play and color commentary of soccer videos.

While doing play-by-play, the system tracks players in real-time and can identify passes, crosses and shots on goal. For pre-selected video highlights, it can also incorporate commentary based on statistics and standings drawn from a database, matching the information to whatever is happening in the video.

The system builds on work the company has already done in creating software that can automatically create highlight videos, honing in on the most exciting moments in sporting events such as tennis and golf.
The company is the official technology partner for several Grand Slam tennis tournaments, including Wimbledon and the U.S. Open, as well as the Masters PGA tournament.

“Soccer is a more challenging sport in terms of what can happen on the field and the number of players the system needs to track,” said John R. Smith, head of A.I. Tech for IBM A.I. Research.

The software, which Smith said is still very much a research project, was trained on videos of about 50 soccer games and their play-by-by commentary. It is what researchers call a single “end-to-end” model, meaning the system receives the raw video as input and tries to predict the right play-by-play commentary to output from that video.

Dan Gutfreund, an IBM researcher who worked on the project, says that for the color commentary for highlight clips, the player identification is done manually, but the software automatically pulls relevant statistics from a knowledge base in real time.

The company demonstrated the system for the first time publicly at the Neural Information Processing Systems (NeurIPS) A.I. conference in Vancouver, Canada, on Sunday.

In a demonstration on five short video clips the system had not encountered during its training, the software seemed to perform well, although it tended to repeat the same phrases, such as “here comes the cross,” more often than a human commentator would and it misidentified at least one pass along the sidelines as a cross-field kick.

“It is definitely still a work in progress,” Smith said.

One thing the system doesn’t do in its play-by-play commentary, for instance, is emote. It registered a goal being scored in the same monotone computer-generated speech as it did with a shot that went far wide. There was no Mexican-sportscaster-style exaggerated and enthusiastic “GOOOOOOAAAAL!!!”

“Yes, that is something we’ve thought about incorporating in the future,” Smith said.

He said he could envision this kind of software helping both amateur and semi-pro teams, whose games might not be covered by professional play-by-play sportscasters, to provide live play-by-play as well as to better market video highlights. The color commentary, meanwhile, could help even professional sporting teams and broadcasters reduce the time and expense needed to produce highlight reels.

IBM’s was one of just several companies showcasing A.I. applications in sports at NeurIPS. Montreal-based company SportLogiq, a technology company that provides data analytics to professional sports teams and broadcasters, including most of the National Hockey League, showcased computer vision systems that tracks players in live video, automatically generates statistics on their performance and can track events, such as passes and goals.

Michael Horton, one of Sportlogiq’s machine learning researchers, said that in hockey, the company has also helped teams find players with similar abilities, helping to facilitate trades.