Ever wondered how Libratus, the celebrated poker playing (and winning) AI software from Carnegie Mellon University, outsmarts its opponents? Turns out Libratus uses a three-pronged strategy which its inventors share in a paper published online yesterday in Science – Superhuman AI for heads-up no-limit poker: Libratus beats top professionals.

Libratus has been turning heads for some time with its ability to win against professional gamblers in Texas Hold’em, a game that emphasizes bluffing. “AI programs have defeated top humans in checkers, chess and Go — all challenging games, but ones in which both players know the exact state of the game at all times. Poker players, by contrast, contend with hidden information — what cards their opponents hold and whether an opponent is bluffing,” according to an interesting account on the CMU website.

The proof, of course, is in the winning; Libratus did this in spades at a 20-day, 120,000-hand competition last year at Rivers Casino, Pittsburgh. It was the first time an AI defeated top human players at Texas Hold’em. Libratus won $1.8 million in chips. (Too bad they couldn’t be cashed in). “As measured in milli-big blinds per hand (mbb/hand), a standard used by imperfect-information game AI researchers, Libratus decisively defeated the humans by 147 mbb/hand.”

Apparently HPCwire readers also like the idea of winning. Libratus received the HPCwire Reader’s Choice Award for Best Use of AI at SC17. Tuomas Sandholm, professor of computer science, and Noam Brown, a Ph.D. student in the Computer Science Department, detail how their AI was able to achieve “superhuman” performance by breaking the game into computationally manageable parts. They also explain how, based on its opponents’ game play, Libratus fixed potential weaknesses in its strategy during the competition.

“The techniques in Libratus do not use expert domain knowledge or human data and are not specific to poker,” report Sandholm and Brown in the paper. “Thus they apply to a host of imperfect-information games.” Such hidden information is ubiquitous in real-world strategic interactions, they noted, including business negotiation, cybersecurity, finance, strategic pricing and military applications.

Here is an excerpt from the paper describing Libratus’ three main modules:

• “The first module computes an abstraction of the game, which is smaller and easier to solve, and then computes game-theoretic strategies for the abstraction. The solution to this abstraction provides a detailed strategy for the early rounds of the game, but only an approximation for how to play in the more numerous later parts of the game. We refer to the solution of the abstraction as the blueprint strategy.
• “When a later part of the game is reached during play, the second module of Libratus constructs a finer-grained abstraction for that subgame and solves it in real time. Unlike subgame-solving techniques in perfect-information games, Libratus does not solve the subgame abstraction in isolation; instead, it ensures that the fine-grained solution to the subgame fits within the larger blueprint strategy of the whole game. The subgame solver has several key advantages over prior subgame-solving techniques. Whenever the opponent makes a move that is not in the abstraction, a subgame is solved with that action included. We call this nested subgame solving. This technique comes with a provable safety guarantee.
• “The third module of Libratus – the self-improver – enhances the blueprint strategy. It fills in missing branches in the blueprint abstraction and computes a game-theoretic strategy for those branches. In principle, one could conduct all such computations in advance, but the game tree is way too large for that to be feasible. To tame this complexity, Libratus uses the opponents’ actual moves to suggest where in the game tree such filling is worthwhile.”

As the CMU researchers point out, one can imagine many “contests” with hidden information in which Libratus AI software techniques might be used.

Link to Science paper: http://science.sciencemag.org/content/early/2017/12/15/science.aao1733.full

Link CMU article: https://www.cmu.edu/news/stories/archives/2017/december/ai-inner-workings.html