If you are interested in games, you surely had thought about your intelligent opponents’ strategies and actions, in the game world there are 3 main approaches for creating such behaviors:
- The Scanner Systems:
These agents or bots utilize the power of calculation to analyze and predict all possible chains of events, the good old chess engines -that could calculate all possible states with the depth of 4 or 5 moves-, some card game bots, etc use these methods.
- The Rule-Based Systems:
These systems use existing knowledge to decide what to do. The newer chess engines utilize pruning, positioning, and strategy knowledge.
- The AI-based Systems:
These systems learn how to play based on existing play logs or sometimes they learn the rules of the game on their own.
What is Beedel and how it is played?
Beedel is a card game (the original game known as Hearts). It needs 3 or 4 players -we focus on 4 player type- and has these rules:
– every heart card is a 1 point, so we have 13 points for all hearts
– the queen of spades is 13 points
– the winner is the one with minimum points, so the player should try to avoid getting hearts and queen of spades
– if someone gets all of the heart cards and also the queen of spades, he gets a zero and all other players get 26 points (the Bump round)
– the game plays till someone crosses 50 points and the winner is the player with minimum points
– there is some pass movement before the round start that we ignored here for the sake of simplicity of the document
So there is a complex approach here, you should avoid getting cards with points and also prevent other players from getting all the cards with points, these situations sometimes contradict each other.
How much the game is based on chance?
Because the cards are randomly distributed, there is a chance that the whole game is based on chance. A few months ago we introduced a rating system between players here are the top 10 players by rating win rate, it shows that skill of a player affects his/her win rate.
“Gorgosaurus” and “Torosaurus” bots:
We named Gorgosaurus as an offensive bot -a carnivorous dinosaur and one of the far cousins of TRex- and Torosaurus – a herbivorous dinosaur- as a defensive bot. We used an artificial neural network approach to create and train the bots. The input model consists of the following info -we trained two networks for each bot one for the passing phase and one for the playing phase the info is about the play phase-:
The selected player cards
The selected player played cards
The 3 other players played cards
The cards that are already on the table
All players points
We used one hot encoding technique to model the Input/Output.
We used the state and decisions of the winner of each round as the input and output, for Gorgosaurus to be aggressive we used all winners’ decisions and for Torosaurus we used non Bump rounds so it will be more defensive.
The learning dataset uses both skilled and fresh players’ decisions.
The win rate of bots over 142 days of playing shows this approach is promising.
Using skilled players decisions for the learning dataset
Using bot vs bot plays and reinforcement learning to improve the system’s intelligence.