Sutton and Barto give an example of a player having a total of 14 and choosing to stick. As we take more and more samples, the average should converge to the expected value.īlackjack is a good domain for using Monte Carlo reinforcement learning because we can easily generate sample hands while not needing to compute any probabilities. We don’t need to know anything about the internal workings of the machines – we can learn from the experience.

If we had five slot machines, we could pull the lever on each one a million times, average the results, and have a predicted return for each. Monte Carlo methods use actual or simulated experience to sample returns from an environment. If the dealer goes bust, then the player wins otherwise, the outcome-win, lose, or draw-is determined by whose final sum is closer to 21.” Monte Carlo Basics The dealer hits or sticks according to a fixed strategy without choice: he sticks on any sum of 17 or greater, and hits otherwise. If he goes bust, he loses if he sticks, then it becomes the dealer’s turn. If the player does not have a natural, then he can request additional cards, one by one (hits), until he either stops (sticks) or exceeds 21 (goes bust). He then wins unless the dealer also has a natural, in which case the game is a draw.

If the player has 21 immediately (an ace and a 10-card), it is called a natural. One of the dealer’s cards is face up and the other is face down. The game begins with two cards dealt to both dealer and player. We consider the version in which each player competes independently against the dealer. All face cards count as 10, and an ace can count either as 1 or as 11. “The object of the popular casino card game of blackjack is to obtain cards the sum of whose numerical values is as great as possible without exceeding 21. The rules of the game directly from the book are below: The Reinforcement Learning book by Sutton and Barto has a blackjack example in chapter 5 that led to many of the ideas in this post.

Monte Carlo Control with Seeing Both Dealer cards.

Monte Carlo Control with “Blackjack” and Doubling Down.

The OpenAI Gym Environment and Modifications.

Monte Carlo Control (Solving for an optimal policy).

Monte Carlo Prediction (Evaluating a fixed policy).

Blackjack monte carlo simulation