Fun with Math: Probabilities

[The Atog Lord]

Thinking about Ad Nauseam got me pondering some math-related questions. How many Pact/Duress type cards do you need to include in your deck to have, say, a seventy five percent chance of finding one by the second turn? So, I decided to do some math, and then share my results with the TMD community. In brief, here are some findings:

By my calculations...
If you play 12 counters (Pact/Duress/Thoughtseize) in a deck, you have an 85% probability of finding one by your eighth card drawn.
If you have 8, that number becomes 70%.
And while I'm doing this out, the probability of having one or more of a 4-of in those eight cards becomes 44%

Here are some more detailed results.

Total In Deck : Probability (In Percent) of drawing at least one in 6 cards | in 7 cards  | in 8 cards

  4: 35 |   40 |  44
  5: 42 |   47 |  52
  6: 48 |   54 |   59
  8: 59 |   65 |   71
10: 68 |   74 |   79
12: 75 |   81 |   85

[Smmenen]

It is important to note that:

How many Pact/Duress type cards do you need to include in your deck to have, say, a seventy five percent chance of finding one by the second turn?

And:

If you play X counters (Pact/Duress/Thoughtseize) in a deck, you have an Y% probability of finding one by your eighth card drawn.

Are different questions.

The latter is easily calculated using HYPGEOMDIST(Z, 8, X, 60), and summing Z = 1, 2, 3, etc (or setting Z at zero and subtracting from 1).

However, the first is much, much more complicated.   Cards like Brainstorm, Ponder, tutors and make simple HYPGEOMDIST calculations very naive.   

[SiegeX]

The latter is easily calculated using HYPGEOMDIST(Z, 8, X, 60).

However, the first is much, much more complicated.   Cards like Brainstorm, Ponder, tutors and make simple HYPGEOMDIST calculations very naive.   

That gives you the probability of finding EXACTLY 'Z' counters in the first 8 draws.  What you want to know is the probability of finding at least one counter in the first 8 draws, to do that you set Z = 0 and then subtract the resulting percentage from 1.

1 - HYPGEOMDIST(0, 8, X, 60)

Cards like Brainstorm, Ponder, tutors and make simple HYPGEOMDIST calculations very naive.

I wouldn't say it makes the function itself naive, it just requires additional probabilities (all using HYPGEOMDIST if you so choose) to find the true probability.  However, said that, the 'naive' calculation of pulling the actual card in 'X' draws accounts for the majority of whatever the true probability is and thus is good enough for a ball park figure.

[Smmenen]

By "using" HYPGEODIST, I was referring to proper usage, which is as you outlined.

if I play turn one Necropotence, I technically won't ever get an 8th draw :p, yet I'm pretty sure my chances of finding one of my Duresses shoots through the roof.   

I was merely pointing out that the chance of finding something by your second turn and by your 8th draw are very far from the same number.  Without also calculating the chances of finding those cards from Ponder, Ancestral, Brainstorm, Necro, tutoring, etc. (and also the chance of those resolving), you won't find the correct chance of finding a card by your second turn. 

[The Atog Lord]

By "using" HYPGEODIST, I was referring to proper usage, which is as you outlined.

I have no idea what HYPNO GEO DIST is. Maybe some Microsoft function.

Anyways, having probabilities like these is useful because it lets you think about how to construct a deck with better numbers. Sure, Ponder changes things as does the ability of the opponent to counter. But you can only dig for so much in a combo deck -- you need to ensure that a certain threshold falls into your hand, and let your draw spells find the missing pieces.

[Duncan]

By "using" HYPGEODIST, I was referring to proper usage, which is as you outlined.

I have no idea what HYPNO GEO DIST is. Maybe some Microsoft function.

HYPGEOMDIST is a function in Microsoft excel and refers to the Hyper Geometric Distribution, which let you compute the probability that you draw have x succes in a sample of y with a population of Y and X being the number of success in the population.
For example:
By entering x = 1, y = 7, Y = 60 and X = 4 you get the chance that you draw exactly 1 copy of a 4-of in your deck in your opening 7.

[SiegeX]

Sure, Ponder changes things as does the ability of the opponent to counter. But you can only dig for so much in a combo deck

This is what I was trying to get at in my first response to Steve.  Yes, calculating just the probability of drawing the actual card itself and not factoring in tutors and search cards to get it is a back of the envelope type calculation but it will get you most of the way there.  I'm not about to calculate it the right way, but I wouldn't be surprised if your accuracy only goes up by 5%.

HYPGEOMDIST is a function in Microsoft excel and refers to the Hyper Geometric Distribution

This HYPGEOMDIST() also exists as a function (and I believe with the same parameter order) in OpenOffice's Calc program if you can't get access to Excel and want a free utility to play around with hypergeometric probabilities as is what we do in this card game.

Edit:

I had a explination here about why setting Z=0 works but I realized I'm preaching to the choir as Atoglord had quite a nice whitepaper on the proper choice of password length which included quite a bit of math.

[Duncan]

Involving Brainstorm, Ponder and the like in the calculation makes the equation rapidly going to a point where you can't easily calculate the probability. Let's not talk about the possibility that your opponent might counter your Brainstorm or plays a draw7/Sphere of Resistance on his opening turn.

Luckily the percentage with all those factors taken into account won't change more than say 5% in total (like SiegeX mentioned). Therefore approaching the probability with a simple calculation is quite accurate and far less time consuming.

Another fun table is the following:
              20         21         22         23         24         25         26
      2    0.324    0.313    0.300    0.286    0.269    0.252    0.234
      3    0.270    0.283    0.294    0.303    0.309    0.312    0.312
      4    0.124    0.142    0.160    0.178    0.196    0.214    0.232
Total   0.717   0.738   0.754   0.767   0.775   0.778   0.778

Where each probability can be seen as the chance you draw exactly 2-4 manasources in your opening 7 with your deck containing 20-26 mana sources. The bottom row is the total of 2, 3 and 4 manasources (the in my opinion mana-wise keepable hands in a straight forward, non-combo deck).

[ChemEng]

I have no idea what HYPNO GEO DIST is. Maybe some Microsoft function.

How did you calculate your probabilities without the Hyper Geometric Distribution?

Anyways, having probabilities like these is useful because it lets you think about how to construct a deck with better numbers. Sure, Ponder changes things as does the ability of the opponent to counter. But you can only dig for so much in a combo deck -- you need to ensure that a certain threshold falls into your hand, and let your draw spells find the missing pieces.

1. Your game results will only approach these probabilities over a VERY large number of games.
2. On the short term (in "game to game" time), the stochastic deck variations will drive your results.

[SiegeX]

How did you calculate your probabilities without the Hyper Geometric Distribution?

I'm going to venture a guess with a long list of conditional probabilities.

1. Your game results will only approach these probabilities over a VERY large number of games.
2. On the short term (in "game to game" time), the stochastic deck variations will drive your results.

True, but this is no different than card counting in blackjack.  Card counters still pick deviations from basic strategy based upon probabilistic outcomes because inevitability will eventually win over variance and fluctuations to get that 2% edge over the house (assuming you are consistent in keeping the correct count.)  I don't think anybody here is expecting anything different.

[The Atog Lord]

I calculated the probabilities using Open Office, but using a chain of much simpler calculations. But there's no need to use that anymore; this interested me to the point where I just wrote a small program to do this. It's in Ruby. If you have Ruby on your computer then it is very easy for you to use. And if you're running a halfway-decent operating system (the sort not created in Redmond), then you have Ruby on your machine already. To use this, simply copy and paste the code below into a plain text file and save it. Change the name to "probcalc.rb". Then in a shell window invoke it as follows:

probcalc <#copies> <#drawn(default=8)> <decksize(default=60)>

For example:

>probcalc 12 8 60
Probability = 0.852518596204902
85.25%

# Calculate the probability of drawing a particular card from a deck

def help
  puts "ProbCalc"
  puts "2008 Rich Shay"
  puts "Freeware"
  puts "Calculate the probability of drawing at least 1 copy of a card."
  puts "There are X copies of the card, Y cards are drawn, and Z cards in the deck."
  puts 'Usage: probcalc <#copies> <#drawn(default=8)> <decksize(default=60)>'
  Process.exit
end

# Check for help
if (ARGV[0] == "h" || ARGV[0] == "help" || !ARGV[0])
  help
end

# Gather Input
begin
  copies = ARGV[0].to_i
  raise if (copies < 1)
 
  drawn = (ARGV[1] or 8)
  drawn = drawn.to_i
  raise if (drawn < 1)
 
  decksize = (ARGV[2] or 60)
  decksize = decksize.to_i
  raise if (decksize<drawn)
 
rescue
  puts "Input Error"
  help
end

draw = 0
current_deck = decksize.to_f
other_remaining = (decksize - copies).to_f
product = 1.0
while (draw < drawn) do
  product *= (other_remaining)/(current_deck)
  other_remaining -= 1.0
  current_deck -= 1.0
  draw += 1
end

prob = 1.0-product
puts "Probability = #{prob}"
puts sprintf("%.2f%", prob*100)

Note: I just typed this out very quickly, so if you get a bug please let me know.

[ChemEng]

True, but this is no different than card counting in blackjack.  Card counters still pick deviations from basic strategy based upon probabilistic outcomes because inevitability will eventually win over variance and fluctuations to get that 2% edge over the house (assuming you are consistent in keeping the correct count.)  I don't think anybody here is expecting anything different.

The impact of variation with a Magic deck is very different from a standard deck(s) in Blackjack due to the different nature of the games.

First, Blackjack games are much quicker than Magic. This means that a good Blackjack player can expect results that are much closer to the deterministic probabilities. Of course there is still variation with the Blackjack deck, but the increased number of hands reduces the variation of the long term play probabilities as compared to Magic games.

Second, with Blackjack the expectation of good play results in hands not played. Magic doesn't hand an analogue to this. So if a good player is dealt 16's all evening, you can cut your losses as a night of bad beats and still wear the "good player" title. With Magic, every game you play is important towards advancing in the tournament. So if you are dealt the same "16's" all evening, you can talking about it with the rest of the 0-X's that are out of the tournament.

[SiegeX]

First, Blackjack games are much quicker than Magic. This means that a good Blackjack player can expect results that are much closer to the deterministic probabilities. Of course there is still variation with the Blackjack deck, but the increased number of hands reduces the variation of the long term play probabilities as compared to Magic games.

Agreed, a BJ player will have a larger sample size over the same amount of time than a Magic player allowing probabilistic effects to be more readily visible.

Second, with Blackjack the expectation of good play results in hands not played. Magic doesn't hand an analogue to this.

I don't necessarily agree with this.  The advantage doesn't come in not playing hands, it comes in your betting variations such that you bet very small when their are many small cards in the deck and you bet large when there count is high to maximize on the high probability of pulling superior hands.  It's not as if the counter refuses to play the next round if the count is bad (although I have done this from time to time on a really bad count pretending I get a phone call, but you can only pull this off once without bringing down the heat.)  I think the analogy for this in Magic would be a scenario where you have a good understanding of your opp's deck, and you've seen that he has already used up many of his counterspells and you decide to combo out with no protection.  You're going all in with the understanding that the likelyhood that he is holding another counterspell in his hand is small.

So if a good player is dealt 16's all evening, you can cut your losses as a night of bad beats and still wear the "good player" title. With Magic, every game you play is important towards advancing in the tournament. So if you are dealt the same "16's" all evening, you can talking about it with the rest of the 0-X's that are out of the tournament.

No matter who you are, you are going to run into a bad string of cards (both BJ and Magic) and that is just the way it is as you cannot control the fluctuations of the game.   However, and here is the key point, just because you may not see the benefits of playing a certain way immediately, it doesn't mean it wasn't the right play.  I think we should build our decks with probability in mind even with the understanding that it will take many many games to see the positive benefits.

[ChemEng]

I think we should build our decks with probability in mind even with the understanding that it will take many many games to see the positive benefits.

I'm not sure what positive benefits you are referencing here because you can not "train" your deck to operate in "deterministic" mode through repetition. It may very well be that all your test games with the deck present you with a perfect 40% chance of drawing a least 1 Force or Will in your opening 7, but all your tournament games have the 4 Force of Will always on the bottom of your deck. In spite of the number of games you have played with the deck, both deck state functions* are perfectly--and probabilistically--valid.

*This is a construct I have put together as a result of too many graduate level engineering classes to try to mathematically quantify the Magic deck. I'd be glad to talk about it some more if there is interest.

[The Acidic Hasidic]

i worked on an analysis of card control in magic. Its about 140 pages long and still unfinished, and i dont see myself finishing anytime soon.

if anybody is interested i will upload it, but youll need a deep understanding of analysis and combinatorics to understand it.

[SiegeX]

In spite of the number of games you have played with the deck, both deck state functions* are perfectly--and probabilistically--valid.

Perfectly and probabilistically valid, yes, likely to happen, no.    You could flip a fair coin a million times in a row and get heads each and every time, but I would be a fool to think that the next flip is more likely to be tails than heads, because its not.   Likewise, just because your last 10 games you never once pulled a FoW doesn't mean that you don't have a 40% chance of pulling one in your opening grip on the next game.  Although adjusting ones play based upon what happened in the past (in reference to probabilitic outcomes) seems like the correct thing to do, it is not and is even given its own name called The Gamblers Fallacy; and there is no better place to see how popular this fallacy is than the roulette tables in Vegas ala the Light Board of numbers that are "due" to hit.

*This is a construct I have put together as a result of too many graduate level engineering classes to try to mathematically quantify the Magic deck.

After graduating the 21st grade myself, you are in good company

i worked on an analysis of card control in magic. Its about 140 pages long and still unfinished, and i dont see myself finishing anytime soon.

Give us a link and I would be very interested in giving it a skim

[ChemEng]

I think we have firmly established that we each understand how the probabilities work. I give you passing marks into the 22nd grade.

[Laplie]

I have no idea what HYPNO GEO DIST is. Maybe some Microsoft function.

How did you calculate your probabilities without the Hyper Geometric Distribution?

You can figure out most probabilities involving magic decks using the binomial coefficient (the choose function). 

Background
The choose function, "c" hereafter, is the number of ways to choose k elements from a set of n (see http://en.wikipedia.org/wiki/Binomial_coefficient).  For example, 4c1 is 4 because there are 4 ways to choose 1 element from 4.  4c2 is 6 because there are 6 ways to choose 2 elements from 4.  Mathematically, c = n!/(k!*(n-k)!). I use my TI-83's "nCr" function.

Finding exactly 1 of a card within the first 7 cards when it's a 4of in a deck of 60 cards.

eq a:  (4c1 * 56c6) / 60c7 = .336 = 33.6%

Expanded out: (4 * 56!/(6!*50!))*7!*53!/60! = (4 * 51*52*53*54*55*56*7*/(54*55*56*57*58*59*60) = 3935568 / 11703240
(Those factorials cancel out nicely.  But most people will just type eq a into a calculator).

The top part of the equation is your hand.  You want 1 card out of 4, and then 6 cards out of 56.  The bottom part represents all of the possible first hands: choosing 7 cards from 60.  This equation can be easily modified to find exactly 2 in first 7:

4c2 * 56c5 / 60c7 = 5.93 %

or exactly 3:

4c3 * 56c4 / 60c7 = .38 %

or exactly 4:
4c4 *56c3/60c7 = .007%

or even exactly 2 in the first 10:
4c2 * 56c6 / 60c10 = .26%

Finding AT LEAST 1 of a card within the first 7 cards when it's a 4of in a deck of 60 cards.
You can find this by adding up the prob of exactly getting 1, 2, 3, or 4:

33.6 + 5.93 + .38 + .007 = 39.917% (hmm... that seems a bit wrong)

or more accurately you can find the probability of not drawing the card and subtracting it from 1:

56c7/60c7 = 60.05%

1 - 60.05% = 39.95%  (ah, that's better)

Finding EXACTLY 1 of a card and EXACTLY 1 of ANOTHER card within the first 7 cards when they are both 4of in a deck of 60 cards.

4c1*4c1*52c5/60c7 = 10.76%

Finding EXACTLY 1 of a card and NONE of ANOTHER card within the first 7 cards when they are both 4of in a deck of 60 cards.

4c1*52c6/60c7 = 21.08%

There are many more crazy probabilities I can show, but as other people have said, brainstorm, ponder,etc make these calculations less useful then they can be on say a deck of playing cards.  Also, in practice, I use the deep deck analysis tools in MWS rather than punch in all of these calculations by hand.

~Laplie

[TimsWheel]

I loves me some combinatorial math as much as the next man, but in my experience it is one of those things that ends up becoming a crutch to a lot of players. While it makes sense from a deckbuilding perspective to have a good idea about maximising your percentages (clearly at it's best in a comparatively linear combo deck), I've seen way too many perfectly good players allow the math to put them on tilt.

Knowing the chances of things going wrong are very low is psychologically setting yourself up to feel even worse when they do with surprising frequency.

[arctic79]

Statistics....Schamistics.  Either way you cut it, there are days where math checks out and bad luck checks in and you won't find one of 12 no matter how hard you try.  It's like those days you name a 4 of with Demonic Consultation and hit them all in the first 6 leaving the game. 

Luck (good or bad) is so much more fun and reliable then statistics.

[SiegeX]

Statistics....Schamistics.  Either way you cut it, there are days where math checks out and bad luck checks in and you won't find one of 12 no matter how hard you try.  It's like those days you name a 4 of with Demonic Consultation and hit them all in the first 6 leaving the game. 

Warning, I'm going to be pedantic in this post.

First, you should really be saying "probability..schmobability," because statistics deals with analyzing the frequency of past events, while probability deals with the likelihood of future events; and I don't think you have any issues with what has already happened.

Secondly, nothing you mentioned has caused math to "check out."   What people often don't realize is that an unlikely event is more likely to be observed as the number of events increases. 

Allow me to use the standard coin toss as a way to explain this.

The chances of flipping a fair coin heads 10 times in a row is  (0.5)^10 = 0.0977% (extremely unlikely)

However, the more we flip this coin, the more likely we are to observe a string of 10 heads in a row.  I'll spare you the math (unless asked for) but after 1421 flips, you are actually MORE likely (50.02%) to see a run of 10 heads in a row than you are not seeing it.

So as you can see, the experience you described doesn't cause math to "check out" but rather it justifies the theory that the more you play the more likely you are to see unlikely events happen.

[arctic79]

Ahhh....I was just trying to drop a little humour into a very dry subject. 
I understand probabilities but thanks for the remedial. 
Probability and luck are the same thing, luck is just an easier scapegoat.  When was the last time you heard a player with a great deck and good ability blame manascrew/flood or any other undesirable event on probability?  They always say luck was the factor. 
Tomato...Tomatoe.

[SiegeX]

Ahhh....I was just trying to drop a little humour into a very dry subject. 
I understand probabilities but thanks for the remedial. 

Sorry, I didn't pick up on the humor/sarcasm.   

Probability and luck are the same thing, luck is just an easier scapegoat.  When was the last time you heard a player with a great deck and good ability blame manascrew/flood or any other undesirable event on probability?  They always say luck was the factor. 
Tomato...Tomatoe.

Well, I'll agree somewhat.  I don't think they are the same thing but they are highly related.  Luck is not probability, luck is looking back on what happened and realized that what you observed was statistically unlikely to happen.

[The Atog Lord]

Probability and luck are the same thing

This is false. Probability is real and luck is not. The more you eschew luck and embrace probability, the more likely you are to win.

[arctic79]

This is false. Probability is real and luck is not. The more you eschew luck and embrace probability, the more likely you are to win.

Splitting hairs.

[SiegeX]

That hair could be as small as a gnat or as large as an oak tree depending on what luck means to you.  I found this quote on Wikipedia's page about luck which exactly expresses what I said in my last post but a bit more eloquently

"Luck is merely an expression noting an extended period of noted outcomes, completely consistent with random walk probability theory."

If this is your definition of luck, then welcome to the club.  However, if luck to you is an ethereal substitute for probability due to pure happenstance, then I foresee many split ends in our future.

[arctic79]

"Luck is merely an expression noting an extended period of noted outcomes, completely consistent with random walk probability theory."

Couldn't have said it better myself.

[dandan]

"The more I practice, the luckier I get." Arnold Palmer I think, I could be wrong, at least statistically.

That is the only useful definition of luck - the quantity that losers claim caused them to lose.

This may be a tangent, and despite the risk of getting a reputation as one who rambles more than a 50-year-old in shorts in the mountains, but I would like the challenge the almost universal acceptance of 'the right move'. Imagine a light is red 70% of the time and green 30% of the time. Guess the colour of the light. The 'right' guess is to always guess that the light is red. However the correct answer is red only 70% of the time. If someone guesses green and the light is green, they guessed correctly, their choice was right. However much you might be tempted to whinge about good luck or bad luck, statistics, probablilities, etc nothing changes the fact that the guess was right. Likewise in Magic, a decision (be it deck construction, SB, mulligan or play) may not be the best choice given the probability of whatever but may prove to be the right choice.

Now this may be splitting hairs (and we know that you can't target furry fish in Stealth bombers) but if you accept that a decision CAN be suboptimal/not the most probable/illogical/lucky/noob and still turn out to be correct, you may or may not be a better player for it (or indeed be exactly the same as you are, for that too is possible).

By all means understand probability but don't let it rule your world. Imagine playing poker against a computer that makes only logical calculated decisions and expects the same from you. Imagine you are playing Magic against a poor opponent. Imagine you are playing against a superior opponent (some of you may have to imagine a little harder than the rest of us). Optimal play isn't the same against each of them but probabilities are (probably).

[PETER FLUGZEUG]

To bring it back to magic: When my brother and me are talking of "I had bad luck" this always means "I have been playing shit, and I lost".
i.e. I had baaad luck with that demonic tutor finding the wrong card...
Luck is when you win although you played shit.

And for the guessing the red or green light thing: The optimal choice is not to always say it's red. the optimal choice is to always guess the right thing. Intuition?

[SiegeX]

Imagine a light is red 70% of the time and green 30% of the time. Guess the colour of the light. The 'right' guess is to always guess that the light is red.

No, the better ( it may not be right ) choice is the one that gives you the highest expected value (EV).  If guessing red or green had no consequences, then yes, 'red' would always be the better choice.  However, if you told me that you would pay me $10 if the light turned on red and $100 if the light turned green, then the better choice is now green.  This is because my EV for choosing red is 0.7*10 = $7, but my EV for choosing green is 0.3*100 = $30.

if you accept that a decision CAN be suboptimal/not the most probable/illogical/lucky/noob and still turn out to be correct, you may or may not be a better player for it (or indeed be exactly the same as you are, for that too is possible).

You'll find no argument here that an outcome that is not the most probable can be the one that actually happens.  However, if I know the play is suboptimal given all the variables, I'd be a fool to choose it regardless of what actually happens. 

Imagine playing poker against a computer that makes only logical calculated decisions and expects the same from you. Imagine you are playing Magic against a poor opponent. Imagine you are playing against a superior opponent (some of you may have to imagine a little harder than the rest of us). Optimal play isn't the same against each of them but probabilities are (probably).

You are adding more variables to the equation but are not accounting for them.  Yes, it is the case that regardless of your opp, the probability of pulling card 'A' from your deck is going to be the same each and every time given the same gamestate.  However, that says nothing about what is the better play to make a favorable outcome (you winning) more probable.  Bluffing, the chance of my opp making a play mistake, the chance of ME making a play mistake etc. all factor in to what is the better play to increase the probability of you winning.