[draft article] Statistics for analysing tournament results.

[the Luke]

Hi all,

After conversing with Dr. Sylvan and Smmenen about tournament statistics, I came up with some ideas based on my PhD research.  I thought I'd put the article up to share my theories and get some criticism.  The last half of the article was a bit rushed, so please keep that in mind.

http://www.csse.monash.edu.au/~lhope/mtg/Bayesian/Bayesian.html


-Luke

[Hi-Val]

That is some damned powerful math, I'll have to take your word for it. In the final version, is there a way to dumb that down so lib arts majors like me can understand it? The article looks very valuable.

[Machinus]

Is r just the average win rate for all possible matchups in a given tournament? The win rate won't be the same for every deck you face, but it seems like it ought to be adjusted according to the number of each deck, i.e, not just the average for all possible matchups, but the average win rate for every deck at the tournament, even if there are multiple copies of the same deck. Great analysis!

[skecreatoR]

That is some advanced math for a fifteen year old, but I even managed to understand it, and I thank you a lot for the work and the amount of thought put into this.

Probably the best statistical thing I read in a while - very good job!

[Swanky]

To a lover of statistics, this is like a cool glass of water on a summer day.  Bravo!

[PucktheCat]

Very interesting article.

One problem is that r will be overestimated because only "successful" results are calculated. This also makes popular decks seem more successful: because they make up more of the population, they're just more likely to make top-8s.

Isn't this like . . . a really big problem with the analysis though?  Since most decks don't have a huge margin against the field any given single deck is likely to be eliminated even if it is the best choice against the field.

Leo

[defector]

I'm jealous, I'll have to break out my calc book just to get a feel for what your doing.  Excellent work, I for one find this much more useful than having someone re-apply the prisoners dilemma.  Thanks for the good effort, now Dr. Sylvan has someone to play numbers with as well as Magic
defector

[Dr. Sylvan]

now Dr. Sylvan has someone to play numbers with as well as Magic

All I know how to do is count and add... the Luke is a real numbers guy. :)

[the Luke]

Machinus: r is the win-rate given an unknown opponent, and yes, it's not going to be the same at all for certain matchups.  I really just can't analyse tournament results according to matchups for two main reasons:

1) I don't have the data!  It's a very limited reporting procedure as is.  The BEST I could possibly expect is a copy of the DCI reporter results the  TO generates. Matchup data goes beyond this.

2) The analysis would be hugely complicated, and since new archetypes spring into existence regularly, there's no good way of coming up with accurate probabilities for the decks you'd encounter.

PuckTheCat: It is a problem, but it isn't at the same time.  Basically if you get a list of ALL the deck archetypes in a tournament and their position after swiss, you control (somewhat) for the popularity of the deck.

Take this example tournament:
Top 8: 4xTog 2xSlaver 1xGay/R 1xKeeper
9-16: 8xTog

Thus you have 12 datapoints with which to update Tog, and it'd probably end up with a win rate slightly <0.5.

All: Thanks very much for the compliments.  It's clear I need to do the application section so I can show exactly how to apply the methods.  I'll try to find time to do this.

Also, what I think would really improve the accuracy of the calculation is to have reports of the following form:
<deck archetype> win-loss-draw

The exact win-loss-draw provides much more information than just knowledge of top-8'ing or not.

-Luke

[theorigamist]

Is there a plan for a future article containing probabilities of drawing?  Because that really adds a lot of possibilities, and a crap load of more complicated math (for example, X-0-2 beats X-1-1, so you'd have to figure out the percentage of the time that the top 8 comes down to 2 ties versus a tie and a loss, etc.).

Also, I'm not a huge statistics guy, but shouldn't there be some kind of formulating a confidence interval here?  Technically every tournament is just a sample among the population of Magic: The Gathering Type 1 competitive players.  I would kind of cringe at a statistic saying for certain that a top 8 rate is .6 (or whatever) rather than .6 +/- .02 (or whatever).

It seems like this is a good time for TOs to start recording everything.  I mean, the Type 1 community runs more of its tournaments than Wizards does.  It doesn't seem like it would be that hard to get the word out for TOs to require players to submit decklists, win-loss-ties, which games who won the roll, etc.

That just made me think of something else, the Luke, what about drops?  Are you planning on including something about if X amount of people drop Y matches through the tournament given your current record of Z?

[the Luke]

About draws, I'd rather approximate them as I did in the article, because I want to keep using well studied binomial theorems rather than derive my own trinomial ones.  Also, I'd rather move away from "top 8 or not" statistics... I think that using Bayes theorem directly on the win-loss(-draw) ratio will provide more precise statistics.

On the confidence interval thing... ahhh, I was wondering when people would point that out!  I have been looking into the statistics more today, and what I've done isn't QUITE correct.  What I should do is have a probability density over coin biases, (sort of like a normal distribution that starts centered around r=0.5), then update that whole distribution as the evidence comes in.  This sort of formulation allows for standard 95% confidence intervals.  It is quite complex, but would much improve the estimation of r.  If I have time (which is an issue ) I could implement it.  For those who are keen, here's a link which shows what I mean, much better than I can describe:

http://www.math.uah.edu/psol/applets/BetaCoinExperiment.html