[Article] July Metagame Update

[Dr. Sylvan]

http://www.starcitygames.com/php/news/expandnews.php?Article=7922

[dandan]

Not much to say except that we appreciate these articles. Good stuff as usual.
Thanks

[Methuselahn]

Always informative.  I'm glad this sort of data is collected.  

I wonder if the set in stone 'Minimum of 50' shouldn't be changed.  Nine tournaments to reflect Vintage in one month seems to really underepresent.  There are alot of power tournies going on in the USA and only 1 made it into your stats for July.  So, I'm not so sure how helpful this data really is to me, except if I lived in Europe.

[LoA]

I've never been shy about creating more work for others, so here's an idea.  Your sample size might not be large enough, but do you think it's possible to break down the metagame based on geography?  I imagine this would require you to change the "rule of 50" but it would be interesting to track changes to Mid-Atlantic, New England, Rhineland metas, etc.  As a side bonus, it might be easier to illustrate how/why certain card choices are made in boot-shaped countries vs. card choices in towns still suffering from corruption scandals.

[Gandalf_The_White_1]

Great stuff.  I have alot of respect for organised data after getting 85% in my gr 12 data managment course .  IMO, the work is time-consuming and boring(although of course greatly aided by compoter sofware), but the conlusions that can be drawn can lead to invaluable developments.  In short, someone has to do it, and I'm glad it's not me.  (Although due to this I will probably soon forget most of what I learned.)

The reason why the 50 minimum standard is important is to prevent the skewing of the data by uncompetitive decks, ie decks that cannot suceed in a competitive environment, and the distinct line has to be draw somewhere.  If we want more data to be collected from here, we should work to have a greater number of large tournements!  That being said, it would be nice to get a picture of the meta as it stands in north america seperate from europian stats.

[Dr. Sylvan]

Oh wow, thanks for reminding me to get back to LoA's and methuselahn's questions, Gandalf.

LoA: Breaking it down regionally is impractical at this point. In all too many months, I only have one or two North American Top 8s. In order to get, say, five for each region (which is my secret "if I don't get this many, I'm putting off the report until I do" lower limit), then I'd have to scour the Tourney Forum for some pretty small events to make even bi-monthly reports. The Italians could generate five tournaments every six weeks without dipping into small events, but to do "Midwestern US/Canada" would mean taking a couple of big events and several 30-50 player ones every couple of months at best. Because of regional variation in regularity of tournaments, I wouldn't be able to assemble consistent numbers that could be compared against each other.

This leads to another big reason why 50 is an important limit: reporting. For instance, right now I haven't heard of decklists from the $250 GenCon event last Friday night going online. I suspect the Friday daytime results would have been similarly elusive without my personally talking to the TO and getting the lists before going to the hotel that evening. I would have tried to pick up the other lists, but my free credit with the guy was running pretty low after getting those daytime lists, and I didn't want to push my luck.

The same thing happens all over the place, I'm sure, to tournaments in the 30-50 range. Some don't even do deck reg, many don't have local data apostles who would constantly make sure to get the lists online if the TO himself doesn't do it. It makes the data less valid the more reporting "errors" there are, and to me, an unreported Top 8 is an error that skews the numbers. July already had a solid example: Browser's Turboland Lotus win at a 56-player event went unincluded because there was no deck registration. If the problem is still there for Lotus tournies, it gets worse the further down the ladder you go. So these reporting errors would make the regional breakdowns very biased toward whatever random Mox tourney locations had people willing to get decklists online. I think getting those lists online would be great, but without a more complete picture reporting on them would be much less attractive to me.

Another point about the 30-50 range is that it is more vulnerable to what I think of as the Magic equivalent of insurance policies' "Acts of God" clauses. Someone doesn't set their alarm clock, or their car explodes in a ball of fire during the night, or their Power gets swiped in lieu of lunch money at their school...stuff like that. At an event like GenCon, the tournaments would still be competitive if, say, the Canadian/Dutchie War Wagon hadn't made it, but no one can argue that it would have been as competitive without them. If one car makes a noticeable difference at the highest concentration of hardcore TMD Vintage players in the last twelve months (possibly excepting a Waterbury or two), then think how that can impact a much smaller event. Another example is the infamous Dulmen WW appearance when the tournament was scheduled on the same day as Regionals. What if instead of worrying about people going to Regionals, the tournament's results could become nonindicative because three guys decided to go to FNM and not wake up the next morning for Type One? That's the kind of random, results-distorting anomaly which is greatly lessened for events where no more than 10% of the players can fit in one sedan. The smaller an event, the less reliable its player pool showing up is, and the more we have to worry about freak nonindicative decks making T8.

That said, Zherbus has been talking to me about midrange events, so there might be something to fill the demand for smaller-event analysis soon.

[Gandalf_The_White_1]

That is of course the other main reason for the 50+ threshold; the fact that the lower sample size of the tournements make it easy for results to be skewed by outliers or random happenstance.

Even if data was collected about 30-50 tournements, there is a good chance that the data would provide very little reliable information about the particular meta anyways, as -1 particular sedan +1 difference sedan could result in an entirely different meta at the next tourney even if no new developments have occured.

The fact that this can be combined with poor judging and/or mistakes in decklists, etc (not to detract from the quality of any particular event), makes data even more unreliable.  

Perhaps a decision could be made based on standard deviation of certain numbers within a selected sample of smaller events. (which could give us an idea of how reliable figures collected from these events are)

[Matt]

Since there's so few tournaments to look at, what's the real problem with abolishing your inclusion requirements and just looking at each tournament on a case-by-case basis? That would eliminate the act of God situations and so on, letting you include high-caliber small tournaments and disinclude large scrubby ones. You're not writing a pogram for a computer to collect data, you don't have to spell out your guidelines ahead of time.

[Zherbus]

I have July done for 30-49 range tournaments... the problem is that I had the idea so late in the game, that I'm just going to bundle August with it and pick it up in September as a monthly thing.

[Gandalf_The_White_1]

Since there's so few tournaments to look at, what's the real problem with abolishing your inclusion requirements and just looking at each tournament on a case-by-case basis? That would eliminate the act of God situations and so on, letting you include high-caliber small tournaments and disinclude large scrubby ones. You're not writing a pogram for a computer to collect data, you don't have to spell out your guidelines ahead of time.

Because that would lead to a horrible bias in the data.  Clear catagoric requirements are necessary for any type of statistical analysis, and they must be upheld; otherwise it means nothing.  Whoever is gathering the data can't just look at it and take what they like, leaving the rest; that's insane.

Edit: Props to Zherbus for taking the time to do the slightly smaller tournies.  Not only does this give us an idea of how things look in the north-east, etc, but it can be compared with Phil's data to analyse the difference between smaller and larger tournies.

[Matt]

Because that would lead to a horrible bias in the data.  Clear catagoric requirements are necessary for any type of statistical analysis, and they must be upheld; otherwise it means nothing.  Whoever is gathering the data can't just look at it and take what they like, leaving the rest; that's insane.

Statistical analysis is already weak with a sample size of less than ten. The gains you'd get from the larger sample size more than offset any (theoretical!) bias.

And if you want insane, think about the requirements that would leave out a meandeck/shortbus face-off tournament but include a 50-player tournament from Nowhere, Bulgaria. Relying on rules instead of using the brain you've got - THAT is insane.

[Zherbus]

Edit: Props to Zherbus for taking the time to do the slightly smaller tournies. Not only does this give us an idea of how things look in the north-east, etc, but it can be compared with Phil's data to analyse the difference between smaller and larger tournies.

Well, I really wanted DrSylvan to do this, personally. The fact is, on a whim I started crunching numbers and lists and it's a SHIT load of work. It would almost be unfair to expect DrSylvan to double his work, so I'm just running the numbers myself. I'm also not claiming anything close to originality since I'm stealing the basic template from him of what sort of lists he presents. I'll try to make up for my severe unoriginality by adding as much commentary as possible.