Testing the Unknown.

[Methuselahn]

Please feel free to answer any, all, or none of the following questions. tks!

The Unknown Variable. You know, that kid across from you at a SCG Chicago event.  You've never seen him before and so you haven't a clue as to what he's playing.  But he's playing Pitch Long.  You're playing Meandeck Gifts.   Now sure, you've practiced this matchup at home on the kitchen table with your test group, but perhaps you knew ahead of time that they were playing Pitch Long during all those practice games.  Maybe you would've Merchant Scrolled turn 1 for a different card if you didn't know for sure what your opponent was piloting.  Does repeated playtesting of a matchup skew the actual results when compared to the random encounters found in tournaments?  And, is it enough to care?

I'm curious because, after a fair bit of testing recently with teammate Rick Shay, I noticed that I lost less often when he didn't know what I was playing.  More so with decks than with card choices.  Perhaps his opening hand wasn't the best choice versus my deck, or maybe he was playing around cards that weren't there.  Regardless, the Unknown Variable sweeps in and steals a win that never happens after game 1.  This is  because of your deck being revealed and new information is available during your/their next opportunity to mulligan.

How do you simulate the random unknown of a tournament in testing? (Is this actually important enough early on to matter?  (often the late rounds of a SCG event are thoroughly scouted anyway.)).

How do you account for randomness in testing?

I'm interested in hearing any thoughts on all this or random pairings results and the comparison with playtesting results.

I like swords.

-Jeff

[never]

You and your testing partner could grab a different deck before each time you play a game, allowing for some level of randomness. Although, I guess you should also sleeve them all in the same color to eliminate subconscious advantages that you might gain from associating decks to certain color sleeves.

This obviously only gives a limited amount of randomness, depending on how many decks you have sleeved up, but it is a start.

[TheDB]

Testing randomly is possible when your test decks all had the same sleeves and your testing partner randomly selects the deck before you test.  This only helps with game one and getting a good feel of your deck.  Which is good to know, but shouldn't be your primary method of testing.  You should already know how to play your own deck.

The majority of your testing should be with post sideboarded decks.  You play more post-sideboarded games than you do pre-sideboard. Even though it seems obivous, I don't think people do this enough.  I still see people throwing cards in their board because they suspect it will be key in a certain matchup, but don't do the testing to back it.

Testing the random game one of a tournament isn't really that important.   It is something, with abundant time, you could tackle, but only after having mastered the games 2-3 of a match.

[etakspeelstae]

For onel, of course, this goes in reverse, as (at least I'm assuming) your opponent doesn't know what your playing.

Personally, I don't think repeated playtesting will skew your results. Now, your theory is correct; I was testing PitchLong vs MDG the other day, and my MDG-wielding friend Scroll'ed for a FoW turn 1, which could have easily been a Recall alot of other times. But I would say that there are things you need to watch out for...

You need to be able to know what your opponent is playing, and know it quickly, either first or second turn. If you know the correct play against a given deck, through playtesting, then the sooner you know your playing against that deck the better your match will be.

Again, personally, I don't think there is a great way to randomly test; or at least I don't use one if there is one. But playing against the same deck over and over, and looking for similarites between plays, or differences between a similar decks plays, you will be able to recognize the deck your playing against fairly quickly.

Or at least that's just my $0.02...

[Moxlotus]

We decide what we are doing before the test game.  "If I knew that you were playing X, I'd definitely keep this...but I 'don't' so I'll mull," has been said many times.  Or we will play like we know the opponent's deck, especially if we want to see how a particular card we just added to address deck X works out.

[EotGiftsGG?]

You're Playing Gifts. End Of Story.


What My oppenent is playing usually doesnt make a big difference, it sometimes however decides the tempo. If youre Playing gifts, hopefully youve made the right meta game calls with your meta-slots, and besides that, theres not much you can do. Win?

[etakspeelstae]

You're Playing Gifts. End Of Story.


What My oppenent is playing usually doesnt make a big difference, it sometimes however decides the tempo. If youre Playing gifts, hopefully youve made the right meta game calls with your meta-slots, and besides that, theres not much you can do. Win?

If you read my post... I think your logic is flawed, if only just because of turn 1 tutoring fx ;/

[GUnit]

You're Playing Gifts. End Of Story.


What My oppenent is playing usually doesnt make a big difference, it sometimes however decides the tempo. If youre Playing gifts, hopefully youve made the right meta game calls with your meta-slots, and besides that, theres not much you can do. Win?

If you read my post... I think your logic is flawed, if only just because of turn 1 tutoring fx ;/

Err, I think that altering the MT target would probably be classed as something that "sometimes however decides the tempo."

The answer, though, is that of course it's going to make a difference. Some hands are amazing against control, but too slow for combo. Some hands are very resillient in the face of stax/aggro control, some fall to pieces. Without information regarding the opponent's deck you are at a disadvantage, clearly.

[wethepeople]

You and your testing partner could grab a different deck before each time you play a game, allowing for some level of randomness. Although, I guess you should also sleeve them all in the same color to eliminate subconscious advantages that you might gain from associating decks to certain color sleeves.

This obviously only gives a limited amount of randomness, depending on how many decks you have sleeved up, but it is a start.

wierd, i thought i was the only one who did that.

i have four decks that i test friends with so they never know what i am playing, and it is always game 1, then we sideboard games 2/3:

EBA
SS-like deck (using Negators)
MDG (using like 24 proxies with uncommons such as Merchant Scroll because i dont feel like buying them and i never play the deck)
Birdshit (not my deck, but using same color sleeves, which is black.)

this method of testing does however get old because they begin to really know your deck, but that is what you are going for, right? many of my decks are fairly similar so its not the best, but it works for me because i play Meddling Mages and need to learn to have an idea of what my opponent is playing early as possible. There are a few other decks we test with on MWS if we dont feel like meeting up, but no one is any good at piloting those so it doesnt help so much.

[Dxfiler]

Knowing what you're up against is huge and will ALWAYS give you an advantage, if ever so slight.  I'm a huge advocate of knowing what I'm up against, a.k.a. scouting.  I'll actively scan the tables around me while IN A MATCH just to memorize faces with decks.  I don't need any lists, just archtypes.  This is a perfectly legal practice that not many people use.  A couple teams actively scout and exchange info with each other, but that's alot more time consuming and not worth it in general.  Like, I'm not advocating the whole pen/paper nonsense where you get card for card lists of the top tables.  That's frowned upon and for good reason.  It's invasive and just plain rude.

I'm merely advocating knowing what archtype someone is running, regardless if they are a known player or not.  I never look for a list, just the type of deck.  It gets easier as the rounds go by, and for any tourney under 100 people you should know what the majority of your possible opponents could be playing by rounds 3-4.  It sounds hard, but it's not.

When I'm in testing, I generally like to know what I'm up against ahead of time.  I do this because chances are I'd have this info when in a tourney, especially if in the later rounds.  I'm not on a team, but I still have never had trouble accumulating general deck knowledge of the majority of a room after those first few rounds go by.   This info is not always crucial, as type 1 decks are easy to figure out by the first two turns 90% of the time, but once in a blue moon it does matter.  Example: Day 2 against TK at waterbury... I lead out turn 1 meddling mage on dark confidant.  He had played SS the last 4 tourneys and mopped up.  I didn't bother to look at any of his matches earlier in the day when I had plenty of chances, because I just assumed he was SS.  This cost me huge, as he made his debut with Grim Long that tourney and completely threw me for a loop.  Once I knew he was long, I was able to play accordingly, but the meddling mage miscue was BIG and set me back when it wouldn't have normally. 

So yeah, I'm for testing the known, because the majority of the time you should be in the know... even if they're completely random.  Just do a little looking around the room, that's all there is to it.

- David Feinstein
(EVIL SCOUTAR)

[wethepeople]

Like I said, it is very important for a Fish player especially playing main deck Meddling Mages to know what exactly your opponent is playing early game because i tend to drop Mage within the first few turns, and i hate to have to guess something random.

Knowing my opponent's deck also causes me to change my first few turns. Sometimes i prefer to Duress right away or drop a turn 1 Confidant, this of course is depending on what my opponent is playing. I am yet to try the scouting thing but it seems like something i should do in a fairly small tournament (I doubt it would work nearly as well in a large event such as SCG.)

[Dominik]

Knowing my opponent's deck also causes me to change my first few turns. Sometimes i prefer to Duress right away or drop a turn 1 Confidant, this of course is depending on what my opponent is playing. I am yet to try the scouting thing but it seems like something i should do in a fairly small tournament (I doubt it would work nearly as well in a large event such as SCG.)

I played at a Beta Mox Pearl (2nd was $150, 3-4 was 50, rest of T8 was 20 each) tournament yesterday, going 4-0 in the first rounds then drawing the next two to to get into T8, and I have to say that anyone from the top eight that DID NOT scout was (or would have been) at a huge disadvantage. I know for sure that at least half of the top eight could recite the names names of the other 7 and what build/deck they were playing in a heartbeat (I know I could). When you gain so much of an advantage from just walking around after your games and watching others play (or even looking around while your opponent is thinking what to do for that turn, etc), it's really folly not to do so. I really recommend scouting, especially when you're in the later rounds.

[yespuhyren]

I really wanted to make it to that tourney but they called a mandatory meeting at work   God damn corporation bastards!