"Wildcard" Testing Method

[Delha]

Started a new thread to stop derailing the other one.

What do you do when you find yourself tinkering with an already good deck.

Evenpence suggested including a blank card in the deck, the idea of which I loved. Whenever it comes up, decide which of the cards you are considering for the slot would be better for you. From there, it's just a matter of keeping a running tally. This obv only works for singleton changes, anything bigger than that will be more hazy.

Other than testing til your eyes bleed, the best answer I can think of is to always ask yourself why whatever card you're considering (be it for adding or cutting) belongs in the deck, and if there is another card that fills that role more effectively.

The problem with single card changes is that they don't usually hurt, but can be enormously helpful.  Ie.  I netdeck Elephant Oath and add a Sorrow's Path.  How much lower is my match win %?  It's "just" a dead card.  By contrast, consider putting Balance in that same slot (ie. You're constrained to have a 61 card maindeck).  Despite the prevailing wisdom, your match win % will probably go up considerably since tutoring for Balance isn't what one would call a bad play.

Now, the problem is the diminishing returns.  For every card you add beyond 60, your odds of seeing either it or the tutor to find it diminish.  Consider trying to build a 100 card version of storm combo that gets 4 Contract from Below.  Is this better than the best netdecks?  I seriously doubt it.

The problem with single card changes is that they don't usually hurt, but can be enormously helpful.  Ie.  I netdeck Elephant Oath and add a Sorrow's Path.  How much lower is my match win %?  It's "just" a dead card.  By contrast, consider putting Balance in that same slot (ie. You're constrained to have a 61 card maindeck).  Despite the prevailing wisdom, your match win % will probably go up considerably since tutoring for Balance isn't what one would call a bad play.

Now, the problem is the diminishing returns.  For every card you add beyond 60, your odds of seeing either it or the tutor to find it diminish.  Consider trying to build a 100 card version of storm combo that gets 4 Contract from Below.  Is this better than the best netdecks?  I seriously doubt it.

That's not how this testing scenario works though. This doesn't tell you should add or remove card X. If someone decides to push their decklist to 100 cards, they've got bigger problems than tuning. This method is just to tell you how useful X is relative to cards Y and Z, without having to test your list separately with each.

To fix your example, I'll arbitrarily pick the slot of Brainstorm to replace with a blank. I assume we can both agree that in the majority of times you draw into the blank, Brainstorm will serve you better than anything else, and thereby defends it's inclusion against other cards. If you chose two cards with very similar utility, then the method tells you that as well.

I disagree.  The marginal utility of "anything I want" over even a "super-cycling" slot is ridiculous.  When I want that slot to be a narrow answer, it's disproportionately rewarding.

As a false dilemma, consider the slot to be only Balance or Brainstorm in emidln's UBw Ad Nauseam Tendrils deck.  To save you the trouble of looking it up, it looks exactly like you'd expect it to.  The white splash deals with Fish and Stax out of the board, including the SB Balance.  When you want the slot to be Balance, it'll win you the game.  When you want it to be Brainstorm, it may be more frequent but may not necessarily contribute to a game win.  I also lose the emotional cost of losing the game to a dead Balance in hand when I'm testing this way.

Now as the tester trying to decide between the two (even though obviously they're almost never in conflict), which do you choose?  Well, obviously Brainstorm.  Does that work with your method?  I don't think so.

Another way of looking at it...you're playing Tez.  The vast majority of the time you draw Mox Pearl *will not* be in your opening hand.  If the blank can be either Mox Pearl or See Beyond, which one would a tester using your method choose?

I had a bunch of point by point counterarguments written up, but decided to simplify to the below instead.

Do you believe that deckbuilding should always select for the card which is most frequently desired?

If you do not, then our disagreement is over a more fundamental principle than selection method, and we can skip the debate over examples to focus on the core of the dispute.

[zeus-online]

Generally speaking i'd go for the card i'd most often want to see. There is a reason that balance, regrowth and timetwister does not see a ton of play.
While they're all awesome in the right situation, it's a question of how often such a situation occurs.

There are some cards that i will gladly play eventhough i usually do not want to see them, such as Echoing truth, chain of vapour, hurkyl's recall, rebuild, tinker-bots etc.

I think the wildcard testing method can be usefull if you can narrow it down to a few similar cards (like, do i want hurkyl's or rebuild...Thoughtseize or duress etc.)

[AmbivalentDuck]

I think the wildcard testing method can be usefull if you can narrow it down to a few similar cards (like, do i want hurkyl's or rebuild...Thoughtseize or duress etc.)

Agreed.  Here there's no conflict between "exactly what I want" vs a more generally good card.

Do you believe that deckbuilding should always select for the card which is most frequently desired?

No.  
1) You never specifically want to draw an off-color Mox (as opposed to Crypt, Mana Vault, a fetchland/dual land).
1a) You never want Ponder either.
2) You never want to draw/Oath your Tinker critter, but it still obviously belongs in your deck.
3) You always want Force of Will in hand, but that doesn't mean you'd run 10 copies if you could.
4) The preponderance of tutoring in our format makes it "right" to run cards you need to be able to tutor for (Hurkyl's Recall) even when in many matchups, you don't particularly want to draw them.

[Delha]

Do you believe that deckbuilding should always select for the card which is most frequently desired?

No. 
1) You never specifically want to draw an off-color Mox (as opposed to Crypt, Mana Vault, a fetchland/dual land).
1a) You never want Ponder either.
2) You never want to draw/Oath your Tinker critter, but it still obviously belongs in your deck.
3) You always want Force of Will in hand, but that doesn't mean you'd run 10 copies if you could.
4) The preponderance of tutoring in our format makes it "right" to run cards you need to be able to tutor for (Hurkyl's Recall) even when in many matchups, you don't particularly want to draw them.

I don't think you understand how this method works. It doesn't only get decided when drawing into it. The first time the WILDCARD becomes relevant in any way, you decide what it's going to be (and presumably stick with that for the rest of the game). If it's in your opener, you decide immediately what it is (before play starts, even if you're on the draw). As another example, if someone is capping you, you decide immediately what card it is.

1. When your opener has Orchard + Oath + WILDCARD, you look at it and decide what you want it to be. Using your example, your options are Pearl and See Beyond. In this case, you obviously want Pearl (ignoring the other cards in hand).

2. You are correct, but it will still select properly. Say you were testing See Beyond in the Terastodon slot. When you Oath, and flip WILDCARD, you decide whether it would be better for it to be the elephant (and thus hit play) or to be See Beyond (and be passed by).

3. This is a moot point. You can't run 10xFoW, so you never test 6-10, with this method or any other.

4. If you find that 80% of the times you see the WILDCARD, Hurkyl's is better than See Beyond, then it deserves the slot. If your meta is shop heavy, and you're testing against decks in the corresponding proportion, then Hurkyl's actually IS the right choice. If Hurkyl's worthless 90% of the time in your meta, this method will show you that as well. By definition, you will choose See Beyond 90% of the times it's drawn, and you will presumably never tutor it up (since there will pretty much always be better targets).

For all intents and purposes, the moment the card becomes visible to either player (be it through draw, Oath, tutoring, capping, Confidant, or something else), you decide what it is.

[nataz]

Generally speaking i'd go for the card i'd most often want to see. There is a reason that balance, regrowth and timetwister does not see a ton of play.
While they're all awesome in the right situation, it's a question of how often such a situation occurs.

honestly balance and regrowth are just in the wrong colors to see lots of play

[zeus-online]

Generally speaking i'd go for the card i'd most often want to see. There is a reason that balance, regrowth and timetwister does not see a ton of play.
While they're all awesome in the right situation, it's a question of how often such a situation occurs.

honestly balance and regrowth are just in the wrong colors to see lots of play

I've splashed for them both before (not in the same deck) I used regrowth in 4 thirst Tez for example...Had 2 tundra and a balance in my gifts sideboard - They're not really that hard to splash for, but they are situational.

I'm wondering - Does this mean that if they where both blue, you would auto-include them? I'd definetly rate them a bit higher but not exactly auto-include them.

[AmbivalentDuck]

1. When your opener has Orchard + Oath + WILDCARD, you look at it and decide what you want it to be. Using your example, your options are Pearl and See Beyond. In this case, you obviously want Pearl (ignoring the other cards in hand).

That's only true of your opening hand.  Given that example, you might even pick Chrome Mox. (So I can cast my Ancestral first to draw out any Force of Wills!)  That and it's a wonderful example of a card that's decent in your opening hand and terrible to topdeck later.

2. You are correct, but it will still select properly. Say you were testing See Beyond in the Terastodon slot. When you Oath, and flip WILDCARD, you decide whether it would be better for it to be the elephant (and thus hit play) or to be See Beyond (and be passed by).

Really?  I think that's only true against decks where I wouldn't rather see one of my other critters.  We still run Iona for a reason.  If Terastodon was the best of the bunch, we wouldn't run Iona at all.

3. This is a moot point. You can't run 10xFoW, so you never test 6-10, with this method or any other.

Hardly.  I always want a Crucible in play when I'm playing Stax, but that doesn't mean that 4 Crucibles is always the right choice.  Even if you could run 10 Force of Will, the right number is probably between 5 and 7 in most decks.  It's an example of a card that's strictly better when you don't have to do it repeatedly.  Having to use multiple FoWs in a game is the same card advantage loss as a redundant Crucible in hand.

4. If you find that 80% of the times you see the WILDCARD, Hurkyl's is better than See Beyond, then it deserves the slot. If your meta is shop heavy, and you're testing against decks in the corresponding proportion, then Hurkyl's actually IS the right choice. If Hurkyl's worthless 90% of the time in your meta, this method will show you that as well. By definition, you will choose See Beyond 90% of the times it's drawn, and you will presumably never tutor it up (since there will pretty much always be better targets).

Whoa... Wouldn't it be 51%?  Anyways, the "deeper" point here is that the odds of seeing a card increase in the matchups where it's useful.  We have tutors and selective card drawing...I hear they're almost all restricted.  THIS is what breaks your testing method.  Would I rather have See Beyond in hand or be able to tutor for Hurkyl's Recall when I need it even if it's dead now?

[Delha]

1. You're blaming player incompetence on testing method. Let's use your example and say a complete scrub is testing with Pearl and Chrome as the two options. Over time, that person will learn what nearly everyone else already does: That the -CA is far too crippling to make it worth running over Pearl. If the tester is unable to realize this, then their evaluation skills are what is skewing the data. Such a player will not be any better off testing through traditional methods (playing many games with Pearl, then many games with Chrome instead).

2. You're still thinking too narrowly. If you are testing only against decks where Tersatodon is the best, then you're testing wrong. Alternately, if your meta actually does consists only of those decks (or if you're testing a sideboarded Terastodon against it's intended match), the method gave you the correct result. This is what I was getting at my response to #4. So long as your testing gauntlet properly reflects your meta, then all of the weighting is borne out naturally.

3. If you run 3xCrucible and your Wildcard options are Crucible + Card B, then every time you rip the WILDCARD with a Crucible already down, you will pick Card B. Since this will happen three times as often as seeing WILDCARD when a real Crucible comes up, testing will properly indicate that the fourth Crucible is undesirable. My statement about FoW holds. You should never be testing to see if you want a fifth FoW, because you will never be allowed to run it. By definition, you are testing the outcome of an event that cannot happen.

4. Yes 51% is technically correct. It's also not realistically attainable, and it doesn't even matter that it's not. The method will correctly tell you that the two cards have a very similar desirability to you.

Also, I don't think you understand how odds work. If you draw 10 cards over the course of a match, those cards are completely random, whether you are playing against Ichorid or TPS. If you tutor during that timeframe, you will see every card left in your library, whether facing down Fish or Oath. You may or may not CHOOSE a given card, but that doesn't mean you didn't SEE it.

[AmbivalentDuck]

1. You're blaming player incompetence on testing method. Let's use your example and say a complete scrub is testing with Pearl and Chrome as the two options. Over time, that person will learn what nearly everyone else already does: That the -CA is far too crippling to make it worth running over Pearl. If the tester is unable to realize this, then their evaluation skills are what is skewing the data. Such a player will not be any better off testing through traditional methods (playing many games with Pearl, then many games with Chrome instead).

You're misconstruing what I said.  Most of the time you "see" a card, it is not in your opening hand.  If we go with your example, yes Pearl is better than See Beyond.  That said, who EVER tutors for an off-color Mox? Since you seem to be using frequency as the deciding factor, tutoring vastly skews your results.  You'll never tutor for Pearl, but you might tutor for See Beyond.

2. You're still thinking too narrowly. If you are testing only against decks where Tersatodon is the best, then you're testing wrong. Alternately, if your meta actually does consists only of those decks (or if you're testing a sideboarded Terastodon against it's intended match), the method gave you the correct result. This is what I was getting at my response to #4. So long as your testing gauntlet properly reflects your meta, then all of the weighting is borne out naturally.

Strictly untrue.  Hurkyl's Recall is a strong example of value trumping frequency.  You don't want it every game, even against Stax.  But when you want it, it wins you the game.

My statement about FoW holds. You should never be testing to see if you want a fifth FoW, because you will never be allowed to run it. By definition, you are testing the outcome of an event that cannot happen.

It's an example of your strategy degenerating.  Nothing more, nothing less.

Also, I don't think you understand how odds work. If you draw 10 cards over the course of a match, those cards are completely random, whether you are playing against Ichorid or TPS. If you tutor during that timeframe, you will see every card left in your library, whether facing down Fish or Oath. You may or may not CHOOSE a given card, but that doesn't mean you didn't SEE it.

I'm a PhD student in computational neuroscience and I've hurt my head plenty with Bayesian stats.  The relevance of a card when tutoring is strongly dependent on whether or not you choose it.  Does having a Mox Pearl in its maindeck hurt Tez's tutoring?  Not much.  That's why its shittiness outside of opening hands is irrelevant: between tutoring, top, and other selection engines you're much less likely to have it wind up in hand when not drawing/mulliganing your opening hand.

[Delha]

...That said, who EVER tutors for an off-color Mox? Since you seem to be using frequency as the deciding factor, tutoring vastly skews your results.  You'll never tutor for Pearl, but you might tutor for See Beyond.

Strictly untrue.  Hurkyl's Recall is a strong example of value trumping frequency.  You don't want it every game, even against Stax.  But when you want it, it wins you the game.

Yes, tutoring weights your results in favor of the tutor target. Why shouldn't it? Will is a very frequent target of tutoring because it is very good at winning the game for you. Value begets desire.

This connection makes the two terms practically interchangable for this testing method. For a given boardstate, let's say you draw the wildcard. Which option do you want more? The one that is of greatest value to you, obviously.

To address Hurkyl's in specific. On the play against Stax, are you telling me that you would be unhappy to see 2xHurkyl's? Sure, you're opponent doesn't have any lock pieces down yet, but you know that they will. Whatever strategy you have planned, I bet it will be pretty easy to execute after resetting their board twice. High value cards have high desirability, just from anticipation of their usage.

My statement about FoW holds. You should never be testing to see if you want a fifth FoW, because you will never be allowed to run it. By definition, you are testing the outcome of an event that cannot happen.

It's an example of your strategy degenerating.  Nothing more, nothing less.

If you can give me a good enough reason why trying out 5xFoW with ANY testing method is relevant to deckbuilding, I will happily concede your point. For the time, I am content not bothering to test illegal decklists.

The relevance of a card when tutoring is strongly dependent on whether or not you choose it.  Does having a Mox Pearl in its maindeck hurt Tez's tutoring?  Not much.  That's why its shittiness outside of opening hands is irrelevant: between tutoring, top, and other selection engines you're much less likely to have it wind up in hand when not drawing/mulliganing your opening hand.

Yes, the relevance of a card is strongly dependent on whether you choose it. That's why the method selects against Pearl whenever you tutor/Top into it and decide you'd rather it were See Beyond. If you went for Oath or Vault or Tinker instead, then it doesn't matter whether it was Pearl or See Beyond.

At the end of the day, here is my stance in a nutshell:
For every single card in a completed deck, the player should be able to say "All things considered, I believe that replacing this card with any other will lower my chances of winning". If that player can think of some other card that would increase the odds of victory, why is that other card not being run instead? As before, this criteria should be applied with all things considered, up to and including player specific issues such as decision complexity. That still doesn't change the fact that by definition, an optimized list (for a given player/event/etc) will have zero cards which can be replaced by something better.