[Discussion] Playtesting Against Yourself

[Hyperion]

It is generally accepted/expected that when a decklist is put up for review on these forums or in an article, the poster has testing results to validate specific card choices. However, the nature of those testing results is usually subject to less scrutiny (if any).

People that are part of testing groups have the luxury and ability to playtest against other players who are presumably capable of piloting decks that are part of that team’s testing gauntlet and willing to invest time in dedicated testing. This is why many recent advancements have come from such groups—the additional resources facilitate innovation.

But where does this leave players that don’t have access to these resources? They are faced with the alternatives of

1) Scrounging around for games in places like #apprentice or #tmdgame. The difficulties with this approach are twofold: that you are playing against someone that has their own agenda for testing, and that quality control of your testing results is questionable because you may not be playing against competent players.

2) Testing by playing both decks themselves.

3) Not testing at all.

It is mainly point (2) that I would like to focus on in here, because its potential usefulness isn’t limited to players without testing groups—even players in groups can use this approach. The obvious advantages to testing two decks yourself are

1) You aren’t constrained to testing when other people are available—whenever you have free time, you can start testing and logging results.

2) You don’t have to worry about finding a competent player to pilot the deck you need to test against if you can play it yourself.

As a result, testing against yourself is potentially very time-effective. With that said, there are disadvantages as well, the main one being that it is less reflective of how real games play out—because in that case, you don’t know the contents of your opponent’s hand. Granted, part of the skill that comes with being a Magic player is being able to anticipate your opponent’s hand, but the point remains that it is nearly impossible to simulate a real game and thus the validity of solitary testing results may be suspect.

I know a few people that playtest against themselves and am interested in other players’ opinion of it.

In your opinion, is it a valid way to record testing results?

If so, is it more/less/equally valid as testing against other players?

If you employ this method of testing yourself, what suggestions can you pass on to other people that could use it themselves?

My impression is that this is a testing method that is not widely used because its conventions haven’t been firmly established yet (I myself have never attempted to do it). However, I think that insight gathered on the subject could perhaps change that. If players that previously didn’t test at all can start testing against themselves, for example, or if members of testing groups have the ability to test against themselves in addition to their teammates, I think that the potential for innovation in Type I could increase.

[MuzzonoAmi]

I actually test like this alot, since my teammates are pretty much blue-based control only, with Brendan (no TMD) being the only compotent person who can do something else. And since I can't always test with him, I often do at least perliminary testing against myself.

It's fine for the first 50 or so test games, when you're working the bugs out. But for true refinement, you need to test against another person.

[ctthespian]

The problem with playtesting against yourself is the fact that you know information you would not otherwise know.  No matter how unbaised you may be the fact is that you know what's in both hands, what's coming and that will most likely effect your decisions made during the game.

For base results, sure it's OK.  However, I think real playtesting requires a another person that has decent knowedge of the deck they are using to test against you.

-Keith

[jpmeyer]

Playing against yourself lets you assume optimal play.

[The Atog Lord]

Within limitations, I've found that testing alone can be quite useful. Most of my alone testing is done by goldfishing decks, most often combo decks. I find combo decks to be better suited to alone testing because they tend to be less dependent than other decks on what the opponent is doing.

First, doing so gives me a chance to see how the mana base of a deck is, how often it mulligans, and how it tends to reward aggressive play. Moreover, I've found that, when first learning to play a new combo deck, goldfishing is the best way to become familiar with how to play that deck optimally.

[Smmenen]

I think testing alone can be quite valuable - I call it "Two fisted testing."  But, I think the drawbacks in twofisted testing are equally apparent in all other forms of testing and are not limited to twofisted testing.  For this reason, I think done properly, it is a legitimate form of testing - although I very infrequently resort to it becuase I have so many good testing partners.

What are the potential drawbacks?

First, players might be more famiilar with one deck than another and play it suboptimally.

Second, players get involved in individual games and aren't able to see the big picture.

Both of those are true of regular testing.  But there are one or two more things I think two fisted testing has drawbacks in:

1) some decks rely on mystery: control decks with mana drain are more likely to have opponents hesitate and thus win more often.

2) when playing two decks, you might have a bias against one deck and that can affect your results.

But there are some benefits exclusive to two fisted testing:

1) you make optimal plays

and most important
2) you get the experience of seeing how other deck operate: the kinds of hands they get, the sort of game plan they have, and the kind of decision trees they make.  

One last thing - as a solution to the first drawback i mentioned from two-fisted testing - not being able to see the bigger picture, and about making play errors and bias, I recommend  notating your two fisted testing so that you can review them.  This will allow you to spot errors AND, perhaps more importantly, see trends.

I hope that helps.

Stephen Menendian

[MaxxMatt]

I think that the first step to do should always be the "Two Fisted Testing" that Smmemen referred to, but after this initial view through the mechanics of this "new" deck, I'm sure that anyone should try his new pile agaisnt other players to see HOW you are good at Bluffing, at Playing, at making different choice for unusual game-play  situations and so on.

To validate this part I usually avoid playing online or with casual players.
We have a really good "magic players circle" that function perfectly for testing because anyone of us is "specialized on playing" one or two good decks. this let me valuate if the new one could comepte or not. the role of a really good opponent is unvaluable IMHO.

The good part of the Single Testing is the deep insight that you can obtain of BOTH the decks mechanics
The bad part of the test is the inability, even if forced to do it,  to think in a really separate way after having Known the cards of Both the players.

The difficulties on testing in this situation is easily circumvented because I live with 2 other teammates..


my<2cents

[Grand Inquisitor]

The good part of the Single Testing is the deep insight that you can obtain of BOTH the decks mechanics
The bad part of the test is the inability, even if forced to do it, to think in a really separate way after having Known the cards of Both the players.

Although I can agree with almost all of the advice given on this subject, I think the above content really is the crux of this strategy.

In order to maximize the usefulness given those limitations, I often use two-fisted testing to isolate certain card interactions within a given matchup.

For example, if prison is a concern for my GAT list, I'll play against a perfect prison hand again and again, and see what cards in GAT allow it to pull out of it.

Another strategy is to test certain sideboard cards, where I'll start my target deck with a single sideboard card and six other random cards against the matchup the card is sideboarded for.

Lastly, when I just want to get a feel for the matchup, I try to hedge conservatively.  This phrase:

when playing two decks, you might have a bias against one deck and that can affect your results

certainly rings true.  In order to overcome this, as I'm usually playing control, I have my control deck decide on its hand first, and then allow the other deck to choose its hand and mulligan appropriately.

[Matt]

One way I've found to reduce that bias is that when you're testing in this way, since you're probably doing on a table or something, be sure to switch off which deck gets the position nearest you. The physically closer deck is usually the one you have a bias towards, because it echoes 'real' games - you have more personally invested in the deck immediately in front of you, so you want it to win more.

[Smmenen]

I think that the first step to do should always be the "Two Fisted Testing" that Smmemen referred to, but after this initial view through the mechanics of this "new" deck, I'm sure that anyone should try his new pile agaisnt other players to see HOW you are good at Bluffing, at Playing, at making different choice for unusual game-play  situations and so on.

To validate this part I usually avoid playing online or with casual players.
We have a really good "magic players circle" that function perfectly for testing because anyone of us is "specialized on playing" one or two good decks. this let me valuate if the new one could comepte or not. the role of a really good opponent is unvaluable IMHO.

The good part of the Single Testing is the deep insight that you can obtain of BOTH the decks mechanics
The bad part of the test is the inability, even if forced to do it,  to think in a really separate way after having Known the cards of Both the players.

The difficulties on testing in this situation is easily circumvented because I live with 2 other teammates..


my<2cents

I honestly only understood one sentence in your post.

Steve

[dandan]

I playtest this way and although it limits the 'bluffing' factor, it helps lesser players (like me!) see stuff like potential answers from the opponent, how often they actually have that answer, how bad that answer actually is in practice and if there is a way around that answer (if not, then at least how I can limit the damage of that answer). This is particularly true of Mana Drain, FoW and Wishes.
I think solo testing is fine to iron out the rough edges of a deck or to test particular things that would bore an opponent. The speed factor is important to allow you to get in more games, which gives you a feel of what the strengths and weaknesses of a deck are.
Personally I consider that fact that you are not testing against different playstyles more damning than the fact that you know the opponent's hand as serious playtesting helps you predict that anyway. Playing a Tog player who likes an early Tog is very different to playing a more control-orientated Tog Game.

[jpmeyer]

Playing decks in different styles is the single most important thing that people need to learn to do in testing.  It can drastically alter matchup win percentages without having to change a single card.

[Smmenen]

That's simply not true and you know it.  It is important but not the single most important thing.  

The best way to do this is to reconstruct game situations where you made a key decision and see how the game would have played out if you had made a different decision.  You can do this best in two fisted testing.

Steve

[jpmeyer]

No no no, it's the most important thing that people need to LEARN to do.  Most people don't think to do this.

[bebe]

The best way to do this is to reconstruct game situations where you made a key decision and see how the game would have played out if you had made a different decision. You can do this best in two fisted testing.

I do exactly that when testing. I find it the best way nto evaluate a deck's real potency. I often reconstruct a number of scenarios and plays. This can lead to exclusion or inclusion of different mixes of cards. It can help determine the proper count for certain spells as well. Rico Sauve mentioned this once before. It is painstacking and time consuming but if you are serious about improving an arch types performance it is worth the effort.

[Azhrei]

I can't test against myself and I'm astonished that so many people do. I find that knowing the opponent's hand and thought process doesn't make for optimal decisions, but rather forces situations where incorrect plays are rewarded.

Basically, knowing what's coming allows you to make decisions based on knowledge rather than strategy. Real knowledge in-game is number of cards in hand, cards in play, cards in graveyard, number of cards in library, life, and potential card pool. By adding the sure knowledge of what the opponent is trying to do, you reduce your ability to make correct inferences based on limited (real game) information. I think playing against yourself undermines your ability to make correct decisions based on incomplete information, and also can move you into situations where the normally good play might not be best (because you know more than you should) and can cause you to react incorrectly in real games.

Remember, the point of testing is not only to tune decks, but to have practical experiences that allow you to play on auto-pilot as much as possible without having to think. Self-testing weakens this, and I personally consider the potential to run on auto-pilot to be one of, if not the, most vital play skills.

[Smmenen]

I can't test against myself and I'm astonished that so many people do. I find that knowing the opponent's hand and thought process doesn't make for optimal decisions, but rather forces situations where incorrect plays are rewarded.

Basically, knowing what's coming allows you to make decisions based on knowledge rather than strategy. Real knowledge in-game is number of cards in hand, cards in play, cards in graveyard, number of cards in library, life, and potential card pool. By adding the sure knowledge of what the opponent is trying to do, you reduce your ability to make correct inferences based on limited (real game) information. I think playing against yourself undermines your ability to make correct decisions based on incomplete information, and also can move you into situations where the normally good play might not be best (because you know more than you should) and can cause you to react incorrectly in real games.

Remember, the point of testing is not only to tune decks, but to have practical experiences that allow you to play on auto-pilot as much as possible without having to think. Self-testing weakens this, and I personally consider the potential to run on auto-pilot to be one of, if not the, most vital play skills.

One thing to keep in mind about two-fisted testing (something I rarely do) is that it gives you insight into how the other deck plays out.  As a result when you sit down against it in a tight spot, you are far more likely to have an idea of what they have, when they are bluffing, etc.    Experience on the other side of the board is key.

It will also enable you to make correct decisions based upon complete konwledge that you will then realize is probably correct when you lack that knowledge for certain, but suspect that the specific scenario has arisen.

Steve

[Phantom Tape Worm]

I think playing with yourself is great.  I do it ALL THE TIME!!

However, someone should point out that it is no substitute for tournament experience.  It is one thing to play a deck with no pressure on you, when you are playing against yourself you can almost always make the optimal play if you are aware of all the deck's synergies.  But it is another thing entirely to play while your brain is locked up to the point that you can no longer do simple combat math.  Keep in mind, the only thing that matters is tournament play.  Familiarity, which is all you get when you are masturbating to cards, will only get you so far.

[Rico Suave]

Somebody who simply playtests by his or herself will be missing out on a lot of in-game experience, obviously.  You need to play with real people behind the other deck.  On the other hand, I think somebody who exclusively playtests with someone else behind the other deck is not going to be as good as somebody who playtests both ways, which is what Smmenen was going at.

Either way, playing by yourself to test whether you need 23 or 24 mana sources is worlds more time efficient than trying to do it with someone else there.  That's where I use this method most.

[Smmenen]

I fully agree with the following statement and hope it clears up remaining issues:

From: theorigamist
To: Smmenen
Posted: Sat Apr 17, 2004 6:37 pm
Subject: "Two-fisted testing"  
I was reading the thread in the Type 1 forum about testing against yourself, and since I can't post there, I was hoping you might be able to post this.

I agree with you about the value of seeing how both decks would play out against each other. From reading your posts, it seems like you are only about half a step away from saying the same thing I'm about to.

When you test this way, if you come to a decision that you know might be influenced by knowledge of both hands, then write down the situation and the decision you made. Then, as you test the same matchup more (because you really should be playing tons of games to make any statement about the matchup, anyway), you can see when you come to similar situations, and you can compare the different decisions that you make in the same situation. This way, even though your decisions may be based on extra information, you can see which decision is most likely to work.

For example, let's say you were to play a hundred games of a given matchup. In situation X, you decided to make play A 50% of the time, play B 35% of the time, play C 10% of the time, and play D 5% of the time. You can then say that in a match where you do not have the extra match, your best play is probably play A.

The overall effect of this is that, in any given situation, though you might have 5 different possible plays to make, it doesn't matter which one you chose in one particular test game. You will still see a trend of which play is usually best in that situation, and that leads to determining what cards a given deck is most likely to have, etc. Then, in a tournament or something, you are still making what is most often the right play, and the extra knowledge in your testing will not affect your tournament play (assuming you play enough test games).

To Azhrei I would say: the absolutely correct play is the one that comes from knowing not only the board position and the numbers you mention, but also from knowing the opponent's hand, and anything else they would know that you normally wouldn't. Think of Fact or Fiction, for example. A person who splits a pile not knowing your hand is likely to do something wrong. But we only call that a wrong split because we know the hand we have that makes is a bad split. In most possible situations, the split they made might be right. But in the particular situation in which you call it wrong, you do so because you know something they don't.

Thus, I would call the correct play the play made knowing everything both players would know in a game. The BEST play, however, would be the one that you make knowing:
A) all the information only you would know in a game; and
B) that it is the play most often correct in the given situation, so that you can almost assume what might be in the opponent's hand.

Notice that I differentiate between the correct play and the best play. The correct play takes everything into account, even that which you normally cannot take into account in a game. The best play is the one that is most often the correct play in a situation. Testing against yourself can show you what the correct play is, and recording your decisions will give you statistics to determine what the best play would be if you didn't have all the information.

Thank you, Smmenen, for reading and posting this,
-Theorigamist

_________________
ORIGAMIZED!

[Azhrei]

One thing to keep in mind about two-fisted testing (something I rarely do) is that it gives you insight into how the other deck plays out.  As a result when you sit down against it in a tight spot, you are far more likely to have an idea of what they have, when they are bluffing, etc.    Experience on the other side of the board is key.

It will also enable you to make correct decisions based upon complete konwledge that you will then realize is probably correct when you lack that knowledge for certain, but suspect that the specific scenario has arisen.

Steve

I think that's markedly inferior to simply picking up the other deck and playing it. If there's a deck you want to beat, play that deck against a real opponent until you've mastered it. That's how you learn weaknesses.

And FoF piles are easy to make. 19 times in 20 there is a clearly correct way to do them, even if half the time you might just lose anyway.

[dandan]

Another major factor is that sometimes no reasonable person would want to playtest a lot of games with some decks. Who wanted to playtest vs. Academy (RIP)? The number of games you can carry out solo is much higher and so this is a good method for sorting out consistency issues like mana, search, etc. It is undoubtedly terrible at gaining experience of how decks interact.

I think that most of us teach beginners by playing out a few games with both hands revealed. The beginner sees that there are things that an opponent can do to them and starts to grasp that good tactics is often to modify your plays to take that into account (sweeping generalisations I know, don't crucify me).

In the same way you can get a feel of how often an opponent has an answer for your threat or vica versa. My wife used to refuse to play vs. MonoBlue Control as she hated me 'countering everything'. A few games with open hands showed her that I couldn't counter everything unless she slowed down her game to try to get around my counters. 'Baiting' and 'bluffing' vs. permission are not easy unless you have an idea of high high a probability it is that they have a counter.

Solo testing can give you that feel, especially with less-interactive decks (combo and aggro), what sort of chance is there that they have a FoW, REB, Swords, etc?

The most primitive test I use is to shuffle, draw a hand of 7, decide if I would mulligan and then repeat. That is solo testing too. I do not confuse this with actual playtesting but I think that there are simpler, faster ways of doing basic honing of decks that playtesting.

I don't think anyone is disputing that playing experience and fine-tuning can only be done by two players, playing out multiple games. However, lesser players (like me) may need a few more steps of preparation before we get to the stage of using up someone else's time.

[Zherbus]

I think this has been said already, but the best aspect of this is to make sure you haven't over looked the obvious in the deck design stages. To see how an ideal game plays out is about as far as you can get with it, but the actual practice (playtesting) really comes from getting down and dirty with some friends.

[Azhrei]

Just to clarify, I am TOTALLY in favor of goldfishing. Actually, if you can't goldfish your deck, it's a good bet that it's bad and needs to be remade.

[rvs]

Just to clarify, I am TOTALLY in favor of goldfishing. Actually, if you can't goldfish your deck, it's a good bet that it's bad and needs to be remade.

Very very true. Goldfishing is really important to get a feel of the deck, and you can often spot if it works or not. I'm not a big fan of 2-fisted testing, simply because it will affect dicisions made. I'd also rather just test against one of my friends because it's more fun.