Godot, 4 Duress is not the way to go if using my stated sideboard plan. Duress is beaten by Brainstorm, can whiff, or help them by dropping a Mindslaver...And as far as combo goes, Red Blast is much better than Duress at giving you double-counter hands, since you can then leave mana up to resolve earlier draw spells (Brainstorm / Ancestral).
I didnt want to address your sideboard plan since it was for an old metagame, but thats part of the problem. I think your sideboard plan is flawed. Against slaver for instance you SB in 7 cards. 7!?! Tog is at its heart a sideboard deck and using this plan weakens your sideboard and the deck as a whole. Only running 2 Duress makes it necessary that you run some absurd number of REBs in the board in order to compete against common control matchups. If you're filling up your board with unnecessary REBs then you're weakening the deck. Thats at least 2 SB slots that could be devoted to something more useful, Old Men for instance. Furthermore, if you're bringing in that many cards you're undoubtedly taking out cards that are part of Togs basic gameplan (especially if, as you implied at the beginning of your post, you're leaving in 3 Wishes). And to even further belabor this point, REB doesnt hit Skeletal Scrying.
I'll say it again, 4 Duress is house. Tog does not play the beatdown role as much as it once did. By running 4 Duress you get (a) early disruption, (b) the ability to force through bombs, (c) you have a much stronger game 1 against combo and cs and (d) you dont have to warp your sideboard to take on certain matchups.
As for your other points:
(i) they Brainstorm: First off you have forced them to react to you which always good. Secondly, even if they hide something, they still had to draw 3 more cards so you'll likely still hit a draw spell or reduce their ability to protect said bomb.
(ii) you whiff: Ok, so I whiffed. Big deal, it means they're not holding anything so the bomb im probably attempting to resolve will. Yes, I spent 1 card and 1 mana for information, but Im probably winning at the point so its a minimal loss.
(iii) you hit a slaver: In the massive number of games I've played against the various forms of Slaver I have never once Duressed away a Slaver. And hell, if they havent resolved a Welder its not a weak play.
(iv) combo: This is just laughable. REB better than Duress? Please. Duress hits ALL their bombs. REB doesnt do shit against Wheel, Wishes, Desire, Skeletal Scrying, Necro, Bargain, Ritual, I think you get my point. But its better with double counter hands you say? Ok, so its only good when they are: (a) defending a spell with FoW or (b) attempting to resolve two bombs in one turn, one of which must be blue. But what is they protect with Duress and take your normal counter? Then what? THEN WHAT? Additionally if that REB had been a Duress you would have been able to deal with their bomb in the same way. Duress deals with not only their bombs but it hits mana sources too which can be even stronger. But again you say, if I run REB I can resolve my draw sooner! That doesnt matter. They are the beatdown. Your first priority is answering thier bombs. If you can do that you will have the time to draw cards. Given this point and that Duress answers their threats just a efficiently and more comprehensively, Duress is far an away the better card against combo.
And Library of Alexandria is almost an auto-include
sLoA is absolute garbage and I cut this immediately after SCG Chicago. Getting UU as soon as possible is waaaaaaaaaaay more important than drawing an extra card. Additionally if you're drawing cards off this after getting it down in the mid or late game you're winning anyway. In the early game it impedes your ability to get UU which costs you tempo and the card advantage from sLoA will not make up for that loss of tempo. We've tested a lot of control mirrors and found that the deck that gets a turn 1 sLoA often loses since it cant drain as soon. Nevermind the previously noted negergy with 4 Duress.
1 slot not decided could be mind twist/mystical tutor/ even skeletal scrying or the 3rd DA
I currently use Mystical but that slot is always up in the air. I dont particularly like Mystical, but in the early game it allows the deck to utilize Ancestral more(a typically under-utilized resource in this deck) and in the late game it either gets you Will or Time Walk. Mind Twist could be randomly good, but more draw is probably better. Overall, as much as I dont like it, Mystical functions both as draw and gets Will so I run it, but its always the first card I consider taking out.
Also, 3 REBs in your sideboard is more than sufficient. I'd also look at Vamp Tutor in the side too if you have the space.
EDIT: fixed so Im not talking like a valley girl