Playtesting 5E

[RobbyPants]

But the long and the short of it is:

• If the designer suggests setting the DC after the players roll the dice, you should not buy rules from that designer.
• If the designer tells you that they intend to hotfix the entire RNG and all the mathematical inputs of the system at the end, you should not buy any rules from that designer.
• If the designer made more than two dozen officially released overhauls to the Skill Challenges rule and never figured out how to incentivize players to contribute instead of not doing so, you should not buy any rules from that designer.
• If the designer has repeatedly put his name on for-profit products that were subsequently admitted to be unfinished and unworkable as sold, you should not buy any rules from that designer.
• If the designer has Bruce Cordell on his staff and no one is complaining about that because everyone knows that he's even worse, you should not buy any rules from that designer.

-Username17

While this is all totally true, it reminds me of that bit that Foxworthy does at the end of his shows.

TGD will have to make a "You might be Mike Mearls" thread.

[Previn]

[*] If the designer has repeatedly put his name on for-profit products that were subsequently admitted to be unfinished and unworkable as sold, you should not buy any rules from that designer.

Huh, are there actually such things plainly state by Mearls out there?

[Username17]

[*] If the designer has repeatedly put his name on for-profit products that were subsequently admitted to be unfinished and unworkable as sold, you should not buy any rules from that designer.

Huh, are there actually such things plainly state by Mearls out there?

I have a pretty simple question, and I'm actually serious about this. This is not asked in snark.

If you think skill challenges have failed, why is it a bad thing that we're tinkering with them? You basically build a case for the rules' shortcomings, but then seem unhappy that we're working to fix them.

I think that R&D now has the attitude that if something doesn't seem to work correctly, we're going to fix it now, rather than hope that nobody notices. Because the truth is that people do notice, and it doesn't help anyone to push aside problems people are having at their game tables.

That doesn't mean we'll errata the hell out of whatever happens to be the topic du jour. We're more interested in gradual experimentation that leads to a final fix.

Note: after over two dozen Skill Challenge overhauls by Mike Mearls personally, they were officially abandoned with little fanfare in 2011. No "final fix" ever came.

-Username17

[virgil]

Mearls has publicly stated that Iron Heroes was incomplete, literally churning out the magic chapter in less than a week and leaving the whole thing without editing, blaming the contract with WotC as his reason for rushing it out. This was his first real product in the industry, so we didn't know it would become a running theme. The actual article is lost to the internet since Monte Cook's website has updated since then and I can't be buggered to work that hard to find it again.

[shadzar]

Well, in theory, you should make new editions to fix the known complaints and problems about the previous edition. If you are competent and have enough resources, you do it in 2E => 3.0 way, where the team not only gathered a lot of info on what pissed people off in 2E and fixed that, but actually improved and streamlined the core engine.

no they didnt. they took the garbage that was 2.5 and developed it further. they never once attempted to work on 2nd after Steve Winter took over and got 2.5 underway. hell listen to what he said in the podcast and had the stupid designers like Winter been totally in charge of 2nd edition, it would have been 4E that that produced, vastly different and mass changes.

Thankfully "Zeb" strove to keep the game so similar that it didn't kill off its playerbase. The correct edition change was from 1st to 2nd. remove the overwordy Gygaxian prose, organize the books where you can actually find shit, and even finish some sentences where it could be figured out what was being said.

3.x -> Pathfinder is what a new edition should be like also. If it wasn't then why do so many damn people play Pathfinder?

3.0 is the worse case scenario, a complete revamp with nobody on the outside, those gamers that play, having any say. All it had was a giant advertising engine behind it, and less bible-thumpers attacking it because how big gaming was at the time. 3.o was just lucky.

It's a pretty small field, they can't really help but hire the people they know.

We all know that is a lie. The ones hired are just the ones "qualified" as per say HASBRO guidelines, and those that will kiss HASBRO ass and hand over all work to be owned by HASBRO, ergo why Monte Cook quit a second time.

The people capable of making games outnumbers those employed to do so solely for financial reason. Pathfinder proves this, OSRIC proves this, even OD&D proves this. The "professional" designer field is purposefully kept small so that the "professionals" can be controlled by the corporations, not unlike banks and such other businesses with a CEO that knows nothing of the job needing to be done, but just a super bean-counter and shareholder ass-kisser.... as well as taking half the profits from the company as "bonuses" rather than putting the money back into the company to make more or better products to make MORE profits.

i will agree 3.0 was made like OD&D and somewhat 1st...it was made to be played by those who made it, and THAT gave it strength. a game made by gamers FOR gamers, not for fattening wallets. when someone is making the game for money like HASBRO and ALL its employees do, then you will end up with garbage and gimmicks made for short term in order to suck money in and then toss it out for the next thing. (see 4th, see 4E, see D&DN, see 2.5, see PHBR/DMGR)


@Mike Mearls....with Bill S now gone Mike has no one else to blame for his shortcomings, and will be the next one gone. His failures in design will be seen, and his inability to do the work IS being seen. How many times has the goals of 5e changed already? in the end 5e is going to end up being 4.75. Mearls may make a good video game designer, but he needs to leave the video game design ideas out of TTRPG. HE doesn't understand the openness it provides people to play how they want like any edition of D&D that came before him, and in his "playing" of older editions, he has shown he NEVER understood them, and is NOT willing to learn them. HE can't "capture the feel" of the earlier editions, if he never experienced them.

I am reminded of some idiot on ENWorld, that claims X never happens in D&D games or venues because he never saw it, while he keeps himself from even going to places where those things happen, and makes the claim is doesn't happen cause he doesn't see it. A closed mind secluding itself from things purposefully and only staying in the same small circles of activity can not begin to know as they haven't experienced it. This too is true for Mearls. Mearls should have sat as a player in a bunch of different games from different editions BEFORE he ever put pen to paper to write anything for D&D and see how people REALLY play the game, and WHAT the feel of it is, rather than walking with blinders on or rose-tinted glasses, and not being able to comprehend the problems people are having because he can't visualize them as he has no imagination, and no experience. Mike Mearls in 3.x D&D terms... Level 0 Commoner. He IS the fucking dirt farmer!

[Voss]

The sheer, staggering amount of irony caused my heart to stutter.

Sometimes, people throw you gems, and even when you're eye is wandering down the wall of gibberish in disinterest, fully intent on not reading it, things can jump out at you.

[theye1]

I've heard that they've made fighters even more useless and irrelevant than before.

edit: The fighter is in good shape. We're likely going to give the fighter a special parry mechanic that doesn't use expertise dice but works much the same way. If anything, the fighter might be a little too good. The feedback pegs the fighter as the most powerful class. The other classes will mostly catch up, but we might reduce expertise dice a little bit at higher levels to keep the game moving quickly.
Link

That would be funny, if it wasn't so sad.

[Voss]

Actually, in the previous iteration the fighter actually was decent, with real combat options, damage mitigation and level-appropriate damage scaling. That only got trashed (literally, the damage scaling went away, the options got turned into feats, and expertise became a very badly done encounter power mechanic rather than a resource you could use every turn) for the current version. It is a step backward in every conceivable respect. They actually had a decent chassis for a viable class for one shining moment, but rather than fine tune the damage aspect, they tossed it out, and rewrote everything from scratch. The result is worse in every way, apparently at least partially because it was too complicated to have a dice pool refresh every turn.

Either that or they decided it wasn't right for the mundane classes to be enjoyable, let alone playable. But the spellcasters still kind of suck (they were toned down a lot from the initial playtest, especially clerics- divine shield doesn't grant disadvantage to attacks, just +1 to AC, for example, and most of the 'buffs' are in that bonus range now), so I'm confused on that point.

[Krusk]

They actually had a decent chassis for a viable class for one shining moment, but rather than fine tune the damage aspect, they tossed it out, and rewrote everything from scratch. The result is worse in every way, apparently at least partially because it was too complicated to have a dice pool refresh every turn.

I don't remember it ever being able to do anything out of the melee fighting schtick. Am I forgetting something or was it more of a "If we are doing the melee fighting thing you can hold your own"? Meaning its still irrelevant once something flies, or the group wants to do anything other than hit it with a stick.

[wotmaniac]

I happened to have run across THIS little piece of commentary -- just thought I'd throw it out there.
All in all, I'd say he's probably on to something. I wonder if Mearls & Co. have seen anything like this.

[zugschef]

I happened to have run across THIS little piece of commentary -- just thought I'd throw it out there.
All in all, I'd say he's probably on to something. I wonder if Mearls & Co. have seen anything like this.

For finding these problems and holes, though, playtesting isn't just inefficient and incomplete, it's also ineffective. Even considering the long odds that a particular rules hole will reveal itself during a playtest, most role-playing groups are not willing to simply leave a rules hole as a rules hole. They will patch it with their pre-conceptions, prejudices, and group social contract. Furthermore -- and there's some interesting social context stuff here which I don't want to get into in this essay -- they are not likely to even remember that the rules hole came up: once they've papered over it, it may as well not exist. The chances of actually getting them to report it to you, the designer are nearly nil. Even if you, the designer, are playing, the chances of you remembering it as a hole are low.

good read.

[Voss]

They actually had a decent chassis for a viable class for one shining moment, but rather than fine tune the damage aspect, they tossed it out, and rewrote everything from scratch. The result is worse in every way, apparently at least partially because it was too complicated to have a dice pool refresh every turn.

I don't remember it ever being able to do anything out of the melee fighting schtick. Am I forgetting something or was it more of a "If we are doing the melee fighting thing you can hold your own"? Meaning its still irrelevant once something flies, or the group wants to do anything other than hit it with a stick.

They did ranged equally effectively out of the box. There were a select few of their combat maneuvers that were specific, but you could easily build a dagger guy or bow guy as easily as sword and board guy.

If you mean really exotic abilities, no one really had any at that point, even the casters. There was some illusion and charm bullshit, but crazy magic had been pretty much written out of the game already (and largely remains out).


wotmaniac- what a terrible article. Just like Wizards, he misses the point. Yes, the way they are playtesting is stupid, because their looking solely for opinion/feeling based feedback, but real, effective playtesting really hammers on mechanics, and divorces itself from standard group play. You test the math repeatedly, adding in all the modifiers and dice roll permutations. Note that you don't actually roll dice, you just plug the inputs into the system and make sure the outputs match expectations and desired results.

You do real group playtests later to make sure the game doesn't suck (and people actually want to play it), but it is separate from mechanical testing.

Who is this guy and why should we care about an uninformed and rather preachy rant?

[Seerow]

I happened to have run across THIS little piece of commentary -- just thought I'd throw it out there.
All in all, I'd say he's probably on to something. I wonder if Mearls & Co. have seen anything like this.

But if your game is simpler, like say 3rd edition D&D

Sorry stopped reading because I couldn't stop laughing after we hit this part.

[OgreBattle]

Every time I read about D&D5e, it increases my like of Adventurer Conqueror King.

[Dogbert]

Mearls would be happiest if he could just make the PHB systemless and just have essays on "How to make your D&D game feel more like D&D and have more D&D in it."

Mind you, as someone who broke up with DnD years ago, that'd make ME happy.

For a moment I thought Monte Cook could re-spark my interest in DnD (you can buy from his site what I'm pretty sure would have been Monte's 5E before Mearls alienated him, now with another name, the serials filed off, and an euro-fantasy setting) but then well... suffice to say Monte is now going through a "Now we do things my way!" Siembieda phase that alienated me in turn.

...that will teach me to pay for products I haven't tried first.

[wotmaniac]

wotmaniac- what a terrible article.

meh.

You test the math repeatedly, adding in all the modifiers and dice roll permutations. Note that you don't actually roll dice, you just plug the inputs into the system and make sure the outputs match expectations and desired results.

His assertion is that you don't need teams of playtesters for this .... there's actually a whole paragraph or 2 that talks about this.

You do real group playtests later to make sure the game doesn't suck (and people actually want to play it), but it is separate from mechanical testing.

Yeah, that was his point. Like, that's literally what he said.

Who is this guy

Some Indie game designer. Apparently, he's won some Indie awards or whatever (not that that really means anything)

and why should we care about an uninformed and rather preachy rant?

I just thought it was an interesting read, and thought I'd share it .... it seemed somewhat relevant to the topic.

[tussock]

@Shadzar: Did you really just complain to me that professionals are paid to do things by their employers? Is the water also annoyingly wet where you live?

Just to be clear, when I mumble away about corporate style and things, it's that Hasbro requires every division to pull in $50 million a year. It's not about making good games or retaining customers or expanding your market or anything, just "you must be this rich to ride here". But that's not at all the fault of their employees. It's like Bioware (before EA bought them) vs EA. Beautiful but buggy games that people love for life and mod the fuck out of forever vs crippled formulaic shovelware full of hidden costs that pulls in more money by leveraging some expensive-ass digital rights packages.


@Seerow: 3e is simpler than MTG, it's it? On a mechanical level.

[RobbyPants]

Note: after over two dozen Skill Challenge overhauls by Mike Mearls personally, they were officially abandoned with little fanfare in 2011. No "final fix" ever came.

What do you mean that they "officially abandoned" Skill Challenges?

[Seerow]

@Seerow: 3e is simpler than MTG, it's it? On a mechanical level.

I've never actually played MTG, but any implication that 3e is a simple game is one that's going to make me laugh regardless of what it's being compared to.

[Mistborn]

@Seerow: 3e is simpler than MTG, it's it? On a mechanical level.

I've never actually played MTG, but any implication that 3e is a simple game is one that's going to make me laugh regardless of what it's being compared to.

The core system of MTG is definitely simpler than 3e, if you include every single keyword mechanic MTG is way more complicated.

[Username17]

Note: after over two dozen Skill Challenge overhauls by Mike Mearls personally, they were officially abandoned with little fanfare in 2011. No "final fix" ever came.

What do you mean that they "officially abandoned" Skill Challenges?

Work on them was discontinued, the regular fucking column they had about throwing up incremental changes to try to fix those fucking things was cancelled, they answered in a Q&A session that they weren't going to be using "skill challenges" in future works, and the people they had working on skill challenges were assigned to other projects or fired. I don't know how much more "officially abandoned" they could possibly be. They stopped working on the things, and they stopped pretending to work on the things. They didn't even bother to release a last gasp of "final skill challenge rules".

-Username17

[DragonChild]

People are being overly bitching to the playtesting article just to have something to bitch over. I've been saying the same thing myself for years - "playtesting", as in, "running a campaign to test the game's balance", is completely useless. What you need to do is first have your material edited (by other people, to check flaws and impressions), then write up characters and compare to each other and monsters (to check balance), and then playtest with no intervention and with the designer observing and not answering questions (ONLY to see if the rules are too cumbersome in play and if people understand them on their own). Thinking actual campaign playtesting is a thing you should do is dumb as hell.

[Wrathzog]

I happened to have run across THIS little piece of commentary -- just thought I'd throw it out there.
All in all, I'd say he's probably on to something. I wonder if Mearls & Co. have seen anything like this.

I appreciate the dude's Intent, even though I'm pretty sure he's only hitting the surface of the actual problem (that tabletop development methodologies fucking suck and are like 20 years behind other, similar fields). But it all starts with introspection, so good on him.

[Voss]

People are being overly bitching to the playtesting article just to have something to bitch over. I've been saying the same thing myself for years - "playtesting", as in, "running a campaign to test the game's balance", is completely useless. What you need to do is first have your material edited (by other people, to check flaws and impressions), then write up characters and compare to each other and monsters (to check balance), and then playtest with no intervention and with the designer observing and not answering questions (ONLY to see if the rules are too cumbersome in play and if people understand them on their own). Thinking actual campaign playtesting is a thing you should do is dumb as hell.

Which is why the article is dumb as fuck. No one sane is talking about running a campaign when they talk about playtesting, in much the same way that you don't put actual people inside when you test cars to destruction. It is a pure straw man- even Wizards isn't really doing that- they're keeping all the mechanical shit at home (even though they don't do it very well) and taking feedback on emotional reactions and whether it 'feels' like D&D or not. The most they ask is if something 'feels' overpowered, but their numbers are coming out of their ass and aren't be adjusted for feedback- they're just tossing out systems that aren't 'liked.' The bad math has stayed more or less intact throughout the entire process, which is a totally different issue.

[MisterDee]

As far as I know, in-house playtesting is still just people running campaigns, however. Certainly, both Paizo's and WotC's crews have come out in the past with stuff that has happened in their campaigns.

It's absolutely terrible procedure, but it would explain why neither companies seems able to spot loopholes. It still doesn't explain why the fuck the caster-martial divide still exists, though.