Defining "objective difficulty" (sans baggage)

[wotmaniac]

Alright, the other thread has become such an abortion because of "Team LiveForever-vs-Team DeadForever" baggage, that I thought that another, not-so-bogged-down thread was warranted.
So, here, the "Team LiveForever vs. Team DeadForever" argument is completely irrelevant -- this is purely a mental exercise meant to finally codify a particular concept, so that the term can be used in a consistent manner.

Defining Objective Difficulty in RPGs:
Obviously, any given game, as a whole, can't be said to have anything remotely resembling an "objective difficulty" -- in that context, it isn't even a thing. Even a given campaign/adventure can only be said to be "difficult" (or not) in only the most general of terms (well, sorta ...). I posit that "objective difficulty" can must be defined at the encounter level.

That being said, let's get a couple of things out of the way ....
First, "[...] deciding factor on the difficulty and lethality of a D&D game" needs to be corrected. "Difficulty" is a thing that can be objectively measured (which I will attempt to show here); while "lethality" is 100% subjective. Since this is about objective difficulty, let's just go ahead and toss "lethality" (with all the associated baggage) straight out the window.
Second, subjective factors have no place in a discussion about defining objective difficulty -- again, leave that for another discussion. (for example, "difficult for 'who'?" is subjective)

Okay, now on to the meat of this.
Alright, with all this build-up, you might be suspect that I'm about to attempt to impart some grand insight -- but I'm not. The term "objective difficulty" simply has only a specific use; and it seems to be getting misused.
The task of "slay the fire giant" actually has an objective difficulty. Said fire giant has clearly defined and quantified offensive and defensive capabilities. As such, you can actually measure the conditions that must be met in order to "slay the fire giant"; and it is that measure by which "objective difficulty" is determined. You can even add terrain features and still measure "objective difficulty", since that terrain has specific, quantified effects.

Now, anything past that is where things get dicey.
Take an actual party for example. Even simply using a "standard" core-only party, subjectivity is through the roof; and that's before you even factor in individual players.
The DM? Yup, how (s)he implements the creature/encounter is completely subjective. However, even with that in mind, on some level, specific tactics are objective, as they are often quite defined and quantified. But, again, implementation introduces a subjective element to a given situation.



TL;DR -- "objective difficulty" is a real thing; if you want to use the term, use it correctly (and even then, a given situation may render it meaningless)


Discuss.

[Vebyast]

To pull some real math behind this, you can measure the "difficulty" of a task by figuring out how many bits of information are expected to be required to locate a series of actions which completes the task in the configuration space of all possible valid sequences of actions. This is a kind of backwards application of a slightly more common formulation in machine learning where the "power" of a learner is measured by how efficient it is at locating hypotheses which fit the given data; in this case we're measuring difficulty by asking how "smart" you have to be to solve the task with the given constraints.

Of course, given that the domain is utterly and completely intractable with modern techniques, this is simply an argument for the existence of "objective" difficulty. Approximately as relevant to coming up with an actual number as the busy beaver argument or a spherical cow in a vacuum. But it is possible to define an "objective" measure of difficulty.

[wotmaniac]

this is simply an argument for the existence of "objective" difficulty. Approximately as relevant to coming up with an actual number as the busy beaver argument or a spherical cow in a vacuum.

But that's just the thing -- this very thing is being argued in no less than 4 threads.

[Wrathzog]

Well, the closest thing to it in 3E D&D is the CR system and while the concept was sound, I don't think that anyone's going to argue that it was implemented well.

4E actually did a better job of this to where you could grab a monster of a certain level and then make it an elite or boss or not and you were good to go (in theory). It helps that 4E is a way simpler game than 3E but whatever.

But that's just the thing -- this very thing is being argued in no less than 4 threads.

And the only thing I've gotten out of all of those discussions is how little Objective difficulty means in the face of Subjective difficulty in regards to an actual game.

[RadiantPhoenix]

There's a difference between the existence of an objective measure of objective difficulty and the existence of objective difficulty itself.

[Foxwarrior]

Actually, Vebyast, isn't that method going to give you the Subjective difficulty? Unless you include character optimization as part of the challenge description, each such impossible calculation will only be good for a single party, (Subjective difficulty), and including character optimization isn't reasonable for assessing individual encounters because you aren't normally allowed to make new characters before every battle.

...Unless you're running some sort of serial TPK campaign. Hmm...

[wotmaniac]

And the only thing I've gotten out of all of those discussions is how little Objective difficulty means in the face of Subjective difficulty in regards to an actual game.

a given situation may render it meaningless

Though, to be fair, those discussions have devolved in to unintelligible nonsense by this point. By reading just those threads, I don't even know what "objective difficulty" is even supposed to mean in any given context, let alone on its face.
I'd like to see at least a consistent usage (if its even going to be used at all).

There's a difference between the existence of an objective measure of objective difficulty and the existence of objective difficulty itself.

Yeah, the CR system, while a laudable idea, really sucked in execution.
As a measure of objective difficulty, it really is rather craptacular; and to try to extract any meaning in relation to subjective difficulty .... well, slamming your head with a 3-lb sledge might render better results. (note: I can only speak on 3.x; as I've not actually played 4e; but from what I understand, when your design paradigm has all the depth of a cookie sheet, then I can't imagine it would be that difficult)
Yes, objective difficulty can be measured; but the CR system is not the way to do it.

Tangential side note: I can remember back in 2e that you were able to simply able to look at the XP awarded by the creature to roughly determine difficulty; and I also seem to remember that I had somehow worked out some sort of formula/matrix in my head for actually figuring out level-appropriateness.
Or not -- that was 13 years ago, so my memory could be slipping. And without going back and actually tearing apart the system, I couldn't tell you if the XP award was even close to an accurate representation of objective difficulty (but my guess is that it was no better than the CR system).

...Unless you're running some sort of serial TPK campaign. Hmm...

[Seerow]

This is a groundbreaking thread that will surely accomplish everything the last 5 on this subject did not.

[wotmaniac]

This is a groundbreaking thread that will surely accomplish everything the last 5 on this subject did not.

you might be suspect that I'm about to attempt to impart some grand insight -- but I'm not.

See, I beat you to the punch way back in the OP. I even made sure to explicate the full intent .... which you obviously missed as well.
So, you might want to try your hand at something other than trolling, because you don't seem to be particularly good at it.

[Seerow]

This is a groundbreaking thread that will surely accomplish everything the last 5 on this subject did not.

you might be suspect that I'm about to attempt to impart some grand insight -- but I'm not.

See, I beat you to the punch way back in the OP. I even made sure to explicate the full intent .... which you obviously missed as well.
So, you might want to try your hand at something other than trolling, because you don't seem to be particularly good at it.

You imply that I actually needed to read anything more than the topic title to know this entire thread was going to be a waste of everyone involved's time. The fact that you were aware of that from the start just makes the fact that you went through with it that much sadder.

[Vebyast]

Actually, Vebyast, isn't that method going to give you the Subjective difficulty? Unless you include character optimization as part of the challenge description, each such impossible calculation will only be good for a single party, (Subjective difficulty), and including character optimization isn't reasonable for assessing individual encounters because you aren't normally allowed to make new characters before every battle.

What you call 'subjectivity' is simply free or unobserved variables that make the task more difficult. Technically speaking, what you're trying to do is bias the distribution of possible futures toward regions of the configuration space of the universe that you think are better. As a trivial example, say that your goal is to maximize your roll on 1d6+c, where c is some variable you can choose from the set [1,2,3]. If you choose c=3, then you remove from the list of possible states the future could be in all outcomes where your roll is less than four. It could be five, it could be nine, it could be six, but it can't be two: you've pushed a whole bunch of the probability distribution of possible future states into outcomes which you prefer. The key to all this is that, thanks to thermodynamics and information theory, you've had to move some bits of information around to coerce the future into this new distribution of states. How many bits of information you had to use, and how successful you were, is a measure of how difficult the task was and how good you at optimizing. Given two problems of equal difficulty, the "smarter" agent will be use fewer bits of optimization to succeed; given two agents of identical power, the easier task can be completed with fewer bits of optimization. We can get an objective measure of difficulty by asking either how many bits we expect the problem to require if handed to a completely random optimizing agent, or by handing different problems to the same agent and comparing the results.

To apply this to the dungeons and dragons problem, your task at any given point is to bias the future toward states in which your character has completed whatever challenges faced it. If you're playing as part of a group, the other players are all making decisions, which affects how the future will look, which in turn affects how well you'll be able to move the future toward preferable states.

This is why I say that this measure is effectively unusable and only useful for an existence proof. Evaluating and influencing the distribution of possible futures is trivial for situations like tic-tac-toe and chess, difficult for situations like the stock market and grand military strategy, and nearly bloody impossible for situations like academic research and social interaction. The exact difficulty of "dungeons and dragons" depends on whether the problem is "wednesday night gaming", "enter a charop competition", or "demonstrate rogue-level performance on the SGT", but typically falls closer to "social interaction" and "grand strategy" than "chess". I mean, we're talking about absolutely ridiculous configuration spaces here. You would have more luck brute-forcing the busy beaver function than you would actually computing any of the numbers I'm talking about here.

As I mentioned, this was originally developed to measure how "smart" AI agents were being on machine learning tasks. It's been applied to AI philosophy in a roughly similar manner. This formulation isn't particularly rigorous or straightforward, mostly because I'm coming up with it on the fly after turning everything involved on its head.

[wotmaniac]

This is why I say that this measure is effectively unusable and only useful for an existence proof.

This is actually part of my point. I'm just going insane over the other 4 threads, where the term keeps getting batted around with no rhyme or reason; and then people want to argue over whether or not it's actually a thing, without even agreeing on what it's supposed to mean.

Evaluating and influencing the distribution of possible futures is trivial for situations like tic-tac-toe and chess, difficult for situations like the stock market and grand military strategy, and nearly bloody impossible for situations like academic research and social interaction. The exact difficulty of "dungeons and dragons" depends on whether the problem is "wednesday night gaming", "enter a charop competition", or "demonstrate rogue-level performance on the SGT", but typically falls closer to "social interaction" and "grand strategy" than "chess".

And now I'm confused as to your point, because these are all subjective factors.
(I'm sure there's something in your intent that I'm missing)

Sure, the CR system was indeed an attempt at measuring objective difficulty .... it was just really bad at it.

[Vebyast]

How about this: if you sum over all possible future configurations of the universe, there is no such thing as "subjective", because by considering all possible configurations you can catch all of the causal effects that you would normally label as "subjective" and ignore. You routinely do so in many circumstances; for example, calculating the expected damage output of a particular beatstick build involves considering every possible way the dice could be rolled. It is a trivial mathematical exercise to extend the same procedure to things like "what decisions are the rest of the players going to make when building their characters" and "what kind of traps do I expect the DM to throw at me", but that makes the complexity of the problem blow up and the explicit approach becomes useless. Which is why you usually mask off parts of the problem by labeling them as "subjective" and ignoring them. It is possible, however impractical, to consider everything completely objectively.

So maybe this is a question of definitions. Mathematics uses the words "possible" and "definition" in ways which are completely different from their casual application.

[Fuchs]

The problem is that in a roleplaying game even a seemingly clear-cut task of "slay the fire giant" needs a lot of restrictions and additional definitions and descriptions. I know of a player whose character's reaction to that task would be "hire someone else to do it". How easy or hard this is depends on money and NPCs available to the characters, which would need to be defined. Then there the enviroment - traps, resources, terrain, etc. - and time limits to consider (for prep times and limits). And even Religion, for the "we use Divination to prepare for each day" tactic that often crops up on some discussions about wizard strengths.

I would say you might be able to define objective difficlty for a very narrow, very limited and very unique single challenge/Encounter, but not for anything beyound that, meaning, for anything that comes even close to an adventure, much less a campaign.

[nockermensch]

I think objective difficulty is a perfectly valid concept to be discussing during the design stage of a game. Having a context free SGT is an useful tool when you're creating and fine-tuning a game.

Even when designing modules for publication, you need to have a good grasp of what consists an easy, medium or hard obstacle for a party of level X. All this it's indisputable.

Some objectivity even survives contact with the players. It's also indisputable that if you conduct a player-base survey, you'll find that everybody considers 24 bugbears more dangerous than 1 bugbear, a beholder more difficult than an owlbear and an adamantite wall harder to bypass than an iron one. Even more: People will (eventually) notice the anomalies in the expected difficulty curve, like dragons and illithids.

But then I think basically all the objectiveness evaporates in the moment it touches actual games. Once you add campaign context, party composition, DMing style, house rules and so, the discussion becomes much more muddled: the DM forgot to act for the extra bugbears, the beholder ends being argued with and becomes an ally, and by the time the party meets adamantite walls people are already insubstantial or have at-will attacks that ignore hardness and don't even care.

[Ancient History]

The objective difficulty of a game is equivalent to the odds of the most difficult roll players are expected to make. The PC's approach to the adventure is compeltely subjective, and should not be taken into account - lots of players make their lives harder for themselves completely discrete from the mechanical toughness of an adventure.

[Foxwarrior]

Well, if when you said "lots of players make their lives harder for themselves" you meant to add "...by not being omniscient", I agree.

For the sake of liking Vebyast's definition somewhat more than yours, Ancient History, let us take a hypothetical TTRPG where encounters are resolved by games of chess between the player(s) and the DM, using exotic board arrangements to symbolize the different conflicts.

No rolling is necessary.

[RadiantPhoenix]

I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."

If skill and effort have no effect on outcome, then the objective difficulty is either zero or infinity.

[zugschef]

If skill and effort have no effect on outcome, then the objective difficulty is either zero or infinity.

that is a totally trivial insight.

[RadiantPhoenix]

that is a totally trivial insight.

It is contradictory to Ancient History's measure.

Trivial case: The campaign consists of one die roll. That roll has a 50% chance of success, no matter what the players do. Success wins the campaign, failure fails the campaign.

By my measure, the difficulty is zero if 50% is "good enough", and infinity if 50% isn't "good enough"

By Ancient History's measure, the difficulty is 50%.

[Vebyast]

I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."

If skill and effort have no effect on outcome, then the objective difficulty is either zero or infinity.

This is almost exactly the core idea of the definition I put together. Add the gory mathematical details, the thermodynamics/information theory stuff, and a way to handle situations more complicated than success/failure, and you've got what I wrote. Sorry I couldn't put it this simply.

The objective difficulty of a game is equivalent to the odds of the most difficult roll players are expected to make. The PC's approach to the adventure is compeltely subjective, and should not be taken into account - lots of players make their lives harder for themselves completely discrete from the mechanical toughness of an adventure.

Consider a session that will, due to structural mechanics, consist of approximately three hundred rolls a d%, each of which must be not a 1 in order for the PCs to succeed. The most difficult single roll will succeed with probability .99, but the overall session will succeed with probability approximately .05.

[shadzar]

I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."

may sound a silly question in regards to this but, "make their choices" would refer to?

a) CharOp (character creation of any kind)
b) courses of action during play
c) both of the above
d) something else?

[wotmaniac]

Furthermore, player inputs only serve to affect the subjective difficulty.
(and MC inputs, for that matter)


[edit]
@shadzar:
Yes.

[RadiantPhoenix]

I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."

may sound a silly question in regards to this but, "make their choices" would refer to?

a) CharOp (character creation of any kind)
b) courses of action during play
c) both of the above
d) something else?

Depends on the game, but generally C.

You can also define multiple "levels" of success and somehow generate some numbers that represent the difficulty of each "success level."

You can also definitely subdivide the game into minigames, each of which has its own objective difficulty.

For example, a given game might have CharOp, Tactical, and "Fellate Mister Cavern" minigames, each of which contributes to "success".

I'm experiencing difficulty coming up with a simple-sounding way of combining the difficulties of minigames, probably because I don't have any concept of what the difficulty number would actually be.

If it's something like, "Brain cycles," then you probably just add them, but if it's more complicated, it's going to get more complicated. (tautology, I know)

[Vebyast]

If it's something like, "Brain cycles," then you probably just add them, but if it's more complicated, it's going to get more complicated. (tautology, I know)

That's why I went into the math and information theory. The measure actually is brain cycles in a way, and the unit is in bits and represents how much entropy you had to shift around to accomplish your task.