Defining "objective difficulty" (sans baggage)

General questions, debates, and rants about RPGs

Moderator: Moderators

User avatar
wotmaniac
Knight-Baron
Posts: 888
Joined: Sun Mar 13, 2011 11:40 am
Location: my house

Defining "objective difficulty" (sans baggage)

Post by wotmaniac »

Alright, the other thread has become such an abortion because of "Team LiveForever-vs-Team DeadForever" baggage, that I thought that another, not-so-bogged-down thread was warranted.
So, here, the "Team LiveForever vs. Team DeadForever" argument is completely irrelevant -- this is purely a mental exercise meant to finally codify a particular concept, so that the term can be used in a consistent manner.

Defining Objective Difficulty in RPGs:
Obviously, any given game, as a whole, can't be said to have anything remotely resembling an "objective difficulty" -- in that context, it isn't even a thing. Even a given campaign/adventure can only be said to be "difficult" (or not) in only the most general of terms (well, sorta ...). I posit that "objective difficulty" can must be defined at the encounter level.

That being said, let's get a couple of things out of the way ....
First, "[...] deciding factor on the difficulty and lethality of a D&D game" needs to be corrected. "Difficulty" is a thing that can be objectively measured (which I will attempt to show here); while "lethality" is 100% subjective. Since this is about objective difficulty, let's just go ahead and toss "lethality" (with all the associated baggage) straight out the window.
Second, subjective factors have no place in a discussion about defining objective difficulty -- again, leave that for another discussion. (for example, "difficult for 'who'?" is subjective)

Okay, now on to the meat of this.
Alright, with all this build-up, you might be suspect that I'm about to attempt to impart some grand insight -- but I'm not. The term "objective difficulty" simply has only a specific use; and it seems to be getting misused.
The task of "slay the fire giant" actually has an objective difficulty. Said fire giant has clearly defined and quantified offensive and defensive capabilities. As such, you can actually measure the conditions that must be met in order to "slay the fire giant"; and it is that measure by which "objective difficulty" is determined. You can even add terrain features and still measure "objective difficulty", since that terrain has specific, quantified effects.

Now, anything past that is where things get dicey.
Take an actual party for example. Even simply using a "standard" core-only party, subjectivity is through the roof; and that's before you even factor in individual players.
The DM? Yup, how (s)he implements the creature/encounter is completely subjective. However, even with that in mind, on some level, specific tactics are objective, as they are often quite defined and quantified. But, again, implementation introduces a subjective element to a given situation.



TL;DR -- "objective difficulty" is a real thing; if you want to use the term, use it correctly (and even then, a given situation may render it meaningless)


Discuss.
*WARNING*: I say "fuck" a lot.
"The most patriotic thing you can do as an American is to become filthy, filthy rich."
- Mark Cuban

"Game design has no obligation to cater to people who don’t buy into the premise of the game"

TGD -- skirting the edges of dickfinity since 2003.

Public Service Announcement
User avatar
Vebyast
Knight-Baron
Posts: 801
Joined: Tue Mar 23, 2010 5:44 am

Post by Vebyast »

To pull some real math behind this, you can measure the "difficulty" of a task by figuring out how many bits of information are expected to be required to locate a series of actions which completes the task in the configuration space of all possible valid sequences of actions. This is a kind of backwards application of a slightly more common formulation in machine learning where the "power" of a learner is measured by how efficient it is at locating hypotheses which fit the given data; in this case we're measuring difficulty by asking how "smart" you have to be to solve the task with the given constraints.

Of course, given that the domain is utterly and completely intractable with modern techniques, this is simply an argument for the existence of "objective" difficulty. Approximately as relevant to coming up with an actual number as the busy beaver argument or a spherical cow in a vacuum. But it is possible to define an "objective" measure of difficulty.
Last edited by Vebyast on Thu Apr 25, 2013 12:57 am, edited 1 time in total.
DSMatticus wrote:There are two things you can learn from the Gaming Den:
1) Good design practices.
2) How to be a zookeeper for hyper-intelligent shit-flinging apes.
User avatar
wotmaniac
Knight-Baron
Posts: 888
Joined: Sun Mar 13, 2011 11:40 am
Location: my house

Post by wotmaniac »

Vebyast wrote: this is simply an argument for the existence of "objective" difficulty. Approximately as relevant to coming up with an actual number as the busy beaver argument or a spherical cow in a vacuum.
But that's just the thing -- this very thing is being argued in no less than 4 threads.
:confused:
*WARNING*: I say "fuck" a lot.
"The most patriotic thing you can do as an American is to become filthy, filthy rich."
- Mark Cuban

"Game design has no obligation to cater to people who don’t buy into the premise of the game"

TGD -- skirting the edges of dickfinity since 2003.

Public Service Announcement
User avatar
Wrathzog
Knight-Baron
Posts: 605
Joined: Mon Mar 21, 2011 5:57 am

Post by Wrathzog »

Well, the closest thing to it in 3E D&D is the CR system and while the concept was sound, I don't think that anyone's going to argue that it was implemented well.

4E actually did a better job of this to where you could grab a monster of a certain level and then make it an elite or boss or not and you were good to go (in theory). It helps that 4E is a way simpler game than 3E but whatever.
Wotmaniac wrote:But that's just the thing -- this very thing is being argued in no less than 4 threads.
And the only thing I've gotten out of all of those discussions is how little Objective difficulty means in the face of Subjective difficulty in regards to an actual game.
PSY DUCK?
User avatar
RadiantPhoenix
Prince
Posts: 2668
Joined: Sun Apr 11, 2010 10:33 pm
Location: Trudging up the Hill

Post by RadiantPhoenix »

There's a difference between the existence of an objective measure of objective difficulty and the existence of objective difficulty itself.
User avatar
Foxwarrior
Duke
Posts: 1639
Joined: Thu Nov 11, 2010 8:54 am
Location: RPG City, USA

Post by Foxwarrior »

Actually, Vebyast, isn't that method going to give you the Subjective difficulty? Unless you include character optimization as part of the challenge description, each such impossible calculation will only be good for a single party, (Subjective difficulty), and including character optimization isn't reasonable for assessing individual encounters because you aren't normally allowed to make new characters before every battle.

...Unless you're running some sort of serial TPK campaign. Hmm...
User avatar
wotmaniac
Knight-Baron
Posts: 888
Joined: Sun Mar 13, 2011 11:40 am
Location: my house

Post by wotmaniac »

Wrathzog wrote:And the only thing I've gotten out of all of those discussions is how little Objective difficulty means in the face of Subjective difficulty in regards to an actual game.
wotmaniac wrote:a given situation may render it meaningless
Though, to be fair, those discussions have devolved in to unintelligible nonsense by this point. By reading just those threads, I don't even know what "objective difficulty" is even supposed to mean in any given context, let alone on its face.
I'd like to see at least a consistent usage (if its even going to be used at all).

RadiantPhoenix wrote:There's a difference between the existence of an objective measure of objective difficulty and the existence of objective difficulty itself.
Yeah, the CR system, while a laudable idea, really sucked in execution.
As a measure of objective difficulty, it really is rather craptacular; and to try to extract any meaning in relation to subjective difficulty .... well, slamming your head with a 3-lb sledge might render better results. (note: I can only speak on 3.x; as I've not actually played 4e; but from what I understand, when your design paradigm has all the depth of a cookie sheet, then I can't imagine it would be that difficult)
Yes, objective difficulty can be measured; but the CR system is not the way to do it.

Tangential side note: I can remember back in 2e that you were able to simply able to look at the XP awarded by the creature to roughly determine difficulty; and I also seem to remember that I had somehow worked out some sort of formula/matrix in my head for actually figuring out level-appropriateness.
Or not -- that was 13 years ago, so my memory could be slipping. And without going back and actually tearing apart the system, I couldn't tell you if the XP award was even close to an accurate representation of objective difficulty (but my guess is that it was no better than the CR system).

Foxwarrior wrote:...Unless you're running some sort of serial TPK campaign. Hmm...
Image
*WARNING*: I say "fuck" a lot.
"The most patriotic thing you can do as an American is to become filthy, filthy rich."
- Mark Cuban

"Game design has no obligation to cater to people who don’t buy into the premise of the game"

TGD -- skirting the edges of dickfinity since 2003.

Public Service Announcement
Seerow
Duke
Posts: 1103
Joined: Sun Apr 03, 2011 2:46 pm

Post by Seerow »

This is a groundbreaking thread that will surely accomplish everything the last 5 on this subject did not.
User avatar
wotmaniac
Knight-Baron
Posts: 888
Joined: Sun Mar 13, 2011 11:40 am
Location: my house

Post by wotmaniac »

Seerow wrote:This is a groundbreaking thread that will surely accomplish everything the last 5 on this subject did not.
wotmaniac wrote:you might be suspect that I'm about to attempt to impart some grand insight -- but I'm not.
See, I beat you to the punch way back in the OP. I even made sure to explicate the full intent .... which you obviously missed as well.
So, you might want to try your hand at something other than trolling, because you don't seem to be particularly good at it. :tonguesmilie:
*WARNING*: I say "fuck" a lot.
"The most patriotic thing you can do as an American is to become filthy, filthy rich."
- Mark Cuban

"Game design has no obligation to cater to people who don’t buy into the premise of the game"

TGD -- skirting the edges of dickfinity since 2003.

Public Service Announcement
Seerow
Duke
Posts: 1103
Joined: Sun Apr 03, 2011 2:46 pm

Post by Seerow »

wotmaniac wrote:
Seerow wrote:This is a groundbreaking thread that will surely accomplish everything the last 5 on this subject did not.
wotmaniac wrote:you might be suspect that I'm about to attempt to impart some grand insight -- but I'm not.
See, I beat you to the punch way back in the OP. I even made sure to explicate the full intent .... which you obviously missed as well.
So, you might want to try your hand at something other than trolling, because you don't seem to be particularly good at it. :tonguesmilie:
You imply that I actually needed to read anything more than the topic title to know this entire thread was going to be a waste of everyone involved's time. The fact that you were aware of that from the start just makes the fact that you went through with it that much sadder.
User avatar
Vebyast
Knight-Baron
Posts: 801
Joined: Tue Mar 23, 2010 5:44 am

Post by Vebyast »

Foxwarrior wrote:Actually, Vebyast, isn't that method going to give you the Subjective difficulty? Unless you include character optimization as part of the challenge description, each such impossible calculation will only be good for a single party, (Subjective difficulty), and including character optimization isn't reasonable for assessing individual encounters because you aren't normally allowed to make new characters before every battle.
What you call 'subjectivity' is simply free or unobserved variables that make the task more difficult. Technically speaking, what you're trying to do is bias the distribution of possible futures toward regions of the configuration space of the universe that you think are better. As a trivial example, say that your goal is to maximize your roll on 1d6+c, where c is some variable you can choose from the set [1,2,3]. If you choose c=3, then you remove from the list of possible states the future could be in all outcomes where your roll is less than four. It could be five, it could be nine, it could be six, but it can't be two: you've pushed a whole bunch of the probability distribution of possible future states into outcomes which you prefer. The key to all this is that, thanks to thermodynamics and information theory, you've had to move some bits of information around to coerce the future into this new distribution of states. How many bits of information you had to use, and how successful you were, is a measure of how difficult the task was and how good you at optimizing. Given two problems of equal difficulty, the "smarter" agent will be use fewer bits of optimization to succeed; given two agents of identical power, the easier task can be completed with fewer bits of optimization. We can get an objective measure of difficulty by asking either how many bits we expect the problem to require if handed to a completely random optimizing agent, or by handing different problems to the same agent and comparing the results.

To apply this to the dungeons and dragons problem, your task at any given point is to bias the future toward states in which your character has completed whatever challenges faced it. If you're playing as part of a group, the other players are all making decisions, which affects how the future will look, which in turn affects how well you'll be able to move the future toward preferable states.

This is why I say that this measure is effectively unusable and only useful for an existence proof. Evaluating and influencing the distribution of possible futures is trivial for situations like tic-tac-toe and chess, difficult for situations like the stock market and grand military strategy, and nearly bloody impossible for situations like academic research and social interaction. The exact difficulty of "dungeons and dragons" depends on whether the problem is "wednesday night gaming", "enter a charop competition", or "demonstrate rogue-level performance on the SGT", but typically falls closer to "social interaction" and "grand strategy" than "chess". I mean, we're talking about absolutely ridiculous configuration spaces here. You would have more luck brute-forcing the busy beaver function than you would actually computing any of the numbers I'm talking about here.

As I mentioned, this was originally developed to measure how "smart" AI agents were being on machine learning tasks. It's been applied to AI philosophy in a roughly similar manner. This formulation isn't particularly rigorous or straightforward, mostly because I'm coming up with it on the fly after turning everything involved on its head.
Last edited by Vebyast on Thu Apr 25, 2013 6:19 am, edited 3 times in total.
DSMatticus wrote:There are two things you can learn from the Gaming Den:
1) Good design practices.
2) How to be a zookeeper for hyper-intelligent shit-flinging apes.
User avatar
wotmaniac
Knight-Baron
Posts: 888
Joined: Sun Mar 13, 2011 11:40 am
Location: my house

Post by wotmaniac »

Vebyast wrote:This is why I say that this measure is effectively unusable and only useful for an existence proof.
This is actually part of my point. I'm just going insane over the other 4 threads, where the term keeps getting batted around with no rhyme or reason; and then people want to argue over whether or not it's actually a thing, without even agreeing on what it's supposed to mean.:confused:
Evaluating and influencing the distribution of possible futures is trivial for situations like tic-tac-toe and chess, difficult for situations like the stock market and grand military strategy, and nearly bloody impossible for situations like academic research and social interaction. The exact difficulty of "dungeons and dragons" depends on whether the problem is "wednesday night gaming", "enter a charop competition", or "demonstrate rogue-level performance on the SGT", but typically falls closer to "social interaction" and "grand strategy" than "chess".
And now I'm confused as to your point, because these are all subjective factors.
(I'm sure there's something in your intent that I'm missing)

Sure, the CR system was indeed an attempt at measuring objective difficulty .... it was just really bad at it.
*WARNING*: I say "fuck" a lot.
"The most patriotic thing you can do as an American is to become filthy, filthy rich."
- Mark Cuban

"Game design has no obligation to cater to people who don’t buy into the premise of the game"

TGD -- skirting the edges of dickfinity since 2003.

Public Service Announcement
User avatar
Vebyast
Knight-Baron
Posts: 801
Joined: Tue Mar 23, 2010 5:44 am

Post by Vebyast »

How about this: if you sum over all possible future configurations of the universe, there is no such thing as "subjective", because by considering all possible configurations you can catch all of the causal effects that you would normally label as "subjective" and ignore. You routinely do so in many circumstances; for example, calculating the expected damage output of a particular beatstick build involves considering every possible way the dice could be rolled. It is a trivial mathematical exercise to extend the same procedure to things like "what decisions are the rest of the players going to make when building their characters" and "what kind of traps do I expect the DM to throw at me", but that makes the complexity of the problem blow up and the explicit approach becomes useless. Which is why you usually mask off parts of the problem by labeling them as "subjective" and ignoring them. It is possible, however impractical, to consider everything completely objectively.

So maybe this is a question of definitions. Mathematics uses the words "possible" and "definition" in ways which are completely different from their casual application.
Last edited by Vebyast on Thu Apr 25, 2013 6:41 am, edited 1 time in total.
DSMatticus wrote:There are two things you can learn from the Gaming Den:
1) Good design practices.
2) How to be a zookeeper for hyper-intelligent shit-flinging apes.
Fuchs
Duke
Posts: 2446
Joined: Thu Oct 02, 2008 7:29 am
Location: Zürich

Post by Fuchs »

The problem is that in a roleplaying game even a seemingly clear-cut task of "slay the fire giant" needs a lot of restrictions and additional definitions and descriptions. I know of a player whose character's reaction to that task would be "hire someone else to do it". How easy or hard this is depends on money and NPCs available to the characters, which would need to be defined. Then there the enviroment - traps, resources, terrain, etc. - and time limits to consider (for prep times and limits). And even Religion, for the "we use Divination to prepare for each day" tactic that often crops up on some discussions about wizard strengths.

I would say you might be able to define objective difficlty for a very narrow, very limited and very unique single challenge/Encounter, but not for anything beyound that, meaning, for anything that comes even close to an adventure, much less a campaign.
User avatar
nockermensch
Duke
Posts: 1898
Joined: Fri Jan 06, 2012 1:11 pm
Location: Rio: the Janeiro

Post by nockermensch »

I think objective difficulty is a perfectly valid concept to be discussing during the design stage of a game. Having a context free SGT is an useful tool when you're creating and fine-tuning a game.

Even when designing modules for publication, you need to have a good grasp of what consists an easy, medium or hard obstacle for a party of level X. All this it's indisputable.

Some objectivity even survives contact with the players. It's also indisputable that if you conduct a player-base survey, you'll find that everybody considers 24 bugbears more dangerous than 1 bugbear, a beholder more difficult than an owlbear and an adamantite wall harder to bypass than an iron one. Even more: People will (eventually) notice the anomalies in the expected difficulty curve, like dragons and illithids.

But then I think basically all the objectiveness evaporates in the moment it touches actual games. Once you add campaign context, party composition, DMing style, house rules and so, the discussion becomes much more muddled: the DM forgot to act for the extra bugbears, the beholder ends being argued with and becomes an ally, and by the time the party meets adamantite walls people are already insubstantial or have at-will attacks that ignore hardness and don't even care.
@ @ Nockermensch
Koumei wrote:After all, in Firefox you keep tabs in your browser, but in SovietPutin's Russia, browser keeps tabs on you.
Mord wrote:Chromatic Wolves are massively under-CRed. Its "Dood to stone" spell-like is a TPK waiting to happen if you run into it before anyone in the party has Dance of Sack or Shield of Farts.
User avatar
Ancient History
Serious Badass
Posts: 12708
Joined: Wed Aug 18, 2010 12:57 pm

Post by Ancient History »

The objective difficulty of a game is equivalent to the odds of the most difficult roll players are expected to make. The PC's approach to the adventure is compeltely subjective, and should not be taken into account - lots of players make their lives harder for themselves completely discrete from the mechanical toughness of an adventure.
User avatar
Foxwarrior
Duke
Posts: 1639
Joined: Thu Nov 11, 2010 8:54 am
Location: RPG City, USA

Post by Foxwarrior »

Well, if when you said "lots of players make their lives harder for themselves" you meant to add "...by not being omniscient", I agree.

For the sake of liking Vebyast's definition somewhat more than yours, Ancient History, let us take a hypothetical TTRPG where encounters are resolved by games of chess between the player(s) and the DM, using exotic board arrangements to symbolize the different conflicts.

No rolling is necessary.
User avatar
RadiantPhoenix
Prince
Posts: 2668
Joined: Sun Apr 11, 2010 10:33 pm
Location: Trudging up the Hill

Post by RadiantPhoenix »

I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."

If skill and effort have no effect on outcome, then the objective difficulty is either zero or infinity.
zugschef
Knight-Baron
Posts: 821
Joined: Sat Feb 02, 2013 1:53 pm

Post by zugschef »

RadiantPhoenix wrote:If skill and effort have no effect on outcome, then the objective difficulty is either zero or infinity.
that is a totally trivial insight.
User avatar
RadiantPhoenix
Prince
Posts: 2668
Joined: Sun Apr 11, 2010 10:33 pm
Location: Trudging up the Hill

Post by RadiantPhoenix »

zugschef wrote:that is a totally trivial insight.
It is contradictory to Ancient History's measure.

Trivial case: The campaign consists of one die roll. That roll has a 50% chance of success, no matter what the players do. Success wins the campaign, failure fails the campaign.

By my measure, the difficulty is zero if 50% is "good enough", and infinity if 50% isn't "good enough"

By Ancient History's measure, the difficulty is 50%.
User avatar
Vebyast
Knight-Baron
Posts: 801
Joined: Tue Mar 23, 2010 5:44 am

Post by Vebyast »

RadiantPhoenix wrote:I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."

If skill and effort have no effect on outcome, then the objective difficulty is either zero or infinity.
This is almost exactly the core idea of the definition I put together. Add the gory mathematical details, the thermodynamics/information theory stuff, and a way to handle situations more complicated than success/failure, and you've got what I wrote. Sorry I couldn't put it this simply.
Ancient History wrote:The objective difficulty of a game is equivalent to the odds of the most difficult roll players are expected to make. The PC's approach to the adventure is compeltely subjective, and should not be taken into account - lots of players make their lives harder for themselves completely discrete from the mechanical toughness of an adventure.
Consider a session that will, due to structural mechanics, consist of approximately three hundred rolls a d%, each of which must be not a 1 in order for the PCs to succeed. The most difficult single roll will succeed with probability .99, but the overall session will succeed with probability approximately .05.
Last edited by Vebyast on Thu Apr 25, 2013 7:20 pm, edited 3 times in total.
DSMatticus wrote:There are two things you can learn from the Gaming Den:
1) Good design practices.
2) How to be a zookeeper for hyper-intelligent shit-flinging apes.
User avatar
shadzar
Prince
Posts: 4922
Joined: Fri Jun 26, 2009 6:08 pm

Post by shadzar »

RadiantPhoenix wrote:I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."
may sound a silly question in regards to this but, "make their choices" would refer to?

a) CharOp (character creation of any kind)
b) courses of action during play
c) both of the above
d) something else?
Last edited by shadzar on Thu Apr 25, 2013 7:14 pm, edited 1 time in total.
Play the game, not the rules.
Swordslinger wrote:Or fuck it... I'm just going to get weapon specialization in my cock and whip people to death with it. Given all the enemies are total pussies, it seems like the appropriate thing to do.
Lewis Black wrote:If the people of New Zealand want to be part of our world, I believe they should hop off their islands, and push 'em closer.
good read (Note to self Maxus sucks a barrel of cocks.)
User avatar
wotmaniac
Knight-Baron
Posts: 888
Joined: Sun Mar 13, 2011 11:40 am
Location: my house

Post by wotmaniac »

Furthermore, player inputs only serve to affect the subjective difficulty.
(and MC inputs, for that matter)


[edit]
@shadzar:
Yes.
Last edited by wotmaniac on Thu Apr 25, 2013 7:15 pm, edited 1 time in total.
*WARNING*: I say "fuck" a lot.
"The most patriotic thing you can do as an American is to become filthy, filthy rich."
- Mark Cuban

"Game design has no obligation to cater to people who don’t buy into the premise of the game"

TGD -- skirting the edges of dickfinity since 2003.

Public Service Announcement
User avatar
RadiantPhoenix
Prince
Posts: 2668
Joined: Sun Apr 11, 2010 10:33 pm
Location: Trudging up the Hill

Post by RadiantPhoenix »

shadzar wrote:
RadiantPhoenix wrote:I think that the objective difficulty of a game is best measured by, "how well do the players have to make their choices in order to achieve a satisfactory success rate."
may sound a silly question in regards to this but, "make their choices" would refer to?

a) CharOp (character creation of any kind)
b) courses of action during play
c) both of the above
d) something else?
Depends on the game, but generally C.

You can also define multiple "levels" of success and somehow generate some numbers that represent the difficulty of each "success level."

You can also definitely subdivide the game into minigames, each of which has its own objective difficulty.

For example, a given game might have CharOp, Tactical, and "Fellate Mister Cavern" minigames, each of which contributes to "success".

I'm experiencing difficulty coming up with a simple-sounding way of combining the difficulties of minigames, probably because I don't have any concept of what the difficulty number would actually be.

If it's something like, "Brain cycles," then you probably just add them, but if it's more complicated, it's going to get more complicated. (tautology, I know)
User avatar
Vebyast
Knight-Baron
Posts: 801
Joined: Tue Mar 23, 2010 5:44 am

Post by Vebyast »

RadiantPhoenix wrote:If it's something like, "Brain cycles," then you probably just add them, but if it's more complicated, it's going to get more complicated. (tautology, I know)
That's why I went into the math and information theory. The measure actually is brain cycles in a way, and the unit is in bits and represents how much entropy you had to shift around to accomplish your task.
DSMatticus wrote:There are two things you can learn from the Gaming Den:
1) Good design practices.
2) How to be a zookeeper for hyper-intelligent shit-flinging apes.
Post Reply