Hi Boris,

I understand what you are saying but it is not at all the basis for what I suggest.  I dont have any notions about what the 'right' rating is for any board.

I think the system is unfair because there are cases when someone can play perfectly and still lose points.  This is because the luck factors of dice, placement, cards, and opponents choices for many boards will bring the maximum achievable H rating for that board well below 100%. 

If we assume, for the sake of argument, that there is a board whose maximum achievable H rating is 80% then consider the comment that I have re-posted from above:

There was a comment that if playing those with lower scores makes you drift lower then your score was artificially high.  I assert that this in incorrect.  Lets assume that we know a player is a 2000 rate player playing at 80% win rate.  If that player continues to win at 80% win rate but plays people rated at 500 their score will drop well below 2000.  So their rate was not artificially high but it dropped because there was not a point loss cap.

But you are stating a priori:

Lets assume that we know a player is a 2000 rate player playing at 80% win rate.  If that player continues to win at 80% win rate but plays people rated at 500 their score will drop well below 2000.  So their rate was not artificially high but it dropped because there was not a point loss cap.

Why do you assume that a player who plays at 80% win rate deserves a 2000 score?  Maybe a 80% player deserves a 1600 score.

In fact their overall win rate of 80% is not relevant.  It is their win rate against 500 score players that is relevant.  So I think what you are claiming is that a player who wins 80% of their games against 500 players should have a 2000 score.  But what is that claim based upon?

IMO, the question of fairness is a valid question.  And I think it comes down to something like this.  Assume three players at equilibrium: a high ranking player A has some score X.  A mid ranking player B has a score Y.  And a low ranking player C has a score Z.

A wins m% games against B players, and n% games against C players.

In a fair system, the n% vs. B games and the m% vs. C games would have the same effect on A's score.  That is, it should not matter what the score of their opponents is (assuming they are all at equilibrium).  Whether you play against higher ranking players or lower ranking players your chance to win should balance against the higher/lower scores to affect you the same.

(actually I think the above should hold true even if A is not yet at equilibrium, as long as B & C are.)

Is wargear currently fair in that sense?   Probably not.  Could we change it to make it more fair?  Maybe.  How would we change it?  I have no idea, but I don't think caps are the way to go, and certainly not hard caps that kick in 100% at some arbitrary level.

Hey Ozy,

I am not assuming someone who plays at 80% deserves a 2000 rate score.  This was just an example.

I am saying that a person who happens to have a certain rate and plays perfectly can lose points, it doesn't matter what your rate is.  If you play perfectly you should not lose points.  An underlying assumption of this system is that it is possible to get a 100% H rating on any board.  I am saying that is not possible and because of that a cap is needed.

Squint:

If you play perfectly you should not lose points. 

I disagree wholeheartedly with the above statement.  Plain and simple: This is not a game of skill only.  Luck is ALWAYS going to play a roll (pun intended!!), and if you're disappointed in that, then I think you will find yourself continually disappointed with this whole site (and any rankings herein). 

At no point (that I know of) has anyone stated that the purpose of the ranking is to link directly (and solely) with the players skill level.  We are trying to make a best-guess estimate of a players skill, but in no way do I (or anyone here, I'd assumed) think that it is even remotely accurate.

Ozy:

it should not matter what the score of their opponents is (assuming they are all at equilibrium)

I don't think I agree with this statement either.  Not sure what I do believe, but this does not feel right to me.  I think a win against a lesser player should matter less than a win against a player who is twice as good, but still worse than you.  That is:  2000 beating 1000 should matter more than 2000 beating 500. 

Or in a more extreme scenario:  2000 (@ equilibrium) beating 1995 (@ equilibrium) 20 times is, statistically, MUCH more significant than him beating 250 (@equilibrium) 20 times.

I think a win against a lesser player should matter less than a win against a player who is twice as good, but still worse than you.  That is:  2000 beating 1000 should matter more than 2000 beating 500.

Or in a more extreme scenario:  2000 (@ equilibrium) beating 1995 (@ equilibrium) 20 times is, statistically, MUCH more significant than him beating 250 (@equilibrium) 20 times.

Yeah, I agree with that.     I guess I wasn't clear.  I meant that your expected win ratio against two  players with different scores should perfectly balance so that if you played a 100 games against both it would have about the same affect on your score.  Obviously you'd win more games against the worse player, but because their score is lower it should affect your score the same as winning less games against the better player.

I think that is the root of the problem.  Right now, playing 100 games against a 1000  player is better for your score than playing 100 games against a 2000 player (<-assumption).

I understand now what you were suggesting, but I'm not sure I agree with that assumption.  We'd have to call Hugh back in here to get him to math it out for us.

I will say, though, that I don't personally feel that the line you're talking about (combining likelihood of winning and points earned) needs to be uniform across the spectrum.  Also, I'd be one to say that if your assumption does happen to be true, I think it's actually a compliment to the current system.  Squint's complaint was "top tier players don't want to play against middle of the road players, because they fear losing their points."  If it turns out that you're right, then that's the argument you need to show them to say "play 100 games against those middle of the road players because it'll be BETTER for your score in the long run."

Let's say a "fair match" is one in which the expected rating change is zero. One player will gain, and another will lose, but the expectation is zero. (In the 2000 versus 1000 case, the match is fair if the 2000 player has an 80% chance of winning, and the 1000 player a 20% chance of winning.) Not all matchups will be fair in this sense, but a good system will, over time, produce ratings that regularly have close-to-fair matches. 

This system, Elo, and Glicko all have an interesting mathematical feature in common. Suppose the matches are fair, so that we know the edge based on ratings alone. If player A has an x-to-1 edge over player B, and player B has a y-to-1 edge over player C, then the ratings predict that player A has an x*y-to-1 edge over player C. For example, suppose tom has a 3-to-1 edge over Hugh, who has a 2-to-1 edge over Bob. Then tom has a 6-to-1 edge over Bob. I don't know of a rating system that doesn't make this assumption, but that doesn't mean they don't exist, or that they can't be engineered.

Anyway, if seemingly reasonable assumptions about skill are true, including skill in games of luck, this "edge-multiplying" property should hold. What should happen in SG's "maximum H-Rating of 80%" situation is that mediocre players will not achieve this maximum level against the worst players. And furthermore, the perfect player can't achieve the maximum H-Rating against mediocre players. They can only achieve it versus the worst players. 

However, if it happens that the perfect player has an H Rating of 80% against mediocre players who have an 80% H Rating against terrible players, and the perfect player has only an 80% edge against the terrible players, then yes, this and every other system will produce many unfair matches.

And just to guess: What I think is actually happening is that because we don't assign reliability to our ratings, a lot of wins occur against heavily unskilled 1000 rated players whose 1000 rating is really unreliable. The difference between the 500 rated and the 1000 rated player is that the 500 rated player stuck around and continued to play a lot.

So is it fair to sum up: "The ratings are unreliable and skill is determined by who wins the game?"

Boris - I got Squint's complaint backwards.  Thanks for pointing that out.

Hugh - your commentary is always interesting.

Hugh - your commentary is always confusing.

I fixed it for you....

(kidding of course, thank you Hugh for the explanation)
But I don't think you have a problem with our system so far, because we haven't seen statistics of PerfectPlayer having 80% HRating against Medicore, and MediocrePlayer 80% against TerriblePlayer.  So it doesn't look to me that we've indicated that the current system is failing in this fashion, no?

BorisTheFrugal wrote:

But I don't think you have a problem with our system so far, because we haven't seen statistics of PerfectPlayer having 80% HRating against Medicore, and MediocrePlayer 80% against TerriblePlayer.  So it doesn't look to me that we've indicated that the current system is failing in this fashion, no?

Right - I don't know that it is a problem, but I also don't know that it isn't. And, I'm not sure that it'd be easy to determine given the won/lost data!

But, if this happens, SG has a legitimate concern, though I wouldn't be sold on hard caps as a solution.

I offer as evidence the Wargear Warfare stats.  If you look at stats for all players who have played more than 50 games, the highest H rating is 75%.  To me this is a strong indication the luck factors for this board (cards, dice, placement, and opponent choices) prevent anyone from going above an 80% H rating in the long run.  I think this is the most popular board so it has been played enough so that the statistics are meaningful.

Of course, all boards are different, so 80% is not a good upper limit to use for all boards, it is just an example I have been using to move the point along.  There are certainly boards with less of an influence from luck where the H ratings can be higher over the long run.  This is why I suggested some solution based on the H ratings of each board.

SquintGnome wrote:

I offer as evidence the Wargear Warfare stats.  If you look at stats for all players who have played more than 50 games, the highest H rating is 75%.  To me this is a strong indication the luck factors for this board (cards, dice, placement, and opponent choices) prevent anyone from going above an 80% H rating in the long run.  I think this is the most popular board so it has been played enough so that the statistics are meaningful.

By itself, a bound on overall H-Ratings is not evidence of the same bound on H-Ratings against all players. It is likely that a person with an overall 75% H-Rating has a better H-Rating than 75% against the population of low rated players.

Consider a game with 3 skill levels. The good players have a 70% H-Rating against mediocre players and a 84% H-Rating against the bad players. The mediocre players have a 70% H-Rating against the bad players, and a 30% H-rating against the good players. Against similarly skilled players, the H-Rating is 50%.

In spite of the bound on H-Rating, our system would be extremely fair in that situation. The good players would be at about 1500, the mediocre at about 1000, and the bad players at 667. No one would prefer to play any particular type of player in this situation because the expected rating outcome would be 0. (Or close to it - some rounding occurred in my calculations!)

What we'd really like to know is how players do against different skill levels. A bound on H-Rating says very little about fairness - the system can still be perfectly fair in such a situation.

Agreed

@ Hugh: Doesn't the overall H-rating describe the H-rating against different skill levels since it is a composite of each player in the game?  Each game has it's different skill-level players so isn't that the same thing?

Or maybe this is the distinction between winning a 4-player game of 2000+ ranked players vs. a 4-player game of 1000+ ranked players?

AttilaTheHun wrote:

@ Hugh: Doesn't the overall H-rating describe the H-rating against different skill levels since it is a composite of each player in the game?  Each game has it's different skill-level players so isn't that the same thing?

Or maybe this is the distinction between winning a 4-player game of 2000+ ranked players vs. a 4-player game of 1000+ ranked players?

H-Rating is more like a win percentage, adjusted for game size variations. A win in a 4-player game against 2000's has the same effect on H-Rating as does a win in a 4-player game against 1000's. However, it is better for score/ranking to win against 2000's.

So, if you see someone with a 75% H-Rating, it might be that they have a 50% H-Rating in games with 2000's and a 90% H-Rating  in games with 1000's, but played more 1000's than 2000's. Or, it could be that they had a 75% H-Rating against opponents of all skill levels. One can't tell from the number alone.

A bound on H-Rating says very little about fairness - the system can still be perfectly fair in such a situation.

This is the phrase I've been fumbling to find for a half dozen posts.  From now on, I defer to Hugh to express my positions for me.

ATH: I didn't think the H-Rating doesn't take into account the ratings of the opponents.
I thought that is a composite of your win percentage with respect to the number of players in the game (independent of their ranking).
Translation - Statistically speaking, the perfectly average player should win about 1 out of every 4 4-player games, 1 of 3 3-players, 1 of 9 9-players, etc
So if you've played 40 4-player games, and you've won 18 of them, then theoretically, you're a better than average player.
And if you've played 60 6-player games, and only won 7 of them, then theoretically, you're a worse than average player.
Combine all of those win percentages (relative to the number of seats that were available in the game), and you get the H-Rating.

Now, if I'm correct:  In the former case, you might have only played much worse (lower ranked) players, and in the latter, you might have been only playing much better (higher ranked) players, which would skew the results.
But H-Rating ignores this fact and assumes you've played a wide spectrum of players, so the high and low ranking opponents balance each other out.

I think it is fairly preposterous that I am the 2nd best Risk player in the world (here). But maybe I really am that good?

soft wizard wrote:

I think it is fairly preposterous that I am the 2nd best Risk player in the world (here). But maybe I really am that good?

SW, if you can maintain that 88% H-Rating, no doubt you will soon be the #1 Globally Ranked Player in the world (of WarGear).  You are precisely the type of player we are discussing here.

Assuming it is not likely that a player can maintain such a rating, perhaps the system should make it such that a top player cannot reach equilibrium until they have played x games (say 75).  Right now SW is at 2094 after 25 games.  Currently the best H-rating out there at 75+ games is poloquebec, with a 75% H-Rating after playing 110 game.  His Global Ranking at this juncture is similar to SW's, weighing in at 2068.

 In the past I have noted (complained about) the volatility of the Global points system, using myself as a case in point (Quick review -- 1000 >UP> 2100 >DOWN> 800 >UP> 2300), it seems to me that the system needs some brakes..