User Tools

Site Tools


aggregate_stat

Hugh said:

“Funny discussion to me. All of these big stats we have, whether CP or GR or ranking on an individual board, are effort-based stats. Skill plays a role, but a minor one.

And GR depends on the choice of game you play as MB/Smoke indicated. A “well-rounded” aggregate stat will suffer the same fate. But, it doesn't matter! It's a goofy/fun accomplishment system and there is no claim about “true skill” being made when someone wins an award like that. It would be a fun award for some people, so why not develop it? I'm all for it!”

Arguments:

General Arguments about Using an Aggregate Stat

Summary: An aggregate stat could reflect the sum total of a players abilities and give a better “whole picture” of the player. However, it is combining stats that are totally different – and unrelated, and would necessarily create different values for the stats. This may or may not become the most sought after stat on the site.

  • The aggregate may not even matter to players outside the top 30 positions
  • New players are not necessarily rewarded for being good a WGWF (which most players are playing) due to rank being based on CP
  • This collection of skills may overshadow the importance of CP or GR as a score
  • Your rank will basically determine which score is most important in many players minds -1
  • Players should be rewarded for plying their efforts, no matter where
  • Highlighting everything also highlights nothing
  • Premium members can play more games and get more points in almost all cases
  • The more you play, the more likely you are to win – with all current stats
  • The aggregate should include as much player information as the site captures +1, -1
  • a well constructed, all inclusive aggregate would not discourage the new arrival to WarGear from feeling that they can't get a promotion for just being a good player alone – where cp rankings do that
  • An aggregate score is not creating any added value
  • The added value of aggregate is a whole picture of the players stats
  • And aggregate should encourage diversity of play styles
  • The aggregate will not diminish the already existing stats - if their important to an individual they will stay that way
  • One board masters will disappear into the shuffle of an aggregate
  • Players may be discouraged by having to jump through even more hoops to improve their scores
  • a multi-track system of achievement (like we have now) recognizes individual accomplishments (like those people who are brilliant at tourneys or a single map), but holds CP as the most important by recognition because it represents ability in diverse environments is better than some aggregate apples + oranges score where individuality and difference will all get lost in the mix
  • • And aggregate should be the most difficult stat to rank highest in

Arguments about the purpose of an Aggregate

Summary: The purpose of the aggregate stat is to give players another score to strive for (“bragging rights”) and to try to create a meaningful comparison system of either all of the players stats, or to just amalgamate their individual play stats (CP and GR).

  • An aggregate should reflect a collection of a players individual skills
  • it's just not self-evident that a scoring system that rewards board specialists on a hot streak (whether individual play, teams, or tournaments) is as meaningful as one that takes a variety of playing skills
  • It will reflect what abilities/performance people most value?
  • “is about rewarding the complete player” vs. the aggregate should measure individual play
  • Rewarding completeness is a measure of individual play +1
  • We already have stats for individual skills - but nothing to reflect the complete player
  • The achievement system accomplishes getting players to diversify - like an aggregate would
  • And aggregate would be stronger than the achievement system
  • I don't think it should be the end-all goal of the site to get people to play as many boards as possible. To the degree this is true, there's already an incentive in place. It's called Championship Points, and they are ostensibly WarGear's most coveted. So that's not what an aggregate should be about. I think there are a bunch of dud boards out there – Heck, I've made a few, and if I'm in it for CPs, then I'm going to feel compelled to play some of the less popular boards (i.e., the duds) because they are probably easier to achieve CPs on. My +1 is to let the CP chasers do that, but don't unduly weight CPs in the aggregate

About the construction of the aggregate score

Summary: There are two big topics here. The inclusion of team, tournament, trophy and H-rating stats in an aggregate, and how much each of the included stats should be weighted. The problems are centered around: what is most important to encourage in players on the site, how easy the stat will be to game, and whether or not team inclusion still reflects an individuals’ skills levels. It has also become important to ensure the stat cannot be dominated by one particular skill, but still rewards someone for having that skill.

  • Weighting the scores and combining is easy. Deciding on the “overall difficulty of achieving the top rank for each subset” is not.
  • The aggregate should be an individual composite - not including team or tourney
  • Aggregate could include tournament wins
  • I define individual play as play that is not influenced by team members - for better or worse. So any team games, including tournament team games, are not 'individual play'
  • Equal weighting of an aggregate makes some games into “chores” or “homework” (like team tourneys)
  • if aggregate has 4 categories: CP, GR, H-score, and Tournament Ranking: It is possible to gain a very high rating in 3 out of the 4 categories (GR, Tournament Ranking and H-score) without playing a lot of different boards. I would think the weight of CPs should be higher than the other categories just because you have to play many boards to get somewhere on that stat (as opposed to specializing in a few boards)
  • No one rank should influence an aggregate such that the aggregate rank mimics that rank, hence, a “lions share” can not be given to any of the aggregate components. If the aggregate is done appropriately it should reward players with all around prowess in all areas of the game and some players who are not even top 20 in either CP or GR should be able to sneak into the top 20 of the aggregate based on having a decent CP and/or GR coupled with their strong team, tourney, team-tourney, roving h-ranking etc. Because of its inclusiveness an aggregate should be the most difficult of all ranks to attain as one could not just focus on any one thing to top it. +2
  • someone who only plays WGWF shouldn't be able to take a top spot in the cumulative rankings, so you can't rank all the various GR at the same level as CP
  • I play ALL boards in an attempt to collect cp points. I lose global ranking because most boards have a learning curb. Cp points give me something to shoot for. Without them being what they are I see less variety being played. I have points on 73 different boards……variety. I am encouraged to play all types not just two or three boards like most players. Every board is equally important is it not? We want map makers to push the boundaries. I could play 16 16 player games, on the same map, win twelve, and what…..be the greatest? Sounds weak……cp points should be worth more than global….if it is given a higher percentage in the aggregate then I'll give my surf bum approval. +2
  • Your CP score IS the score of a well-rounded player. It pushed me to start playing more of a variety of maps. And, it forced me to learn new skills and styles because every time you play a new map you need to overcome the learning curve to start winning. And, if there are others like this pushes variety, which raises the overall quality of players and which increases the number of maps played, which encourages designers, which… cp should be the lions share.
  • I don't think the point of an aggregate is even about CP or GR and which is more “coveted”, or “better”, or whatever. If people want CP they'll play for them and if they want GR, then that. An aggregate has a whole different point to it though. It should say that if you want to be at the top for this ranking then you must play all boards (CP), do that well (GR), succeed in tournaments (tourney) AND know how to succeed with teammates (team), if you want to top this rank… That is why it is an aggregate
  • making any rank too low down the list is a deaths blow to one's aggregate ranking even if their other ranks are quite good.
  • The only reason CP are WarGear's most coveted right now is because they are used to determine your rank. If you were to average CP, GR, Team-GR and Tournament-GR, than all 3 of the GR could be based on a single board, and diversity of play would only count for 25% of your cumulative rank.

About Inclusion of Team Games:

Summary: The debate is centered on whether or not an individual’s skill is reflected by team play, or if it is a different and desirable skillset. There is also some worry about “super teams” being exclusive in order to get points, but the point system works with diminishing returns, so that team would be forced to diversify eventually.

  • Team games don't necessarily reflect an individuals skills (due to random nature of teammate)
  • However team play is a skill in and of itself, included in the players skillset
  • And the individual skill set is transferable to team play
  • Team play is based on individual skills and communication
  • It creates a synergy, by encouraging mentorship and cross team learning
  • If Team scores are included in the aggregate, a culture is created where players only play with certain players, and more specifically, less skilled players would rarely, if ever, get to play with more skilled players
  • Team ranking is perhaps the most game-able of the stats
  • regarding randomizing teams: In theory, this is unnecessary because the super-team gets to the point of diminishing returns. The great player would have incentive to team up with lesser players because more points are gained in a win and fewer points are lost in a loss.
  • I think team ranking should be worth half value…………… If my teammate has poor Internet access or has a ruptured appendix I could lose out. My appendix ruptured and a few of my turns got skipped…..cost Amidon37 a trophy

About including Tournament stats

Summary: Tournament stat inclusion is argued for by the aggregate being an inclusion of all stats, and against by Tournaments being a coveted practice ground, and being a narrow subset of all games.

  • if Team Ranking is included, I would say then Tournament Team Ranking should be included as well
  • Tournament games are essentially a subset of all possible games - Tournament games with 5+ players are extremely rare, and thus it is very difficult to achieve a high ranking unless you exclusively play one board well or dominate 1 vs 1 games or something like that. I'm not convinced a subset of games should get the same ranking as something that is much bigger.
  • Encouraging players to excel in tournaments dampens the learning ground that is tourney play
  • team and tourney are subsidiary to standard in the aggregate and therefore everyone ultimately controls their own destiny on it by being solid standard players first and then tourney and team are secondary achievements to augment and fill out their aggregate score as they compete more completely by participating more fully on the site to build up their aggregate in all it's component areas.
  • Such a division is inbuilt into the aggregate by weighting the aggregate to be 50/25/25 or 60/20/20 Standard/tourney/team. In effect, team is 1/2 or 1/3 weighted compared to standard depending on what set of components (GR/CP/tourney/team) or (GR/CP/roving-h/tourney/team) that the majority here agree upon

Argument for h rating inclusion:

Summary: H-rating inclusion is argued for by it being a stable and decent indicator, and should be included for completeness, which would balance out the other more volatile stats. It is argued against because of it’s complexity and it measuring a completely different thing than the other stats, in a different way – so how to include it?

  • It’s more stable and balances out the volatile nature of the other scores
  • If a player wins a bunch of games initially it will boost them onto the charts
  • I'd vote to keep it simple and keep H-Rating out, at least at first
  • H-rating is a more difficult stat to understand initially than the others

Moving forward action:

First, put forth a vote/tally on what scores should be included in the composite, If more than two options are offered then I suggest a voting system where the lowest vote getting option is dropped and another vote is taken on the remaining options - rinse and repeat. Second, based on the winner from above. I/we can present formulaic options for the composite which will then be voted upon.

Proposed Systems for the Aggregate:

Summary: The proposed systems all fall into one of the following categories: - No aggregate - Combination of GR and CP – with equal weighting - Combination of all stats with equal weighting – based either on their normalized scores, or their ranking positions - Combination of all stats with simple ratios for weighting - Some mathy ones I don’t get, because I do people, not math. - Alternatives

None:

No aggregate at all - combining stats is like bunching apples and oranges or fruit salad

Equal weight CP and GR:

  • CP and GR because they are a reflection of individual skills

Combination of all stats – equal weights

  • CP, GR, Team and Tourney (and their associated permutations) and the roving H rating if it comes into being.
  • Basing it on an average of the rank position
  • Each rating is listed as a ratio of the highest rank in that category, then the four ratios are added and scaled to 100
  • Player A = (ind.GR/highest.GR + ind.team/highest.team + ind.tourn/highest.tourn + ind.CP/highest.cp)/4 = .76 ←- rank is based on that (this was ozys suggestion and he -1'd it)
  • I think if you want to combine everything, then GR, Team-GR and Tournament-GR should be averaged to get a 'cumulative GR' and then that should be averaged with CP.
  • For each category set the maximum score at that time to 100% and then scale everyone else according to that and add both numbers up. Maximum score is 200, if you want you could rescale it so the maximum is 100. Alternatively, give everyone a rank for each category and add them up. Highest score is a rank of 1, etc. Arrange from lowest to highest. Both of these plans avoid trying to create some formula to account for the discrepancies between the two systems. Simply, if you are the best at each, you are the best overall.
  • Aggregate should equally weight everything +1
  • CP x GR x HR / 100
  • We could just add up the ranks from each category and the guy with the lowest score is number 1!

Combination of all stats with weighting

  • An aggregate might be weighted based on the top rank of each subset setting its weight percent +1
  • Since CP gives weight to each board based on how high the scores get on it, the GR portions of an aggregate could be weighted based on how high the top scores are as well. -(For instance if CP is given a guaranteed x% of the aggregate then the rest is 100-x=y% At the moment of writing this the top GR is 3424, tourney is 2809 and team is 2364. As of such GR would be 39.8% of y%, tourney would be 32.67% of y% and team would be 27.5% of y%. The more people participate in any area and drive the top score up in it, the more weight it attains. - Ideally then the weighting would then be applied to the rank position of the player.)
  • an aggregate weighting system can be devised to keep some of the CP/GR diehards happy
  • 50/25/25 standard/tournament/team - implicit in this approach is equal weighting by component, and no weighting by number of games played. Doesn't matter for combining GR and CP, but here we're saying that 1 team game has the same impact as 100 tourney games and 1000 regular public games

Mathy ones

log10 ( CP+2) * GR * HR * TP / 4000

Alternatives

  • Perhaps we could increase the number of games for standard members, but limit number of boards they can play. -2
  • We may be able to use the Trueskill ranking algorithm to tell us actual player skills levels
  • What if there could be an interactive page that computes an aggregate based on percentages provided by the viewer of the page. This way, the only thing to arg.. uh, I mean discuss is what the default settings should be. Additionally, if there were a number of schools of thought that are consistently contrarian, like 'even weighting including GR' vs. 'well-rounded featuring nothing but CP and Team Play CPs', then a handful of presets could be included or added over time so that folks could brag and compare to their heart's content. Perhaps most importantly, for those who wish to actually use the stat for their own devious informational purposes, it would be customizable.
  • === And finally the Alphabet soup solution - As per Amidon37: ===

Here's a long thought on how to “combine rankings” -

We have a Public Global Ranking (currently called GR, but I'll call it PGR) and we have a Tournament Global Ranking (currently called T Score, but I'll call it TGR.) Soon (I expect) we will have a Team Global Ranking - but actually we should have a Public Team Global Ranking and a Tournament Team Global Ranking - call them PTGR and TTGR respectively.

Each of these can be calculated separately with the standard start-with-1000-point-system, but we could also create a Composite Global Ranking (CGR) by (retroactively) starting everyone off with 1000 points here and performing the standard calculation on this at the end of every (public/tournament/team) non-private game.

We also have a Ranking Score for each board that uses the standard start-with-a-1000 -points system. These currently change only for public non-team games and are used to determine Board Championship points. (There are also stats for private games calculated but we won't consider those here) I would like to see all games (public/team/tournament) count in those calculation also, but since that probably won't happen we could also have a PRS (Public Ranking Score), a TRS (Tournament Ranking Score), a PTRS (Public Team Ranking Score) and a TTRS (Tournanment Team Ranking Score) for each board. (Note the PRS and TRS are already calculated, and the others probably will be when Tom does Team Rankings). Then we could also have a Composite Ranking Score CRS that is calculated at the end of every non-private game where everyone again starts with the 1000 points and we go from there.

So, at the end of every game your CGR and CRS will change and either your PGR and PRS, or your TGR and TRS, or your PTGR and PTRS, or your TTGR and TTRS will change depending on the type of game. There are a couple of directions then we could go with the BC points, but I suggest keeping them on just the PRS as they are now, but the graphics that say what boards you are #1 on, and the rankings on the board info page be based on the CRS for that board. +1 from himself

aggregate_stat.txt · Last modified: 2014/02/13 14:49 by ratsy