I was looking into it a bit, and I now see this TrueSkill implementation on github with a BSD license:

https://github.com/sublee/trueskill

It looks simple to use & well documented:

http://trueskill.org/

Looks to me like you can use that code, you just can't use the trademark TrueSkill(TM).   I started playing around with it and could fairly easily mock up a web interface so everyone could experiment with it.  If I don't run into too much trouble I might have it done this weekend.

Am I correct in understanding that to be able to update Trueskill scores after a multiplayer game, it's required as an input to the ranking function that the players are ranked in order? So we would need a notional 1st, 2nd, 3rd place etc?

This will be tricky to implement as older games in the database don't have the eliminated by field set so the game logs would need to be parsed to establish a finishing order. It's not impossible but it's going to be a pain to do.

tom wrote:

Am I correct in understanding that to be able to update Trueskill scores after a multiplayer game, it's required as an input to the ranking function that the players are ranked in order? So we would need a notional 1st, 2nd, 3rd place etc?

This will be tricky to implement as older games in the database don't have the eliminated by field set so the game logs would need to be parsed to establish a finishing order. It's not impossible but it's going to be a pain to do.

If finishing order is part of the trueskill algorithm (assuming that lasting longer in a game is rewarded), then I question that it's what we want 'in its current format.' 

Is there a way to bypass or modify TS such that finishing order is not a consideration?

As a kid back in the 1980s, I developed a ranking system for the games i played with my friends that awarded points based on ranking order (order of elimination). I think this would be a fairer system, but I understand that it might be too much work if we have the option to bypass finishing order

Chele Nica wrote:

As a kid back in the 1980s, I developed a ranking system for the games i played with my friends that awarded points based on ranking order (order of elimination). I think this would be a fairer system, but I understand that it might be too much work if we have the option to bypass finishing order

As a kid back in the 'early' 1970's I played to win. Elimination order only impacted who got bored first. :P

But seriously. I submit that rewarding by elimination order would be a big mistake.  It would change the way people play the game, not to mention it would have no relevance with boards where the winner is determined without any eliminations (like Renaissance Wars).  If elimination order can't be extracted from TrueSkill then it is a bust.

tom wrote:

Am I correct in understanding that to be able to update Trueskill scores after a multiplayer game, it's required as an input to the ranking function that the players are ranked in order? So we would need a notional 1st, 2nd, 3rd place etc?

This will be tricky to implement as older games in the database don't have the eliminated by field set so the game logs would need to be parsed to establish a finishing order. It's not impossible but it's going to be a pain to do.

Might it be possible for all non-winners to be tied for 2nd?

or tied for n-th place, where n is the number of players in that game...

weathertop wrote:

or tied for n-th place, where n is the number of players in that game...

..or the median of the n.  As long as ties are allowed, something should work.

This is slightly OT because it's half about the current CP discussion, but..

Because TrueSkill brings players up to equilibrium much quicker (with fewer games), most any of the proposed CP systems, including ICPs will even more heavily reward diverse play because any good player who's in the hunt for points will realize they can amass points very quickly by simply playing a lot of boards. 

Someone correct me if I'm wrong, but if TS is anything like H-Ratings, I suspect that another artifact of the TS system will be that the top scores for less popular but reasonably well played boards like Australian Risk or War of the Roses will be right in line with the top scores for the most popular boards like WGWF and Colossal Crusade.

If all the above is true, then the Global system used to determine the top players will probably have to be designed from the ground up. The top-ranked site player should not be the one who plays more boards than anyone else and gets to add up more points, but nor should it be the player who specializes, achieving the top spot from having the highest mean score across boards.  In fact, it's possible that we'll all want to go with a straight medal/place system not unlike the one we have now.

I 'think' I still want TrueSkill, but if it becomes a reality, there can be no doubt it'll  throw one big old honkin' wrench in the Global Points Machine.  Remember, this is not one video game. Not all games are equal on this site, and CPs may not be summative for reasons I mentioned in the last paragraph, so depending on how the composite is derived, there may be even less incentive to play boards where one is uncomfortable.

With Trueskill as a foundation, the current methods used to determine site-wide metrics will be thrown on their heels. It could be a hornet's nest, but it might very well be the best thing since factories for the site, as newbies quickly rise in the ranks with seeming effortlessness, and toppling many a player who got there via the 'hard work' that the old system demanded.

Eyes wide open..

tom wrote:

Am I correct in understanding that to be able to update Trueskill scores after a multiplayer game, it's required as an input to the ranking function that the players are ranked in order? So we would need a notional 1st, 2nd, 3rd place etc?

This will be tricky to implement as older games in the database don't have the eliminated by field set so the game logs would need to be parsed to establish a finishing order. It's not impossible but it's going to be a pain to do.

I think it's like M57 said, you'd just want all the "non-winners"  (i.e. losers) to be  tied for 2nd place.  (or possibly tied for N/2 place or something.) 

I'd be happy to play around with the library and post some results.  Then I'd know more.  Will probably have some time this weekend.

M57 wrote:

This is slightly OT because it's half about the current CP discussion, but..

Because TrueSkill brings players up to equilibrium much quicker (with fewer games), most any of the proposed CP systems, including ICPs will even more heavily reward diverse play because any good player who's in the hunt for points will realize they can amass points very quickly by simply playing a lot of boards. 

Someone correct me if I'm wrong, but if TS is anything like H-Ratings, I suspect that another artifact of the TS system will be that the top scores for less popular but reasonably well played boards like Australian Risk or War of the Roses will be right in line with the top scores for the most popular boards like WGWF and Colossal Crusade.

If all the above is true, then the Global system used to determine the top players will probably have to be designed from the ground up. The top-ranked site player should not be the one who plays more boards than anyone else and gets to add up more points, but nor should it be the player who specializes, achieving the top spot from having the highest mean score across boards.  In fact, it's possible that we'll all want to go with a straight medal/place system not unlike the one we have now.

I 'think' I still want TrueSkill, but if it becomes a reality, there can be no doubt it'll  throw one big old honkin' wrench in the Global Points Machine.  Remember, this is not one video game. Not all games are equal on this site, and CPs may not be summative for reasons I mentioned in the last paragraph, so depending on how the composite is derived, there may be even less incentive to play boards where one is uncomfortable.

With Trueskill as a foundation, the current methods used to determine site-wide metrics will be thrown on their heels. It could be a hornet's nest, but it might very well be the best thing since factories for the site, as newbies quickly rise in the ranks with seeming effortlessness, and toppling many a player who got there via the 'hard work' that the old system demanded.

Eyes wide open..

Completely true.  TS would bring a lot of changes to the way we rank and think about rankings.   For example, there is not currently a maximum GR on any board.  In theory if someone dominated completely, their GR could rise to infinity.  And IMO, GR has kind of an inflationary aspect to it, where newbs join, give up some GR to the experienced players, and then never come back, so new GR points kind of keep getting added to the pool of active players. 

I don't think TS is like that.  All TS scores will be bound by some maximum and minimum value (which I believe we would have control over).  And I think there will be a more true equilibrium that as you said will be reached quicker.  Due to the inflationary nature of GR, I think equilibriums slowly rise over time.  If we change to TS it will have profound impact on our discussions of CPI, etc. 

I think I want TS also, because I think it has stronger mathematical foundation.  GR is kind of "hand wavy".  But it would affect a lot of things, and really what we want is whatever is the "most fun".  And if TS isn't more fun, maybe it's not the right choice.  I also think I have a bias toward GR because I understand it better right now & it's familiar, but I'm trying not to let that influence me.

Mainly I was excited to see that TS might be an option and wanted to bring it up for discussion, even though I'm still figuring it out.  Can't wait to get home and play with it some more, although I'm not sure if I'll have time tonight to get it working.

Ozyman wrote:

Can't wait to get home and play with it some more, although I'm not sure if I'll have time tonight to get it working.

It would be cool if Tom could send you the data for one board, like Invention ..or something with less data like Roses, or a dueling board like IWO or Go-Geared, and let you run with it so we could compare against the existing ratings for those boards.

Ozyman wrote:

tom wrote:

Am I correct in understanding that to be able to update Trueskill scores after a multiplayer game, it's required as an input to the ranking function that the players are ranked in order? So we would need a notional 1st, 2nd, 3rd place etc?

This will be tricky to implement as older games in the database don't have the eliminated by field set so the game logs would need to be parsed to establish a finishing order. It's not impossible but it's going to be a pain to do.

I think it's like M57 said, you'd just want all the "non-winners"  (i.e. losers) to be  tied for 2nd place.  (or possibly tied for N/2 place or something.) 

I'd be happy to play around with the library and post some results.  Then I'd know more.  Will probably have some time this weekend.

As I mentioned before the official TrueSkill page that was linked to in another thread said that ties were allowed. So one of those two options for the losers tying seems ideal (allowing non-win eliminations to add to your score would change the strategy of the games considerably (although this might be a fun per-board option)).

I am not familiar with how rankings are derived in TrueSkill, but it seems like tied for 2nd place would be better. Wouldn't the N/2 penalize losers of larger games?

>I am not familiar with how rankings are derived in TrueSkill, but it seems like tied for 2nd place would be better. Wouldn't the N/2 penalize losers of larger games?

I'm not familiar enough to say either.  It could be that whether you say the losers tied for 2nd place or N/2 place doesn't make any difference.  I figured I'd try each out, and report back with results.

I got the library installed and ran a simple test script.  I created 3 players, and had player1 win a few, then player2 wins a few...

INITIAL SETUP:
r1:  trueskill.Rating(mu=1000.000, sigma=333.000)
r2:  trueskill.Rating(mu=1000.000, sigma=333.000)
r3:  trueskill.Rating(mu=1000.000, sigma=333.000)
R1 Wins
r1:  trueskill.Rating(mu=1204.263, sigma=269.056)
r2:  trueskill.Rating(mu=897.761, sigma=238.505)
r3:  trueskill.Rating(mu=897.975, sigma=238.589)
R1 Wins
r1:  trueskill.Rating(mu=1280.365, sigma=236.399)
r2:  trueskill.Rating(mu=867.874, sigma=187.635)
r3:  trueskill.Rating(mu=868.039, sigma=187.713)
R1 Wins
r1:  trueskill.Rating(mu=1320.284, sigma=216.485)
r2:  trueskill.Rating(mu=855.313, sigma=156.404)
r3:  trueskill.Rating(mu=855.439, sigma=156.470)
R1 Wins
r1:  trueskill.Rating(mu=1345.384, sigma=202.850)
r2:  trueskill.Rating(mu=848.771, sigma=135.428)
r3:  trueskill.Rating(mu=848.872, sigma=135.483)
R2 Wins
r1:  trueskill.Rating(mu=1081.963, sigma=153.316)
r2:  trueskill.Rating(mu=943.692, sigma=121.531)
r3:  trueskill.Rating(mu=871.414, sigma=118.946)
R2 Wins
r1:  trueskill.Rating(mu=990.075, sigma=128.081)
r2:  trueskill.Rating(mu=997.579, sigma=111.854)
r3:  trueskill.Rating(mu=875.115, sigma=106.665)
R2 Wins
r1:  trueskill.Rating(mu=945.407, sigma=112.178)
r2:  trueskill.Rating(mu=1033.150, sigma=104.743)
r3:  trueskill.Rating(mu=873.752, sigma=97.315)

I'll try to get something setup so the rest of you can play with this, but might take me a bit.

Something to think about - TS requires certain constants be set that describe the competition in a game .

I just set the initial score to 1000, and then used the defaults for the rest:

    MU = 1000
    SIGMA = MU / 3
    BETA = SIGMA / 2
    TAU = SIGMA / 100

But if you are interested there is a brief overview here:

http://trueskill.org/#trueskill.TrueSkill

And details here:

http://www.moserware.com/assets/computing-your-skill/The%20Math%20Behind%20TrueSkill.pdf

I haven't looked at the details link yet.

Awesome!! Can this be set up on a spread sheet with something like 10 players? ..or is it all code and has to be run in a dedicated program?

Looking at the numbers above is fascinating.. after winning just 2 games in a row, #2 is rated higher.  Also, #3 seems to have hit a temporary equilibrium (possibly because confidence in #1 is wavering?).  What's disconcerting is that after 5 games, no one has a rating above 1000.  This is ok if the site only has one board because everything is relative, but..??

I'd be curious to see if the numbers spread out more if you add a fourth player and input the same results.. 1,1,1,2,2,2.  

Some good points about impact on CP etc - it could be parallel run so we can model the impact on ranking tables, CP etc.

If it helps Ozy I can pass on some ranking data - as the dataset can get pretty large this could be a subset of a more popular board or the entirety of a less popular board's games.

It seems like ranking finishing order might not be be a practical option, but i want to spell out why I think it would be a superior ranking system:

1. Players currently finishing 2nd or 3rd place are penalized equaly than people finishing last. In a 10 or 15 player game, it is pretty significant to finish 2nd place, it shows skill and strategy.

2. Currently, players who had a bad start often end up forfeiting or taking a lot of time to take their turns. If there was an incentive to finish 3rd vs 4th place, these players might continue playing actively and maybe even make a turnaround.

3. On the negative side, I do recognize it could lead to more truces and negotiating, as number 1 would not be the only thing to vie for.

4. On the other hand, you'd have to have a more careful strategy, as players would be less likely to just give up or take revenge just for the sake of it.

5. I think it would enxourage a better spirit in the site. Currently the winner of a tourney or a game is a winner, everyone else is a Loser. Having more "winners" would likely reduce negativity and have more people feeling like they're having fun and not just slupping in the ranks

I know all this is probably moot, but I wanted to state my case as we touched on this issue.

Chele Nica wrote:

It seems like ranking finishing order might not be be a practical option, but i want to spell out why I think it would be a superior ranking system:

1. Players currently finishing 2nd or 3rd place are penalized equaly than people finishing last. In a 10 or 15 player game, it is pretty significant to finish 2nd place, it shows skill and strategy.

2. Currently, players who had a bad start often end up forfeiting or taking a lot of time to take their turns. If there was an incentive to finish 3rd vs 4th place, these players might continue playing actively and maybe even make a turnaround.

3. On the negative side, I do recognize it could lead to more truces and negotiating, as number 1 would not be the only thing to vie for.

4. On the other hand, you'd have to have a more careful strategy, as players would be less likely to just give up or take revenge just for the sake of it.

5. I think it would enxourage a better spirit in the site. Currently the winner of a tourney or a game is a winner, everyone else is a Loser. Having more "winners" would likely reduce negativity and have more people feeling like they're having fun and not just slupping in the ranks

I know all this is probably moot, but I wanted to state my case as we touched on this issue.

Your points are well taken, but as you have clearly outlined above, for better and worse, it would change the way people play the game. So we have to ask ourselves, is that what we really want?

Not mentioned in your list is the strategy of targeting top players first because more points are to be gained by making them the 'bottom' losers. That would be my first perfectly 'legal' tactic if I was playing in a system where order of elimination counted, an of course with better players, this 'correct strategy' would undoubtedly create an environment where no player could ever hold on to the #1 spot - they would be too easy to unseat.

Another example. If I was in a position to run the board, and got to a point where victory was imminent, I might go out of my way to eliminate players in an order that benefits me the most - again, most likely by eliminating the best players first.

My vote: When it comes to war, it doesn't matter 'when' you die. This isn't golf. The scoring/ranking system needs to reflect that paradigm.

I don't know that it would significantly change the game, but it would give people who are not winning more of a reason to play well till the end. Is that worse than having despondent players give up or prolong the game? 

As for the other two points:

I don't usually try to attack top players early in the game, as that is a good way to get eliminated early on, or at least significantly weakening yourself from retaliatory attacks. And I remember reading in these forums that some people attempt to use this strategy in the current set-up because they want to avoid that person from getting the winner points. 

I've had a few multi-player victories on this site, and many u=in real life, and I don't remember having a choice between players to eliminate. On high card games you can eliminate someone and then seek to eliminate another and so on, but it only works if you do it just right, at least in my experience you can't pick and choose who you want to eliminate if you suddenly get a boost of armies, you got go for whoever's weakest at that point. 

When it comes to real war, it doesn't matter when you die, but when it comes to make-believe world domination, it matters how good you are relative to other players, because you "live" to play another game tomorrow.