I find Allegskill absolutely one of the best ranking systems I've seen in any multiplayer game, and it's emphasis on helping a team win is appropriate for us.
The sliding window of the last 1k games does sound good though.
Allegiance Ranking System
Elzam V. Branstein wrote:QUOTE (Elzam V. Branstein @ Feb 21 2012, 01:07 PM) No one cares about rank anyway. If you make a rankless game the newbs won't care that they're not "gaining levels" because there are no "levels" in the game!
Indeed. Admittedly, I find the whore-stats more interesting but still: Ranks are fun.RoyBrown wrote:QUOTE (RoyBrown @ Feb 21 2012, 01:11 PM) Bullcrap. Rank changes, achievements, and micro-badges and so forth are important game tools, it's industry standard.
But I can understand any 20k hour Vet who says "whatever".
Thumbs up!pkk wrote:QUOTE (pkk @ Feb 21 2012, 04:40 AM) Problem
Newbie phase (or "Kick them while they're down"):
Mu goes down rapidly because they loose games (they don't know better). While they're learning the game, their sigma goes down, too. (See A1r(-1) on leaderboard.)
That means, people without looses in the beginning (returning vets, people who play longer than any ranking system, lucky newbies) end up with a higher rank, than people which learned the game the hard way.
Solution
Don't count games, until:
a) People enable ranking system.
b) People reach the "magic number" of games (autoenable ranking system).
c) Enforcement enables ranking system.
A "reset"-button if you wanna start over would also be nice.
The "sliding-window" is a premium idea though, although I find 1000 HOURS somewhat ridiculous:
If you play 10 counted hours a week, which is already a lot for regular people, you would need 2 years to play 1000h!
That's really not going to encourage pretty much anybody, not to mention the more casually oriented newbies.
Looking at actual game-times in the leaderboard, guys like Dorjan, Nobu or Ryujin would still have to play as much as they already have played before they even start hitting a 1000h window. Mr Chaos barely starts to "profit" from it by having his first 4 hours stripped down ... And these are people who probably could care less.
So much more realistic would probably be some value around 200h.
Or perhaps, the sliding window could adjust with the total time played by the specific player, in a linear fashion such as 1/5th of total game-time perhaps:
50 hour Noob -> 10h sliding window
1k hour Voob -> 200h sliding window
3k hour Vet -> 600h sliding window
10k hour God -> 1000h sliding window (capped at 1000h)
Something I don't really understand about Sigma:
Does that mean games after ~100h don't get counted as an even share?
So a game at hour >100 counts significantly less than any game at hour <100?
If so, that is just plain stupid.
Recent performance should impact more than the goddamn painful hours of apprenticehood. (sliding window ftw!)
Last edited by GoodWill on Fri Mar 02, 2012 5:51 pm, edited 1 time in total.
No, sigma is a measurement of how certain the rating system is of your rank.
If you've only played 10 games, the system has only a very fuzzy idea how good you are. I mean, there's tons of other people on the team, so it's very hard to say how much of the win was due to your involvement, and how much it was due to someone else's. But that's also why the system initially fluctuates wildly in the first few games trying to figure out your real skill level: are you a brand new player, or are you a returning vet?
Now if you've played 1,000 games, the system has a lot more information to determine what your skill is, so it has a better idea where you are. Hence, sigma is lower if you've played a lot of games.
Ratings move slower at this level, but they do move. At this point we run on the very reasonable assumption that people don't magically jump 5 ranks overnight. However, if the system notices that teams that you happen to be on win more often than they statistically should, your rank will start rising to reflect this. Sigma may even go up temporarily while this happens. That's why there's not a perfect correlation between sigma and games played.
And one more thing: people have this misconception that we recalculate your rank from a database of every game that's ever been played. That's not true. Your new rank is calculated from your current rank plus the results of the game you just finished playing. If we lost half the database of games tomorrow, your rank would not change one bit. We can't "go back" and erase the contribution of the first few games to your rank.
The rating system we have is mathematical in nature. We didn't just create it ad hoc. We didn't say, okay, let's give people x points for a win, take away y points for a loss, unless the game was stacked by z amount, in which case we adjust by w amount. At the core, we assume people have a certain skill level, and that skill level can measured to a certain confidence level by looking at the outcomes of team games. In order to do this we also need to make some assumptions about how fast skill ratings can fluctuate -- and there's a constant in there, called gamma, which controls this. People never ask about changing gamma. Why? I think it's because they generally have no idea how the system works. It's really hard to take people's input seriously when they ask for things like sliding windows which show they have some serious fundamental misconceptions how ratings are actually measured, despite the fact that all the information is right there in the wiki.
If you've only played 10 games, the system has only a very fuzzy idea how good you are. I mean, there's tons of other people on the team, so it's very hard to say how much of the win was due to your involvement, and how much it was due to someone else's. But that's also why the system initially fluctuates wildly in the first few games trying to figure out your real skill level: are you a brand new player, or are you a returning vet?
Now if you've played 1,000 games, the system has a lot more information to determine what your skill is, so it has a better idea where you are. Hence, sigma is lower if you've played a lot of games.
Ratings move slower at this level, but they do move. At this point we run on the very reasonable assumption that people don't magically jump 5 ranks overnight. However, if the system notices that teams that you happen to be on win more often than they statistically should, your rank will start rising to reflect this. Sigma may even go up temporarily while this happens. That's why there's not a perfect correlation between sigma and games played.
And one more thing: people have this misconception that we recalculate your rank from a database of every game that's ever been played. That's not true. Your new rank is calculated from your current rank plus the results of the game you just finished playing. If we lost half the database of games tomorrow, your rank would not change one bit. We can't "go back" and erase the contribution of the first few games to your rank.
The rating system we have is mathematical in nature. We didn't just create it ad hoc. We didn't say, okay, let's give people x points for a win, take away y points for a loss, unless the game was stacked by z amount, in which case we adjust by w amount. At the core, we assume people have a certain skill level, and that skill level can measured to a certain confidence level by looking at the outcomes of team games. In order to do this we also need to make some assumptions about how fast skill ratings can fluctuate -- and there's a constant in there, called gamma, which controls this. People never ask about changing gamma. Why? I think it's because they generally have no idea how the system works. It's really hard to take people's input seriously when they ask for things like sliding windows which show they have some serious fundamental misconceptions how ratings are actually measured, despite the fact that all the information is right there in the wiki.
Globemaster_III wrote:QUOTE (Globemaster_III @ Jan 11 2018, 11:27 PM) as you know i think very little of cashto, cashto alway a flying low pilot, he alway flying a trainer airplane and he rented
-
Spunkmeyer
- Posts: 2013
- Joined: Fri Jun 27, 2003 7:00 am
- Location: Contact me regarding: CC, Slayer and AllegWiki.
What you are calling gamma is tau (unless I missed something)
And we can re-calculate the ranks - just because the current system doesn't work like that doesn't mean it's not doable. We can even avoid hitting the database every single time by calculating sporadically (there are other ways to optimize this too, but that's most straightforward and good enough)
EDIT: Having said that a continuous sliding window is probably not even necessary with slower convergence and a larger dynamic factor, so this could be a one time operation to trim the earliest "crapshoot" ~50 games where a newbie has literally zero contribution to the game's outcome.
And we can re-calculate the ranks - just because the current system doesn't work like that doesn't mean it's not doable. We can even avoid hitting the database every single time by calculating sporadically (there are other ways to optimize this too, but that's most straightforward and good enough)
EDIT: Having said that a continuous sliding window is probably not even necessary with slower convergence and a larger dynamic factor, so this could be a one time operation to trim the earliest "crapshoot" ~50 games where a newbie has literally zero contribution to the game's outcome.
Last edited by Spunkmeyer on Fri Mar 02, 2012 9:00 pm, edited 1 time in total.
Want bigger games? Log on to play at the official game time: 9pmET/8pmCT/7pmMT/6pmPT every day of the week. Also Saturdays 8pm UTC.
The wiki calls it gamma. You may be looking at another formula where it's called something else, but I'm pretty sure we're talking about the same thing -- the dynamics variable.Spunkmeyer wrote:QUOTE (Spunkmeyer @ Mar 2 2012, 12:50 PM) What you are calling gamma is tau (unless I missed something)
And we can re-calculate the ranks - just because the current system doesn't work like that doesn't mean it's not doable. We can even avoid hitting the database every single time by calculating sporadically (there are other ways to optimize this too, but that's most straightforward and good enough)
EDIT: Having said that a continuous sliding window is probably not even necessary with slower convergence and a larger dynamic factor, so this could be a one time operation to trim the earliest "crapshoot" ~50 games where a newbie has literally zero contribution to the game's outcome.
Globemaster_III wrote:QUOTE (Globemaster_III @ Jan 11 2018, 11:27 PM) as you know i think very little of cashto, cashto alway a flying low pilot, he alway flying a trainer airplane and he rented
Another way to look at it is:
Back when you only had 10 games, each additional game contributed a great deal to your rank. You could say something like, on average, each game contributed 10% to your total rank. Now that you have 1000 games, are each of those first ten games still 10% of your rank? No, they're worth more like 0.1%.
But I'm making a big simplification here. It's not technically accurate to say that your first half of your Allegiance career counts 50% to your rank and the second half contributes the other 50%. I didn't get to be a 16 by playing like a 6 for two years, and then playing like a 26 for two years after that. Actually the system doesn't have "memory" of individual games. It doesn't assign "weights" to any of them.
What it does is take your current rank, and rolls in the results of the last game you played. So you can kind of say that 5% of your rank was the last game you played, and 95% was your old rank prior to that game (regardless of how many games it took to get you there). If you keep iterating that you'll have a system where more recent games have already been weighted much more than the games you played 4 years ago.
Again, it doesn't work exactly this way, but it's less wrong of an explanation than others have expressed in this thread. What you'll find is that a sliding window actually won't change anybody's mu by very much. The only thing it will do is increase everybody's sigma, which actually will have the effect of lowering people's rank displayed in game.
Back when you only had 10 games, each additional game contributed a great deal to your rank. You could say something like, on average, each game contributed 10% to your total rank. Now that you have 1000 games, are each of those first ten games still 10% of your rank? No, they're worth more like 0.1%.
But I'm making a big simplification here. It's not technically accurate to say that your first half of your Allegiance career counts 50% to your rank and the second half contributes the other 50%. I didn't get to be a 16 by playing like a 6 for two years, and then playing like a 26 for two years after that. Actually the system doesn't have "memory" of individual games. It doesn't assign "weights" to any of them.
What it does is take your current rank, and rolls in the results of the last game you played. So you can kind of say that 5% of your rank was the last game you played, and 95% was your old rank prior to that game (regardless of how many games it took to get you there). If you keep iterating that you'll have a system where more recent games have already been weighted much more than the games you played 4 years ago.
Again, it doesn't work exactly this way, but it's less wrong of an explanation than others have expressed in this thread. What you'll find is that a sliding window actually won't change anybody's mu by very much. The only thing it will do is increase everybody's sigma, which actually will have the effect of lowering people's rank displayed in game.
Globemaster_III wrote:QUOTE (Globemaster_III @ Jan 11 2018, 11:27 PM) as you know i think very little of cashto, cashto alway a flying low pilot, he alway flying a trainer airplane and he rented
-
Spunkmeyer
- Posts: 2013
- Joined: Fri Jun 27, 2003 7:00 am
- Location: Contact me regarding: CC, Slayer and AllegWiki.
The effect on sigma will depend on how many games you have in the system. For someone like you it'd be a very minimal effect. For me it'd be big.
The effect on mu depends entirely on what happened early on. If it's in line with your later results, it won't change. The whole argument here is for a lot of people it is not. For newbies it's a lack of clue early on. For me, it's lots of sporadic bs late night sessions (I assume) all resulting in losses when the system started collecting data.
I think the problem is here:
QUOTE If you keep iterating that you'll have a system where more recent games have already been weighted much more than the games you played 4 years ago.[/quote]
Maybe, assuming you have a very large number of games played. That's not true for everyone. It'll never be true for me, I simply don't play as much as I used to. A lot of other people play fairly regularly, but again nowhere near as much as you. You have 4K games... I can clearly see people who have ~1500 games and whose rank is quite out of whack with their skill level as I observe it and about half of our player base have fewer games than that. If we manage to increase the player base (and we HAVE to, remember, otherwise the game is dead soon) there will be a lot more players with fewer games.
It takes a long time to dig yourself out of the mu hole.
Also, someone asked why this is important. It's not for posterity. It's important in order to have a working autobalance. Otherwise garbage in, garbage out.
The effect on mu depends entirely on what happened early on. If it's in line with your later results, it won't change. The whole argument here is for a lot of people it is not. For newbies it's a lack of clue early on. For me, it's lots of sporadic bs late night sessions (I assume) all resulting in losses when the system started collecting data.
I think the problem is here:
QUOTE If you keep iterating that you'll have a system where more recent games have already been weighted much more than the games you played 4 years ago.[/quote]
Maybe, assuming you have a very large number of games played. That's not true for everyone. It'll never be true for me, I simply don't play as much as I used to. A lot of other people play fairly regularly, but again nowhere near as much as you. You have 4K games... I can clearly see people who have ~1500 games and whose rank is quite out of whack with their skill level as I observe it and about half of our player base have fewer games than that. If we manage to increase the player base (and we HAVE to, remember, otherwise the game is dead soon) there will be a lot more players with fewer games.
It takes a long time to dig yourself out of the mu hole.
Also, someone asked why this is important. It's not for posterity. It's important in order to have a working autobalance. Otherwise garbage in, garbage out.
Last edited by Spunkmeyer on Fri Mar 02, 2012 11:44 pm, edited 1 time in total.
Want bigger games? Log on to play at the official game time: 9pmET/8pmCT/7pmMT/6pmPT every day of the week. Also Saturdays 8pm UTC.
Only if you have a low sigma. Which is what you get when you play a lot of games. Your sigma is three times mine. It should be much easier for you to gain rank than it is for me, and I've gone up at least 1.5 ranks since Allegskill inception.Spunkmeyer wrote:QUOTE (Spunkmeyer @ Mar 2 2012, 03:41 PM) It takes a long time to dig yourself out of the mu hole.
If you had my sigma, in fact, you would be 0.84 points higher just on that alone.
Also remember that your idea of a person's skill level includes things like how good a dogfighter they are or how much they know about core development. Allegskill doesn't care about any of that. It only cares if you win or lose games. You'd think there'd be a correlation between all that other stuff and winning, but not always. None of that matters unless you know how to be in the right place at the right time in the right ship -- which is not something you can notice as easily as dogfighting prowess.
Last edited by cashto on Sat Mar 03, 2012 12:39 am, edited 1 time in total.
Globemaster_III wrote:QUOTE (Globemaster_III @ Jan 11 2018, 11:27 PM) as you know i think very little of cashto, cashto alway a flying low pilot, he alway flying a trainer airplane and he rented
-
Spunkmeyer
- Posts: 2013
- Joined: Fri Jun 27, 2003 7:00 am
- Location: Contact me regarding: CC, Slayer and AllegWiki.
No, my idea of their skill level is really only about whether I'd want them on my team. Obviously it's not possible to prove anything objectively based on that, since it's strictly IMHO, but coupled with my personal experience it makes me conclude AS rankings are off.
EDIT: I think what we should do at this stage is run different simulations and see if we can tell the results look "better" or "worse". It may not get us anywhere, but it's easy enough that it's worth a shot.
EDIT: I think what we should do at this stage is run different simulations and see if we can tell the results look "better" or "worse". It may not get us anywhere, but it's easy enough that it's worth a shot.
Last edited by Spunkmeyer on Sat Mar 03, 2012 1:00 am, edited 1 time in total.
Want bigger games? Log on to play at the official game time: 9pmET/8pmCT/7pmMT/6pmPT every day of the week. Also Saturdays 8pm UTC.
Thanks for the plain explanations cashto.
I'm guessing whoever put that in the wiki, didn't anticipate a variety of other problems?
QUOTE (Wiki)However gamma; is the dynamics variable which prevents sigma from ever reaching zero, which in turn determines how quickly mu can in/decrease once sigma has stabilised. If we discover that sigma-stabilised ratings are moving too slowly to reflect genuine changes in skill, we will increase gamma.[/quote]cashto wrote:People never ask about changing gamma. Why? I think it's because they generally have no idea how the system works.
I'm guessing whoever put that in the wiki, didn't anticipate a variety of other problems?