jgbaxter wrote:QUOTE (jgbaxter @ Dec 11 2007, 01:26 PM) There will never be an accurate system, too many variables, I'd think you can agree on that.
No, I do not agree. That's the same thing people said about Elo's system for rating chess players. Then FIDE adopted it, and it's now the basis for the standard system used worldwide, as a mathematical system for estimating the skill of a chess player by abstraction. The principles of Bayesian theory are applied in nearly every rating system worth a damn, and coincidentally, distorted massively by HELO.
jgbaxter wrote:QUOTE (jgbaxter @ Dec 11 2007, 01:26 PM) The idea I mentioned is definately more accurate, therefore it would be an improvement. For that reason it's worth looking at.
How on earth is taking a system that is right now, entirely empirical in its approach (even if the algorithm it filters the data through is rubbish) and adding fuzzy feels-good anti-math to it '
more accurate'? Did the above-mentioned example of a returning vet or a new account for an existing vet being artificially constrained to a rank much below their apparent skill for
months slide by your eyes?
jgbaxter wrote:QUOTE (jgbaxter @ Dec 11 2007, 01:26 PM) Absolutely nothing will ever create an accurate ranking system, at least without a team of Nasa scientists, 25 metric tonnes of twinkies, and a jar of magic powder.
Luckily for me, a team of well-paid researchers already has come up with a statistically valid, mathematically rigourous skill-rating system for complex team-based games. It's called TrueSkill, done by the fine folks at MS Research, and a couple of more mathematically-oriented folks have been hard at work applying it to Allegiance. I am not the one who did the majority of the work, someone else is, and he'll have to tell you who he is if he wants to. The thorniest issue was the additional variable of fractional time-in-game. That variable was resolved by the aforementioned team leader, through communication with the high-paid Ph.Ds at MS Research to confirm the mathematics underlying it.
jgbaxter wrote:QUOTE (jgbaxter @ Dec 11 2007, 01:26 PM) Ed.: poisoning the well, strawman arguments removed
takingarms1 wrote:QUOTE (takingarms1 @ Dec 11 2007, 01:33 PM) How do you define a mathematical system to describe someone's skill at allegiance? The measure of the system's accuracy is ultimately going to be a subjective judgment, since there is very little in the way of objective criterion to evaluate what HELO is supposed to evaluate.
The same way Elo did it for chess: by abstracting out the unimportant and vaguely defined variable, and only considering what, in the end, truly matters: results. If you have more skill than the other guy, you will be more likely to win. If you have more of some vaguely-defined or tangential quantity like 'better probing,' but you lose anyway, then your probing wasn't as important to winning the game. In the end, the objective criterion is whether you won or you lost.
takingarms1 wrote:QUOTE (takingarms1 @ Dec 11 2007, 01:33 PM) You have a very nice, well-reasoned critique of HELO which isn't terribly useful since it fails to provide a better alternative or even a way to improve the current system.
See above, and you can't improve that which does not work.
takingarms1 wrote:QUOTE (takingarms1 @ Dec 11 2007, 01:33 PM) Personally, I think hours played is just about as accurate as HELO in assigning rankings that can help create more balanced games. Which is to say, neither one is very accurate but either one is probably better than nothing.
Two responses:Picking numbers out of a hat would also be inaccurate but better than nothing. Should we incorporate that as well?Starfire and KingArthur both have several times more hours in-game than you do. Should their rank be triple (Starfire) or sextuple (KingArthur) yours?
takingarms1 wrote:QUOTE (takingarms1 @ Dec 11 2007, 01:33 PM) If we want a real, practical system for creating more balanced games, how about this simple idea: code the game so that newbs can only join the side with fewer newbs on it, thus resulting in even newb distribution before and after launch. I guarantee that would do more than HELO or some balance button to aid in bringing about more balanced games.
Mandatory auto-balancing is essential to any accurate rating system. Turning it off for SGs is certainly going to be the appropriate thing to do, and those games will also have to not count towards any of the players' rankings, as well. The historical fact that the vast majority of allegiance players are against auto-balance in pick-up games is one data point that points in the direction that the 'majority of the community' doesn't really want an accurate rating system nearly as much as the people in said group proclaim.
