Ranking System

jbansk · Post by **jbansk** » Mon Feb 18, 2013 4:19 pm

Nightflame wrote:QUOTE (Nightflame @ Feb 18 2013, 12:41 PM) Rock placement will even out over large samples of games.

That would only be correct if the same teams with the same commanders and players and factions, played those games with varying rock placement over and over until you had enough samples to wean out the players individual influence on outcome.

Each time you introduce a variable that has such overwhelming influence on game outcome, you taint the sample of player influence with the variable's influence and therefore will need more data to wean out the players influence. Because of the 3 variables I mentioned (commander, rocks, development), you have completely buried the players value of responsibility so deep inside the sampled data that it is impossible to retrieve AND you have violated the NUMBER ONE rule of the methodology you are using. THE PLAYERS BEING MEASURED MUST BE THE SOLE REASON FOR GAME OUTCOME.

This is why it has never / will never work.

My method of classifying pilots, achieves in an instant what ELO, HELO and Allegskill has failed to do over the last 7 years.

TravisLangen · Post by **TravisLangen** » Mon Feb 18, 2013 4:40 pm

I think instead of complex changes to the ranking system,
It would be a lot better just to make your more recent games effect your rank more than the games you played 1-10 years ago.
Make say, your last 100 games effect your rank a lot, last 250 effect your rank moderately, 250 - ~ games effect your rank very little.
This would especially help for newbs who pretty much lost their first 50 games (like me) and for vets who used to be good, but have their rank adjusted to their current skill, instead of their skill 5 years ago.

Nightflame · Post by **Nightflame** » Mon Feb 18, 2013 5:00 pm

Suffice to say that large enough sample sizes solve most problems. Rock placement is independent of playerskill, and so it will even out. The main confounding variable is non-random teaming, and that's accounted for to some extent by stack adjustments.

It may be that the number of games for decent accuracy is in the thousand range, but I very seriously doubt it. I will note that by your own logic, you can't provide much evidence that the current system is failing to rank accurately. How would you? Provide anecdotes about how you think such and such player is better than their rating? Like THAT'S evidence?

EDIT: This isn't to say your idea is bad. I'm all for it. However, ranking players is not by ANY means impossible.

jbansk · Post by **jbansk** » Mon Feb 18, 2013 5:41 pm

Nightflame wrote:QUOTE (Nightflame @ Feb 18 2013, 02:00 PM) ... ranking players is not by ANY means impossible.

It is using the method you're trying to apply here. I do not argue that the method itself, isn't sound. My argument is that applying that methodology to a game like this, is inappropriate.

If you want proof of why it doesn't work, you can simply read bayesian's underlying premise (players being sole responsible party for game outcome).
If you want evidence those methods don't work, I give you ELO, HELO, and Allegskill.

Nightflame · Post by **Nightflame** » Mon Feb 18, 2013 6:37 pm

You provided proof that it isn't perfect. However, since players have a nonzero influence on game outcome, information theory says that you can use game outcomes to predict player ability. Since we are defining player skill as their ability to win games, as opposed to their ability to dogfight, or ram rocks, it is unlikely that we will find a better way to measure player skill. We cannot isolate player skill from games won. We ALSO cannot isolate it from kills, time in a bomber, or any other statistic. Since any other variable is ALSO affected by commander/development/rocks, we expect the simpler measurement (wins) to be more accurate.

You point to allegskill as evidence this method doesn't work. I point to allegskill as evidence that it does work. Allegskill puts phantom, babel, weed, ryu, and whoever narg is on the top of the leaderboard. Are these players actually bad at the game? Ha! No, they're very good. It's working. If it didn't work, if there was something fundamentally wrong with the method, you would see great players at very low ranks, and bad players at very high ranks. While there's quite a bit of uncertainty, good players are heading to the top, just like they're supposed to.

Spunkmeyer · Post by **Spunkmeyer** » Mon Feb 18, 2013 7:09 pm

The "large enough sample size" would have worked better if we were simply counting wins and losses. However, the magnitude of ranking adjustment is different for each game. It's also heavily skewed towards earlier games in one's career.

So, how much this $#@!s up AS depends on other factors such as how frequently rock distribution screws up a team. For example, you could say 0 exp rocks within the first four adjacent sectors confers a significant disadvantage. Chances of that happening without the other team having the same issue is about 15%. Percentage of those games where the team is evenly matched or superior to the other team and lose? 50%? So perhaps you can get 7-8% invalid AS adjustment after a game like this on average.

Same situation applies where the game is "unfair" for other reasons - crappy commander, crappy settings.. Perhaps better players migrate to the better commander, so the team with the crappy commander is usually the underdog, which means the frequency of AS $#@!ups would be even less, perhaps at 5%.

So yeah, it doesn't invalidate the results, but it adds up. Trying to ensure fairness (disregarding players) of rules will improve the accuracy of AS.

Spunkmeyer · Post by **Spunkmeyer** » Mon Feb 18, 2013 7:15 pm

Also I'm not happy with letting the commander do the balancing - it's better than nothing, but A LOT of @#(! happens:

The other commander tries to pull a fast one...
The commander is short on time and just wants to play...
Commander gets busy during the game and does not pay attention to joins...
Commander doesn't notice a specific setting before game start...
People switch at the last minute...
People get tired and complain about game not starting, commander says $#@! it, game launches with sub-optimal settings...

I've seen all of this yesterday in the span of a few hours..

jbansk · Post by **jbansk** » Mon Feb 18, 2013 7:45 pm

It is vitally important to encourage commanders to stack their team (as any commander of any military would do) and equally as vital to make his Counterpart capable of preventing that stack.

Commanders are responsible for preventing their opposing enemy from gaining any advantage, this includes his personnel. You want to be a good Commander? Learn to select the right teams and how to keep it balanced.

@Spunky,

How much % would you say game outcome was solely influenced by the Commander's action or inaction regardless of player activity? In other words, how many games that were sure to win, failed because of the Commander? How much % would you say outcome was solely influenced by rock placement regardless of Player or Commander influence? How much % would you say game outcome was solely influenced by faction choice regardless of Player or Commander influence.

How do you account for those OVERWHELMING variables when trying to wean out the player's contribution to games overall? As player base shrinks and variables become smaller, we can begin to see patterns, but I fear that this becomes seriously skewed (more so than now) when the player base is larger.

Xynth · Post by **Xynth** » Mon Feb 18, 2013 7:54 pm

jbansk wrote:QUOTE (jbansk @ Feb 18 2013, 01:45 PM) It is vitally important to encourage commanders to stack their team (as any commander of any military would do) and equally as vital to make his Counterpart capable of preventing that stack.

Commanders are responsible for preventing their opposing enemy from gaining any advantage, this includes his personnel. You want to be a good Commander? Learn to select the right teams and how to keep it balanced.

Problem is it doesn't work that way in Alleg. Comms do not have control of their team, only who they allow on. If you are a newer comm trying to earn a rep it can be tough to pull good pilots. If you leave newbies hanging and requesting they will quit and find something else to do.

This is a horrible philosophy for any comms to have for pugs. Pick your squad that way, sure, but you cares about AS when it comes to SGs.

jbansk · Post by **jbansk** » Mon Feb 18, 2013 8:03 pm

PUGS should be selected the same way SG's are. The only problem is that the PUGS Commander doesn't know his team like the SG Commander knows his team.

Give the PUGS Commander the same information (as I'm suggesting by use of icons), and you have an informed Commander in a pick up game.