Ranking System

[Nightflame]

The easiest way for a comm to win is to not accept any player below the average while team imbal is set at 1 max. This, however, doesn't show any damn skill, and leaves all the newbies out if both comms do it.

Accounting for 'overwhelming' variables is easy. We don't actually NEED all of those variables.

Commander skill distribution. (how well does a commander play what percent of the time?) (losing if you don't get your precious exp rock goes in here)
Player skill. (duh)
Player skill distribution. (vets + newbies? all mid skill?)

We assume distributions to be a bell curve, easily defined by two numbers. This is inaccurate of course, but is generally close enough to be useful. That leaves a few numbers:
Team 1 Mu (estimated skill)
Team 2 Mu
Team 1 Sigma (uncertainty in skill)
Team 2 Sigma
Commander...

We currently don't use the commander ratings in allegskill. Happily, a little time with a decent database of games could let us add it into the calculations. I suspect a straight up multiplier by commander rank would go a long way.

The really convenient thing? If one side auto wins 25% and the other side auto wins another 25%? It doesn't change a damn thing except how far apart the skills are. You see this in commander ranking. Commander rankings are MUCH closer than normal rankings.

[jbansk]

QUOTE (Nightflame @ Feb 18 2013, 04:54 PM) Commander skill distribution. (how well does a commander play what percent of the time?)

This is the flaw in your thinking.

1. You are assuming that you can deduce how well a Commander does from the outcome that is supposedly; most influenced by players.

2. You are assuming that you can deduce how well a Player does from the outcome that is likely; most influenced by Commanders.

3. You are completely ignoring rock placement, faction changes and development and their influence on outcome which further skews the already unreliable data.

[Nightflame]

Yes, I do think I can deduce those things. I'm sorry you don't understand how to deal with more than one number at a time.

Rock placement is simply a random var of no consequence. Faction changes/development don't matter except in that they change a commander's skill. If a comm can't handle a few changes, perhaps they aren't actually very good.

There are a LOT of different things going into the outcome of a game. They DON'T MATTER. Why? Because we only want one number: How much the player affects that outcome relative to other players.

[jbansk]

QUOTE (Nightflame @ Feb 18 2013, 05:55 PM) Yes, I do think I can deduce those things...

As I said, there is the flaw.

[SunTzu]

QUOTE (Nightflame @ Feb 18 2013, 03:54 PM) We currently don't use the commander ratings in allegskill. Happily, a little time with a decent database of games could let us add it into the calculations. I suspect a straight up multiplier by commander rank would go a long way.

In the mean time, adding the comm rank to the team rank might be a nice way to alleviate comm issues.

More noticeable, also it goes a long way to the past when people gave the weaker comm a little better players.

Another related as mentioned prevously is going from 30 ranks to 50, thus increasing the granularity of ranks, easier to see overwhelming stacks for those that aren't aware of the nature of the pilots on teams.


---

Now one thing that I read and hear a lot is that people know others skill, well, yes to a degree that is true, however frankly we're mainly talking about the players that don't know how well someone really is. And those new people are the ones that we are trying desperately to get into the game.

[Spunkmeyer]

Nightflame, you are correct in saying AS correlates with player skill, but wrong in claiming uncontrolled variables do not matter.

Trueskill assumes a chess-like environment. The more uncontrolled variables you introduce, the less accurate your results will be. The error is then propagated because the results from the "invalid" game is used to calculate the next change in rank. This is not black and white of course. AS balanced game is better than a random distribution, but we can almost always do better manually. The challenge is to get the system to do a better automated job and relieve the commanders of the responsibility, which they usually $#@! up.

So it is crucial to control the variables with better rock distribution, only allowing equally matched commanders, maintaining faction balance, not allowing crappy settings, taking measures against gaming the system etc. And allow only the results from such controlled games to be used for rank calculation.

Which unfortunately would mean another rank reset, so given the development resources etc. this is going to be some time, if we ever get there.

The idea is not to shoot for some sort of ideal but improve the fairness of your average PUG, step by step.

[Nightflame]

You may be right about uncontrolled vars and trueskill. Honestly, I don't see it unless you're mostly talking about stacking sides/stacking settings. Vars that are independent of players just shouldn't matter besides compressing ranks. Now, compressed ranks with the same precision does mean less accuracy, but trueskill still works as it's supposed to. Trueskill makes a bunch of more problematic assumptions, like that you won't pair up players with massive skill differences. I personally think we can safely lump most of those variables into the commander's rank. Techpath flexibility, faction, etc. are generally decided by the commander after all. We just need a good modification of trueskill for commander ability.

Stacking does of course skew results, oh well.

OTOH, trueskill is not the only possible algorithm. What we need is access to the databases of stored games we have, to figure out what's best.

[sgt_baker]

*waves*

[jbansk]

QUOTE (BackTrak @ Feb 17 2013, 04:48 PM) ...I would need someone to add AGC events that would fire at the end of the game to let me know total minutes spent in each ship type...

Is there any possibility that you could pull # of kills/ejects per ship type from that data?

QUOTE (Spunkmeyer @ Feb 18 2013, 07:24 PM) The challenge is to get the system to do a better automated job and relieve the commanders of the responsibility, which they usually $#@! up.

Commanders should not be relieved of that responsibility. They should become masters at it.

The reason they can't manage it is because they don't have the information necessary to control it. If STATS were pinned onto each player, any noob Comm could pick / maintain a team.

Using the AS method, we're basically saying "Here are your players, that should be even, good luck Commander!" and we get a bunch of pissed off Comms who are tired of being stacked with fake noobs and inaccurate ranks. STATS are not inaccurate and tell a Commander what he really wants to know.

[kramari]

There is a problem within AS code.

Sgt_Baker said this. He is willing to help with this but he doesn't want to discuss things here.

[14:01] Sgt_Baker: I'm happy to write the update for AS, and also assist with tweaking its settings suc hthat people do gain rank more quickly.
[14:01] Sgt_Baker: But get involved with FAO "discussions"?
[14:04] Sgt_Baker: There's a bug that was introduced during unneccessary "performance optimisation" when it was being transcoded to SQL.
[14:04] Sgt_Baker: Effectively, we're assuming that there's no such thing as an honest draw in Alleg.
[14:05] Sgt_Baker: Despite this the system still thinks there's a 1.09% chance of a draw, but the calculation that deals with that possibility has been stripped out.
[14:05] Sgt_Baker: So the setting needs to be set back to 0% and a recalc needs to take place.
[14:06] Sgt_Baker: Or we could introduce the missing code and recalc.
[14:06] Sgt_Baker: Either way...

This 1% mistake adds up on the number of games played. The mistake will only increase. While it is not an improvement on slow rising ranks. It can still be used to eliminate a mistake in the code. I do not know who has access to this and who can fix it, but it should be given some consideration.