ranking system

l1ngus · Post by **l1ngus** » Tue Dec 11, 2007 1:39 pm

This post replies to the discussion about the alleg-ranking-system from the following threat:
http://www.freeallegiance.org/forums/index...showtopic=37369

Some questions to come to a conclusion:

How much effort would it be to implement a new ranking system?

Is there anybody in the community, who has the knowledge and the will to make it happen?

I support the arguments of Terralthra. Our ambition should be to objectify the ranking system. Therefor a system orienteded at the chess-elo-ranking-system sounds good to me. The biggest problems of elo are, that: 1) itÂ´s not really possible to compare ranks from different eras 2) the system fails (produces too high ranks) if the same (i.e. world-top-players) play against each other many times.
Both issues are not relevant for alleg, since rank is only required for the presence (to balance a present game) and the community is small enough, so it wonÂ´t happen, that the topplayers play only against each other.

I have no idea, how to adjust elo (which is developed for two-player-games) to a teamgame. But it seems to be common knowledge, that actual helo is broken and the fact, that stacking improves your rank has a negative effect on the fun-to-play.

I would like to have a ranking system, which is based on elo for allegiance.

Related to the discussion about the democratic legitimation of a new ranking-system:
If a rankig system is valid or not is not a question of politics. Let the math-experts do a proposal for the technical stuff.

The political question is: Do we want a valid ranking system, and who will benefit from it?

I think, that most players want to have fair and exciting games. This requires that the teams have balanced skills. For this issue a correct working ranking system would be a pleasure. The stackers on the other hand would have the disadvantage.

If we implement allegage to the rankingsystem the elitist longtime-players can keep a good feeling independent from their actual skill. It wouldnÂ´t improve the possibility to form equally skilled teams on the basis of ranks.

(Please excuse my bad English (IÂ´m not a native speaker). Anyhow I hope that it is possible to understand my ideas.)

Malicious Wraith · Post by **Malicious Wraith** » Tue Dec 11, 2007 2:08 pm

The problem with allegiance ranking system is that your win/loss is based on your teams performance, and it is much too easy to tell what team is most likely to win before the game, hence the inflated/deflated ranks.

Do we want a new ranking system: Sure. Who will benefit from it? Nobody.

People just like to see numbers next to their names : P

sgt_baker · Post by **sgt_baker** » Tue Dec 11, 2007 3:38 pm

QUOTE How much effort would it be to implement a new ranking system?[/quote]
To implement a new ranking system in a planned, organised and tested fashion takes a fair amount of time and effort.

QUOTE Is there anybody in the community, who has the knowledge and the will to make it happen?[/quote]
Yes. For the past year MrChaos, Terralthra and I have been working on exactly this problem. For future reference, the system we've developed is based on Microsoft's Trueskill, and we've decided to name our implementation AllegSkill.

QUOTE The political question is: Do we want a valid ranking system, and who will benefit from it?[/quote]
QUOTE The problem with allegiance ranking system is that your win/loss is based on your teams performance, and it is much too easy to tell what team is most likely to win before the game, hence the inflated/deflated ranks.[/quote]
The response to this question and statement belong together. If the outcome of most games of allegiance are easy to predict, then the games are unbalanced by definition. In this context a 'balanced game' refers to a condition where it is close to impossible to predict the outcome of the game prior to it being played. In other words, both teams have close to a 50% chance of winning. Based on this it is reasonable to assume that the entire community would benefit from a rigorous and mathematically sound ranking system (or more accurately 'skill measurement system'), since such a system, again by definition, would allow teams to be arranged in such a way as to provide the least predictable game outcome.

There are a number of reasons behind the AllegSkill project having been developed behind closed doors, the foremost being the desire to avoid a community flame fest at the merest suggestion of a new/different ranking and balancing system. It was felt that any such bickering would serve only to detract from the task at hand. Since quite a number of people outside of the project have been aware of AllegSkill's existence for some time, it is only fitting to begin a broader discussion in a thread such as this.

I will, however, reiterate: Please don't turn this into a flame war!

I've got to head to work now, but I'll make a detailed post describing AllegSkill and attempt to answer any questions concerning our system, or ranking in general, tomorrow.

Baker

Post by **BlackViper** » Tue Dec 11, 2007 3:55 pm

Since he has publicly acknowledged the project I want to say this.

This has been a direct result of feedback to us. I will not let this topic turn into a flamewar, shouting match, etc. Please read what he posts and reread it again. Please be respectful of the work they have done so far. I am sure they will pay attention to any feedback that makes sense.

This topic has been hashed to death over the last year or so. They have read and paid attention to those posts. EVERYONE has an opinion /biggrin.gif" style="vertical-align:middle" emoid=":biggrin:" border="0" alt="biggrin.gif" /> Just post yours based upon facts.

jgbaxter · Post by **jgbaxter** » Tue Dec 11, 2007 4:03 pm

My only thoughts are;

1) Autobalance shouldn't be mandatory.
2) Ratings should only accumulate with Autobalance on.
3) AllegAge should be averaged with AllegSkill so experience and skill work in together.

rojomojo915 · Post by **rojomojo915** » Tue Dec 11, 2007 4:08 pm

I dont think age should be a factor. The way system is setup now, you would have to be playing a very long time to reach the top expert ranks anyways. Also, age has no true bearing on skill, it is just a way to slow the rank increase of a player unless you make AllegAge based on hours played and not days in alleg.

Terralthra · Post by **Terralthra** » Tue Dec 11, 2007 7:07 pm

The quick answer is yes, but we've fudged with that to get some preliminary ranks to test the feasability of the system. Rather than go into the exact methodology involved, I'll wait for Sgt_Baker to post his explanation, and then we can answer questions afterward, if they still exist.

(Hint: the first game of the system was automatically balanced, since everyone had identical (though inaccurate) ratings...)

Ahaneon · Post by **Ahaneon** » Tue Dec 11, 2007 9:28 pm

What are the goals you want to archieve with a new stat system?

Anti-stacking?
I honestly don't think there's a solution based on rank that can solve this issue, not as long as autobalance is disabled. To many factors play a role with stacking, not just wanting to win... Mainly playing with friends, Who is commanding the teams, which faction they are playing, etc. A real "points" stacker would probably be more likely to anti-stack. Plus, the most important point everyone always forgets: The stack starts with the commander. A commander can't win games (the team wins), but he sure as hell can lose them. Add a simple command rating algorithm to the existing Elo ranking system, where the balance of the commanders is taken into account to determine the total ratings of a team and then study the stack problem, you will notice a definite corelation.

Accurate skill reflection?
What kind of skill do you want to reflect?
- Stacking? (use ELO)
- Commandskill? (use commander rating ELO)
- Anti-Commerce (# of miners/Constructors/towerlayers/minelayers killed + assists?)
- Dogfighting ability? (Kills/Pods/Who-was-killed-Dogfightingrating/Assists)
- Bombing? (Kind of bases destroyed/Game won or loss/ Game Winning basekill)
- Nanning (can info be saved on how long ago you nanned a bomber that jsut killed a base?)

Damn gotta go... no time to finish this.

MrChaos · Post by **MrChaos** » Wed Dec 12, 2007 2:33 am

jgbaxter wrote:QUOTE (jgbaxter @ Dec 11 2007, 11:03 AM) My only thoughts are;

1) Autobalance shouldn't be mandatory.
2) Ratings should only accumulate with Autobalance on.
3) AllegAge should be averaged with AllegSkill so experience and skill work in together.

You'll be surprised to know this Bax maybe but we agree to a degree on #1
I agree with #2 but not for the same reasons I fear
I will not again address #3 since you are presenting a modification without any work. Yes it's going to being ignored but if it makes you even a little better I do feel bad.

rojomojo915 wrote:QUOTE (rojomojo915 @ Dec 11 2007, 11:08 AM) I dont think age should be a factor. The way system is setup now, you would have to be playing a very long time to reach the top expert ranks anyways. Also, age has no true bearing on skill, it is just a way to slow the rank increase of a player unless you make AllegAge based on hours played and not days in alleg.Do the work, present the proof, otherwises

Obviously we agree, I appreciate the support but in this bit the proof is in the pudding. You haven't even tasted the pudding

spideycw wrote:QUOTE (spideycw @ Dec 11 2007, 11:18 AM) We should just do away with ranks entirely

Why?

Dengaroth wrote:QUOTE (Dengaroth @ Dec 11 2007, 11:19 AM) For the record, having seen the rankings AllegSkill gives to various members of our community (and their progressions), I have the utmost faith in its accuracy.
Any chance you could, like, you know... wait for details on how the system works before you start bitching and offering suggestions on how to improve its functionality?

Thanks for the support and yeah at least wait for the details before you start the hating

Malicious Wraith wrote:QUOTE (Malicious Wraith @ Dec 11 2007, 11:25 AM) I cant wait to see this stuff /tongue.gif" style="vertical-align:middle" emoid=":P" border="0" alt="tongue.gif" />.

The funny thing is MW a ranking system ends the need to throw yourself into the breech if you truly are The Next Big Thing.... you also may be shocked to know your type of case is another reason I want an accurate ranking system.

adam4 wrote:QUOTE (adam4 @ Dec 11 2007, 12:01 PM) Seconded

Appreciate the support the kool aid is wonderful but don't you want to see it's color?

quackdamnyou wrote:QUOTE (quackdamnyou @ Dec 11 2007, 12:27 PM) I have faith in those who have put in a lot more time than I have in thinking of this problem. I eagerly await a chance to review said system, even if it takes Two Weeks.

This isn't about Two Weeks since it's basicly a done deal with about 3430349309 things to change /wink.gif" style="vertical-align:middle" emoid=";)" border="0" alt="wink.gif" />

Lykourgos wrote:QUOTE (Lykourgos @ Dec 11 2007, 01:51 PM) To be fair, Frag, Pook was throwing a lot of silly stuff into ELO from the beginning, it would be possible to argue that it wasn't The One True Rigorous Ranking System due to that.

I am interested that apparently this system has already managed to produce ranks for us. It isn't a prerequisite that the games played for data were balanced under this system?

First it's important to note that all those who have been regularly, repeatedily, and continuously villified for being dictators, the other n word etc have been completely reasonable, forthcoming, and supportive during the on again and off again development. Once we showed that we weren't being fan boys but truly were giving it a significant effort the flood gates opened.

Secondly while the answer is more complicated as you've obviously ginned out the system with accurately gathered, orthoginal data can produce a rank based on the exsisting game base. The less stacked the quicker the results.

BlackViper wrote:QUOTE (BlackViper @ Dec 11 2007, 02:18 PM) Frag, these guys would not be working on an alternative if we did not think one was needed. /smile.gif" style="vertical-align:middle" emoid=":)" border="0" alt="smile.gif" /> The zone leads and admins long ago discussed the need to have an outside group work on this. Hence you are just starting to see what they are doing.

The following is my own OPINION (not as an admin/zone lead)

I personally agree in part that I wish there were no ranking system. But we have seen how the community reacts under that in the past. So an alternative is being worked on. Let's see what they come with and how it works before we make any other type of decision whether to continue or not.

But I do strongly see where in the future something is going to be required to balance teams. It has been our #1 issue from day one. But again, that is a subject for a different topic, not here. Let these guys present what they have and how it will work. We can address your concerns in another discussion.

The crys of stacking have haunted my sleep since the very first day I logged on to AZ and KevDude repeatedily point scummed me in a scout against a cap.

Ahaneon wrote:QUOTE (Ahaneon @ Dec 11 2007, 04:28 PM) What are the goals you want to archieve with a new stat system?

Anti-stacking?
I honestly don't think there's a solution based on rank that can solve this issue, not as long as autobalance is disabled. To many factors play a role with stacking, not just wanting to win... Mainly playing with friends, Who is commanding the teams, which faction they are playing, etc. A real "points" stacker would probably be more likely to anti-stack. Plus, the most important point everyone always forgets: The stack starts with the commander. A commander can't win games (the team wins), but he sure as hell can lose them. Add a simple command rating algorithm to the existing Elo ranking system, where the balance of the commanders is taken into account to determine the total ratings of a team and then study the stack problem, you will notice a definite corelation.

Accurate skill reflection?
What kind of skill do you want to reflect?
- Stacking? (use ELO)
- Commandskill? (use commander rating ELO)
- Anti-Commerce (# of miners/Constructors/towerlayers/minelayers killed + assists?)
- Dogfighting ability? (Kills/Pods/Who-was-killed-Dogfightingrating/Assists)
- Bombing? (Kind of bases destroyed/Game won or loss/ Game Winning basekill)
- Nanning (can info be saved on how long ago you nanned a bomber that jsut killed a base?)

Damn gotta go... no time to finish this.

It seems like you already based your opinion on.... what /wink.gif" style="vertical-align:middle" emoid=";)" border="0" alt="wink.gif" />
I mean you haven't even seen what we are proposing and already you saying it won't work.

Fragtzack wrote:QUOTE (Fragtzack @ Dec 11 2007, 05:19 PM) Correct on the vote.

Regarding ego, my Ego would be bruised if people voted against a project I worked so hard on. There is no doubt a huge amount of work went in this. The developer of Helo is a lot better person then me if he felt no dings to pride. I hope to mature to that state of zero pride some day, currently though I still contain pride in me. I was probably wrong in my ego statements, I was speaking from how I would feel.

I learned a few years ago that your perception is your reality. Great leaders of humanity can perceive other's reality with great accuracy. Once a person understands another's reality, the possibilities are endless in leading that person.

I do not perceive other reality to a degree of accuracy. I was just speaking from my own personality about ego. If I am wrong in assessing another reality, thank the heavens in this case.

Actually Frag please read the bit above to find out the reactions from those in charge of the current system. At no time where we hindered in the efforts AT ALL. So if I might, and with all due respect, please take this bit somewhere else as it is only tangential to the this topic

Lykourgos wrote:QUOTE (Lykourgos @ Dec 11 2007, 05:48 PM) * snipped insult * behave damn it
Perhaps its worth it to hide the leaderboard and disconnect TAG from the servers, that wouldn't be too much effort, probably.

Oh yes, and may I suggest that if you institute a new ranking system, don't display visible ranks or a ranked leaderboard. Vets will be able to tell if the system works by the evenness of the games it generates.

I see STATISTICS being availible for view no matter the direction taken

edit: changed from black to bolded at the suggestion of TheBored

MrChaos · Post by **MrChaos** » Wed Dec 12, 2007 3:14 am

Lykourgos wrote:QUOTE (Lykourgos @ Dec 11 2007, 09:55 PM) MrC: You said orthoginal, did you mean orthogonal or original? This is a serious question.

Being trained in matrix algerbra and as you suspected I meant orthogonal [ Im a horrible speller]. Obviously not every game could be used for any variety of reasons and thus my comment. Im going to trust you Lyk that isn't going to turn into a gotch ya and you are really interested in the results.

Matter of fact yourself, along with people like Shiney, DogBones, Saxy, Koffi and many others Im sure Im unaware of are the people we WANT to ask questions.

MrChaos

edit: spelling ><