ranking system

Catch-all for all development not having a specific forum.
sgt_baker
Posts: 1510
Joined: Wed Oct 20, 2004 7:00 am
Location: London, UK.
Contact:

Post by sgt_baker »

Raveen wrote:QUOTE (Raveen @ Dec 12 2007, 03:17 PM) Baker, would you be adverse to me adding these posts to the wiki as you post them as a permanent reference on AS and HELO?
Not at all. Anyhow, we can ninja edit the wiki when we've realised that Baker has been thinking with his prostate for the past year. /tongue.gif" style="vertical-align:middle" emoid=":P" border="0" alt="tongue.gif" />

jgbaxter wrote:QUOTE (jgbaxter @ Dec 12 2007, 03:21 PM) Sounds like you are saying AllegSkill will use the game data from all previous games, I must certainly be missing something because surely that's not to be the case? /huh.gif" style="vertical-align:middle" emoid=":huh:" border="0" alt="huh.gif" />
We'll cover the issue in detail when I get round to that post, but essentially we have the option to go either way. One of the biggest deciding factors is how the community would react to the prospect of yet another ranks reset.

B
Image
Granary Sergeant Baker - Special Bread Service (Wurf - 13th Oct 2011)
Kltplzyxm
Posts: 2623
Joined: Thu Apr 12, 2007 4:36 pm

Post by Kltplzyxm »

Question about AllegSkill: How does it address numbnut comms who can't win games to save their lives despite the excellent team they have? How does the system deal with games being won/lost mainly by an extremely bad/good comm? Does the team get penalized for the loss even though it's not their fault?
Last edited by Kltplzyxm on Thu Dec 13, 2007 7:46 pm, edited 1 time in total.
sgt_baker
Posts: 1510
Joined: Wed Oct 20, 2004 7:00 am
Location: London, UK.
Contact:

Post by sgt_baker »

Mr. Kltplzyxm wrote:QUOTE (Mr. Kltplzyxm @ Dec 13 2007, 07:45 PM) Question about AllegSkill: How does it address numbnut comms who can't win games to save their lives despite the excellent team they have? How does the system deal with games being won/lost mainly by an extremely bad/good comm? Does the team get penalized for the loss even though it's not their fault?
First it's worth noting that commanding skill and player skill are ranked separately. Assuming we still have in-game display of ranks, it'll be a relatively trivial matter to ensure that both commanders and their teams are well balanced. Specifically, wrt your question, a loss is a loss. AllegSkill doesn't care how you lost the game. Losing due to poor command is an risk inherent in the very concept of a command/team setup.

I actually ran a study on how much commander skill contributes to the outcome of a game. When considering all 40,000 games in the ASGS database the effect of commander skill was surprisingly small when compared to the effect of team balance. In the vast majority of cases having a better team will win the game. I did, however, find that good comms tend attract better teams, which is a result of understandable player behaviour.

B
Image
Granary Sergeant Baker - Special Bread Service (Wurf - 13th Oct 2011)
sgt_baker
Posts: 1510
Joined: Wed Oct 20, 2004 7:00 am
Location: London, UK.
Contact:

Post by sgt_baker »

jgbaxter wrote:QUOTE (jgbaxter @ Dec 11 2007, 04:03 PM) My only thoughts are;

1) Autobalance shouldn't be mandatory.
2) Ratings should only accumulate with Autobalance on.
3) AllegAge should be averaged with AllegSkill so experience and skill work in together.

1) I agree entirely.
2) I disagree. AllegSkill is extremely sensitive to stacking and such shenanigans. I'll explain in detail when I make the big post regarding AS functionalty, but for now I'll just say that AS generates accurate ratings with or without autobalance.
3) AllegSkill already includes an 'alleg age' factor. Again, more when I make the big post.
Image
Granary Sergeant Baker - Special Bread Service (Wurf - 13th Oct 2011)
jgbaxter
Posts: 2181
Joined: Mon Apr 25, 2005 7:00 am

Post by jgbaxter »

sgt_baker wrote:QUOTE (sgt_baker @ Dec 13 2007, 02:11 PM) 1) I agree entirely.
2) I disagree. AllegSkill is extremely sensitive to stacking and such shenanigans. I'll explain in detail when I make the big post regarding AS functionalty, but for now I'll just say that AS generates accurate ratings with or without autobalance.
3) AllegSkill already includes an 'alleg age' factor. Again, more when I make the big post.

1) So that's not an issue then. /smile.gif" style="vertical-align:middle" emoid=":)" border="0" alt="smile.gif" />

2/3) I await the 'big post'. /wink.gif" style="vertical-align:middle" emoid=";)" border="0" alt="wink.gif" />

And I say again, simply because I may disagree with how the system may or may not work, doesn't mean I don't respect the effort put into its development. Clear? /mrgreen.gif" style="vertical-align:middle" emoid=":D" border="0" alt="mrgreen.gif" />
n.b. I may not see a forum post replied to me or a pm sent to me for weeks and weeks...
cashto
Posts: 3165
Joined: Mon Sep 10, 2007 5:40 am
Location: Seattle

Post by cashto »

jgbaxter wrote:QUOTE (jgbaxter @ Dec 13 2007, 05:17 PM) 1) So that's not an issue then. /smile.gif" style="vertical-align:middle" emoid=":)" border="0" alt="smile.gif" />

2/3) I await the 'big post'. /wink.gif" style="vertical-align:middle" emoid=";)" border="0" alt="wink.gif" />

And I say again, simply because I may disagree with how the system may or may not work, doesn't mean I don't respect the effort put into its development. Clear? /mrgreen.gif" style="vertical-align:middle" emoid=":D" border="0" alt="mrgreen.gif" />
Regarding #2, I know that the Glicko / TrueSkill systems work best when the teams are evenly matched. But that's not to say that an unbalanced game won't ever give useful information -- it will, just at a slower rate. It's not necessary to throw out games which didn't fit a "balanced" criterion, because the system already accounts for that mathematically. Allegiance has a fair mix of balanced and unbalanced games that I don't see the time-to-converge as a big problem.

Regarding #3, I am not privy to the details of how AllegSkill works -- I don't think TrueSkill has a built-in age limitor. But there's two ways I could envision for building that in:

The first way is to show an artificially low rating for short-timers in the lobby, regardless of what their real rating may be. In the example of Aarmstrong picking a new hider, the system will quickly learn he's a good player (he wins all his games!) and will give him a high score, which in turn is used in the calculation of the score of everyone who plays against him (so losing against him isn't penalized as strongly as losing against RandomNoob(1)). But there's also a multiplying factor that starts at 0.0 and increases to 1.0 over the course of a month. The rank displayed in the lobby is calculated via (multiplying factor) x (real rating), so he still appears as a novice.

The second way has to do with the fact that Glicko and TrueSkill makes a distinction between someone who has played a lot of games and absolutely, definitely is a (15), and someone who hasn't played many games, but is somewhere between (5)-(25). In the second case, you could average the top and bottom values of the range and display a (15) in the lobby. Or you could give a conservative estimate of the player's skill, and simply display a 5 -- even if the player in all probability is much better than that. As time goes on and he plays more and more games, the upper and lower bounds of the estimate converge towards the player's real score.
Globemaster_III wrote:QUOTE (Globemaster_III @ Jan 11 2018, 11:27 PM) as you know i think very little of cashto, cashto alway a flying low pilot, he alway flying a trainer airplane and he rented
Papsmear
Posts: 4810
Joined: Sun Jul 06, 2003 7:00 am
Location: Toronto, Canada

Post by Papsmear »

GapDragon had an excellent post a while ago revamping various points awarded in-game with the idea that it was of more value to a team to bomb/capture and enemy base than killing single craft. The in-game point system as it is, is not a really good indication of a player's skills or his usefulness to the team overall.
Revap the point system first.
Image
Image
MrChaos
Posts: 8352
Joined: Tue Mar 21, 2006 8:00 am

Post by MrChaos »

Papsmear wrote:QUOTE (Papsmear @ Dec 15 2007, 12:15 AM) GapDragon had an excellent post a while ago revamping various points awarded in-game with the idea that it was of more value to a team to bomb/capture and enemy base than killing single craft. The in-game point system as it is, is not a really good indication of a player's skills or his usefulness to the team overall.
Revap the point system first.
Pappy this is not points based system
Ssssh
Pook
Posts: 1758
Joined: Tue Aug 13, 2002 7:00 am
Location: Texas, USA

Post by Pook »

Hi all.

HELO isn't really based on ELO (at least not our previous ELO implementation). It was written from scratch as an attempt to mimic Bungie's scoring implementation for Halo 2.

For example, while it does include a "newbie helper function" (as does Bungie's) it does not do much averaging. The point awards are determined by comparing win/loss vs each individual on the opposing team. (Allegiance has no FFA).

Here's a link to Bungie's: http://www.bungie.net/Stats/content.aspx?link=h2statoverview. HELO is basically the same thing.

I had at one point posted all the source code, if it's still here in the forums it may have other information for you.

Cheers,
Pook

EDIT: I realized the the source that was previously posted may have been for ELO, not HELO. Here's HELO:

CODE-- RANK:
-- CONVERT(int,CONVERT(real,ELORanking) / POWER(50 + (CONVERT(real,ELORanking) / 100),1.15936))

-- GAME DURATION ISN'T CURRENTLY USED FOR SCORING
DECLARE @GameDuration real
SET @GameDuration = (SELECT DATEDIFF(second,GameStartTime,GameEndTime) FROM Game WHERE GameIdentID = @GameID)

-- WHEN GAME DOESN'T COUNT NO CHANGES ARE RECORDED
DECLARE @GameCounted int
SET @GameCounted = 1

-- TO BE OUTPUT VIA THE WEBSERVICE
DECLARE @GameReason varchar(100)
SET @GameReason = 'Game Counted'

-- SEE IF STATS WERE ENABLED IN GAME SETTINGS
DECLARE @GameStatsCounted int
SET @GameStatsCounted = (SELECT GameStatsCount FROM Game WHERE GameIdentID = @GameID)

-- OVERRIDE AND MAKE STATS ALWAYS COUNT
SET @GameStatsCounted = 1

-- SHOULDN'T HAPPEN
IF @GameDuration < 1
BEGIN
RAISERROR('Game Duration was Zero or Negative',16,1)
RETURN
END

-- WHOLE GAME LESS THAN 5 MINUTES (300 SECONDS)
IF @GameDuration < 300
BEGIN
SET @GameCounted = 0
SET @GameReason = 'Game duration was less than 5 minutes'
END

-- GAMES WITH MORE THAN 2 TEAMS CANNOT COUNT
IF (SELECT MAX(GameTeamNumber) FROM GameTeam WHERE GameID = @GameID) > 1
BEGIN
SET @GameCounted = 0
SET @GameReason = 'Game had more than 2 teams participating'
END

-- DRAWS DO NOT COUNT
IF (SELECT MAX(GameTeamWinner) FROM GameTeam WHERE GameID = @GameID) = 0
BEGIN
SET @GameCounted = 0
SET @GameReason = 'Game was a draw'
END

-- AT LEAST 10 PLAYERS THAT PLAYED AT LEAST HALF THE GAME
IF (SELECT COUNT(gtm.GameTeamMemberMemberID)
FROM GameTeamMember gtm
INNER JOIN GameTeam gt ON gt.GameTeamIdentID = gtm.GameTeamID
WHERE gt.GameID = @GameID
AND gtm.GameTeamMemberDuration > (@GameDuration / 2)) < 10
BEGIN
SET @GameCounted = 0
SET @GameReason = 'Less than 10 players that played for at least half the game'
END

-- STATS CAN BE TURNED OFF TO NOT AFFECT RATINGS
IF @GameStatsCounted <> 1
BEGIN
SET @GameCounted = 0
SET @GameReason = 'Stats were disabled in the game settings'
END

-- PLAYERS TABLE HOLDS ALL THE STATISTICS FOR THIS GAME
DECLARE @Players TABLE (Member_ID int, Callsign varchar(50), Rank int, Team int, Winner int, MinutesPlayed int, ParticipationPercentage int, Points int, Pass int)

-- GET ALL THE PLAYERS FOR THIS GAME AND INSERT THEM INTO THE TABLE (Must have played at least 5 minutes)
INSERT INTO @Players
SELECT DISTINCT
gtm.GameTeamMemberMemberID AS 'Member_ID',
gtm.GameTeamMemberCallsign AS 'Callsign',
0 AS 'Rank',
gt.GameTeamNumber AS 'Team',
gt.GameTeamWinner AS 'Winner',
gtm.GameTeamMemberDuration AS 'MinutesPlayed',
((gtm.GameTeamMemberDuration + 1) / (@GameDuration + 1) * 100) AS 'ParticipationPercentage',
0 AS 'Points',
-1 AS 'Pass'
FROM GameTeamMember gtm
INNER JOIN GameTeam gt ON gt.GameTeamIdentID = gtm.GameTeamID
WHERE gt.GameID = @GameID
AND gtm.GameTeamMemberDuration > 300

-- REMOVE DUPLICATES
DECLARE @DupCheckID int

WHILE (SELECT COUNT(*) FROM @PLAYERS WHERE Pass = -1) > 0
BEGIN
SET @DupCheckID = (SELECT TOP 1 Member_ID FROM @Players WHERE Pass = -1)

SET ROWCOUNT 1
UPDATE @PLAYERS SET Pass = 0 WHERE Member_ID = @DupCheckID
SET ROWCOUNT 0

DELETE FROM @Players WHERE Member_ID = @DupCheckID AND Pass = -1
END

-- UPDATE THE PLAYERS TABLE TO INCLUDE THEIR CURRENT RANK
UPDATE @Players
SET Rank = CONVERT(int,CONVERT(real,e.ELORanking) / POWER(50 + (CONVERT(real,e.ELORanking) / 100),1.15936))
FROM @Players p, ELO e
WHERE e.Member_ID = p.Member_ID

-- WORKAROUND FOR EARLY DURATION BUG
UPDATE @Players
SET MinutesPlayed = @GameDuration
WHERE MinutesPlayed > @GameDuration

-- PLAYERSTEMP IS USED TO COMPARE PLAYER TO PLAYER IN A LOOP
DECLARE @PlayersTemp TABLE (Member_ID int, Rank int, Team int, VsScore int, Pass int)
DECLARE @PlayerID int
DECLARE @PlayerRank int
DECLARE @PlayerWon int
DECLARE @PlayerTempID int
DECLARE @PlayerScore int
DECLARE @ScoreWin int

-- OUTER LOOP CONTROLS THE PLAYER WE'RE SCORING FOR
WHILE (SELECT COUNT(*) FROM @Players WHERE Pass = 0) > 0
BEGIN

SET @PlayerID = (SELECT TOP 1 Member_ID FROM @Players WHERE Pass = 0)
SET @PlayerRank = (SELECT TOP 1 Rank FROM @Players WHERE Member_ID = @PlayerID)
SET @PlayerWon = (SELECT TOP 1 Winner FROM @Players WHERE Member_ID = @PlayerID)

IF @PlayerWon = 0 SET @ScoreWin = -1 ELSE SET @ScoreWin = 1

-- OUTPUT FOR DEBUGGING
IF @DebugMode = 1 SELECT @PlayerID, @PlayerRank, @PlayerWon, @ScoreWin

INSERT INTO @PlayersTemp
SELECT Member_ID, Rank, Team, 0, 0
FROM @Players
WHERE Team <> (SELECT TOP 1 Team FROM @Players WHERE Member_ID = @PlayerID)
AND Member_ID <> @PlayerID

-- INNER LOOP ITERATES THROUGH THE OPPONENTS
WHILE (SELECT COUNT(*) FROM @PlayersTemp WHERE Pass < 1) > 0
BEGIN

SET @PlayerTempID = (SELECT TOP 1 Member_ID FROM @PlayersTemp WHERE Pass < 1)

UPDATE @PlayersTemp
SET VsScore = (45 + ((Rank - @PlayerRank) * @ScoreWin)) * @ScoreWin
WHERE Member_ID = @PlayerTempID

UPDATE @PlayersTemp SET Pass = 1 WHERE Member_ID = @PlayerTempID

END

-- OUTPUT FOR DEBUGGING
IF @DebugMode = 1 SELECT * FROM @PlayersTemp

-- PLAYERS SCORE IS THE AVERAGE OF THE INDIVIDUAL VS SCORES
SET @PlayerScore = (SELECT AVG(VsScore) FROM @PlayersTemp)

-- UPDATE THE PLAYERS TABLE
UPDATE @Players
SET Points = @PlayerScore,
Pass = 1
WHERE Member_ID = @PlayerID

-- CLEAR THE TABLE FOR THE NEXT ITERATION
DELETE FROM @PlayersTemp

END

-- THIS IS THE 'HELPER' THAT HELPS PEOPLE AT LOW RANKS INCREASE IN RANK
UPDATE @Players
SET Points = Points * ((CONVERT(real,Rank) + 1) / (15 + 1))
WHERE Winner = 0
AND Rank <= 15

-- OUTPUT FOR DEBUGGING
IF @DebugMode = 1 SELECT * FROM @Players

-- FAILSAFE FOR MISSING PLAYER RANK / POINTS
UPDATE @Players SET Points = 0 WHERE Points IS NULL
UPDATE @Players SET Rank = 0 WHERE Rank IS NULL

-- ARCHIVE THE INDIVIDUAL ADJUSTMENTS
INSERT INTO GamePlayerELO (GameID, GamePlayerELOMemberID, GamePlayerELORating, GamePlayerELOAdjustment, GamePlayerELOModifier)
SELECT @GameID, Member_ID, Rank, Points, 0
FROM @Players

-- CACHE KILLCOUNTS
DECLARE @AWE TABLE (Member_ID int, Kills int, Ejects int)
INSERT INTO @AWE(Member_ID, Kills, Ejects)
SELECT Member_ID, Kills, Ejects
FROM dbo.ASGSNetGetGameAWECount(@GameID)

-- MAKES UPDATING EASIER
DECLARE @ELOMemberID int
DECLARE @ELOWinner int
DECLARE @ELOScore int
DECLARE @ELOKills int
DECLARE @ELOEjects int

-- THIS LOOP UPDATES THE ELO TABLE TO TABULATE THEIR ACTUAL SCORES
WHILE (SELECT COUNT(*) FROM @Players WHERE Pass < 2) > 0
BEGIN

-- SEE HOW EASY THAT WAS
SET @ELOMemberID = (SELECT TOP 1 Member_ID FROM @Players WHERE Pass < 2)
SET @ELOWinner = (SELECT TOP 1 Winner FROM @Players WHERE Member_ID = @ELOMemberID)
SET @ELOScore = (SELECT TOP 1 Points FROM @Players WHERE Member_ID = @ELOMemberID)
SET @ELOKills = (SELECT Kills FROM @AWE WHERE Member_ID = @ELOMemberID)
SET @ELOEjects = (SELECT Ejects FROM @AWE WHERE Member_ID = @ELOMemberID)

-- IF THE PLAYER DOESN'T EXIST, INSERT A RECORD FOR THEM. (Fixes first-game-ignored bug)
IF (SELECT COUNT(*) FROM ELO WHERE Member_ID = @ELOMemberID) = 0
BEGIN
INSERT INTO ELO (Member_ID, ELORanking, ELOWins, ELOLosses, ELOExpected, ELOKills, ELOEjects, ELOCR)
VALUES (@ELOMemberID, 0, 0, 0, 0, 0, 0, 0)
END

IF @GameCounted = 1 AND @DebugMode = 0
BEGIN

-- UPDATE THE SCORE
UPDATE ELO
SET ELORanking = ELORanking + @ELOScore,
ELOWins = ELOWins + @ELOWinner,
ELOLosses = ELOLosses + 1 - @ELOWinner,
ELOKills = ELOKills + @ELOKills,
ELOEjects = ELOEjects + @ELOEjects
WHERE Member_ID = @ELOMemberID

END

-- MARK THE PLAYER AS PROCESSED AND LOOP FOR NEXT ONE
UPDATE @PLAYERS
SET Pass = 2
WHERE Member_ID = @ELOMemberID

END

-- IF ANYONE ACTUALLY WENT BELOW 0, ADJUST TO 0
UPDATE ELO
SET ELORanking = 0 WHERE ELORanking < 0

-- ARCHIVE GAME ELO HISTORY
DECLARE @TotalGamePoints int
SET @TotalGamePoints = (SELECT SUM(Points) FROM @Players)

INSERT INTO GameELO (GameID, GameELOTeamNumber, GameELORating, GameELOScore, GameELOOpponentRating, GameELOExpectedOutcome, GameELOAdjustment, GameELOCounted, GameELOReason)
SELECT @GameID, Team, SUM(Points), MAX(Winner), @TotalGamePoints - SUM(Points), -1, -1, @GameCounted, @GameReason
FROM @Players
GROUP BY Team
Last edited by Pook on Tue Dec 18, 2007 2:43 am, edited 1 time in total.
Image
MrChaos
Posts: 8352
Joined: Tue Mar 21, 2006 8:00 am

Post by MrChaos »

Much respect Pook
Ssssh
Post Reply