Pulse Leaderboard and Rating System
Do you wonder how the leaderboard works? What determines your current rank, and what can you do to improve it? In this article, we unveil the details and make it transparent for all competitive players and our community. This includes the original rating system, the replacement, and a new concept, Activity Decay.
The original system: Elo
When we launched TTA Pulse just one month ago, the goal was to release a minimum valuable product (MVP) as quickly as possible, so players can start using it and we can receive community feedback. Than we can start adding more features based on this feedback, and fine tune the platform based on demand and data. We knew that the rating system would be one of things to need adjustments, so we launched with a simple solution: the Elo rating system with a couple of extra bonus points per win. This works, but Elo has some disadvantages. The issues are well-documented online, and you can read about them by searching for ELO rating problems. We can do better.
Other rating systems
In our research for the best solution, we looked at a few options:
For details on each system, follow the links to the Wikipedia articles above. As usual, everything has advantages and disadvantages.
The considerations
When selecting a rating system, it's important to ask the right questions.
How many matches are needed for accurate ratings?
How does activity relate to rating?
How punishing are losses to newly-joining players?
We also want to be mindful of the season length, and what that means for late-joining players.
Other goals: prevent opportunities for abuse, use the ratings for the internal matchmaker, and minimize the complexity of the implementation.
The winner: Bayesian Interference
We decided on using a Bayesian Interference system for internal ratings. It allows us to track the degree of confidence in a player's rating, to reward those who have played enough to get a precise read on their skills. But this alone doesn't sufficiently address our goal of encouraging consistent activity through the season.
A second challenge: inactivity
We noticed a major theme while monitoring the leaderboard. This is a common issue on many competitive platforms: When a player reaches the #1 spot, they stop playing to preserve their ranking. The reasoning makes sense: Given the status, there is less to win and more to lose. Unfortunately, this leads to a stale environment, where it's harder for other players to catch up because they don't get the direct opportunity to defeat the leader. And when the rankings finally change, the problem often repeats itself.
The solution
It took us a lot of discussion and research to come up with the right solution. We studied other platforms, such as StarCraft2, Dota 2, and CounterStrike, and considered bonus pools, win bonuses, and activity bonuses. Finally, we decided on Activity Decay, a diminishing decay system.
Here is the concept:
- After two days pass without playing a match, a player loses a certain amount of decay points from their rating, initially 70.
- After decay begins, each additional day of inactivity causes further decay, diminished by 10 each day.
- Simply win a match to recover the most recent day of decay.
Here is a specific example:
- Day 1: The player doesn't play a match, but two days of inactivity have not passed. No change.
- Day 2: The player doesn't play a match, so they lose a decay step (70 points).
- Day 3: The player doesn't play a match, so they lose another decay step (60 points).
- Day 4: The player plays a match, but loses. Nothing is added or subtracted from their decay. Their internal rating adjusts as usual.
- Day 5: The player doesn't play a match, but two days of inactivity have not passed. No change.
- Day 6: The player doesn't play a match, so they lose another decay step (50 points).
- Day 7:
- The player wins a match. The latest decay points (50) are recovered.
- The player wins again. The next decay points (60) are recovered.
- Day 8:
- The player plays another match and wins again. They recover their final decay step (70 points).
- The player wins one more match. All of the decay is already recovered, so no decay points are added or removed.
Hopefully this helps to illustrate this simple concept. Here are some other details:
- The decay is tracked separately from the internal rating, then combined to determine the leaderboard rank.
- Since the decay is diminishing, it's capped at zero on the final day (and fully recoverable with 7 wins).
- Decay is always recoverable by winning matches, so the effects are temporary. The internal rating determines the final rankings between active players.
- While the internal rating is tracked from the moment player joins the platform, decay is only applied to Master Division players.
- Decay does not accumulate during the final week of the season.
Other Leaderboard changes
This release brings several new features.
- The leaderboard is publicly accessible (no login) at https://app.ttapulse.com/leaderboard.
- The leaderboard updates once every 24 hours.
- The leaderboard displays player ratings from the time of the last update.
- The leaderboard shows the direction of recent changes in rank.
- The leaderboard includes only Master Division players.
- The leaderboard show progress over time on a new ranking chart.
At the end of the season (Sept 30), after we announce the winners, we will also publish a report with statistics on which players had the most consistent ranking throughout the season.
We've worked hard to design this system. As promised, we want to be transparent about how everything works. We hope you truly enjoy it.
Subscribe below if you are curious about how we designed the other major component of this system: the matchmaker.
As always, looking forward to match you Through the Ages!