Reputation and Leaderboard

Share #feedback at RocketChat


  • Redefine reputation to make sense with the new tournament design

  • Fix reputation bonus to be sybil resistant and remove phone verification

  • Increase user profitability, especially during burn weeks

Design goals

  • Reward long term performance

  • Balance experience for new and existing users

  • Don’t accidentally reward high volatility aka p,1-p

  • Don’t allow staking on past performance


  • Filling with 0.4 punishes new/flaky users but give them a chance to catch up

  • As before, non-stakers can be on the leaderboard

Example: Leaderboard for 156-137 window

Reputation bonus

  • Stakes now get paid out “twice”, the second time without any burning!

  • Guaranteed payouts for users with top long term performance

  • Scores are averaged across all tournaments, stakes are summed

Example: Reputation bonus for 156-137 window

To calculate the bonus, we first rank everyone by mu. Then we take the stake amount in round 1/20 and select the top 1000. As you can see, stochastic_geometry_1’s stake is only partially selected. The 50% bonus percentage is then applied to the selected stake amounts.


Sybil attack

  • % back on stake is sybil resistant, so we can remove phone verification

  • Top “ranks” can be filled by models by the same user, but gives no material advantage

p,1-p attack

  • A p,1-p attack may be profitable if average AUC of either model can beat the threshold

  • In practice, it is difficult to predict threshold and the risk of it being too high makes it problematic to carry out this attack


  • Announcement on Rocket.Chat May 29

  • Leaderboard and learn pages updated before next round 162 opens (Jun 1)

  • First bonus paid at resolution of next round 158 (Jun 4) using window of 158-139

  • No user changes required