Update to Dota’s Skill Bracket Size

I ran with 80/16.5/3.5 as a size estimate because I couldn’t find a statistical structure that better explained what was going on.  Turns out I was a bit blind.

My new theory is that the skill brackets are determined using the standard deviation of the rating distribution.  The slice between the mean and one standard deviation above the mean is 34.1% of the distribution.  The slice below the mean is 50%.  34.1+50 is 84.1, and the difference between 84 and my estimate of 80.5 can be explained by the High and Very High brackets having players that are, on average, more active than the Normal bracket.

This theory puts the High bracket cutoff at 2 standard deviations above the mean, or 13.6% of the population.  The remaining 2.3% is the Very High bracket.

Advertisements

2 Responses to Update to Dota’s Skill Bracket Size

  1. xdv says:

    Very interesting post. So you have a script that scrapes their stats database and tries to figure out how many games are being played?

    This is of course assuming that all players play equally.

    In HoN my best effort put me at the 90th percentile in skill – they published a total list of ranked players (total number of players about 250,000 back in the day, and I would be ranked 25,000 from the top)

    In DOTA2 I’m in the Very High Skill bracket despite taking no effort to “level up” my account, I’m frequently messing around and playing with low rated friends. I say I’m in Very High Skill because the last 5 solo queue matches I’ve had were tagged Very High Skill: the rest of the time I’m playing with other friends in Normal bracket.

    From my personal experience in HoN it felt like DOTA2 only needed you to be in the top 20% to be in Very High Skill but your stats say otherwise. Maybe my experience in HoN was distorted by the fact that most high rated players also had “alt” low rank accounts to mess around with, artificially inflating the lower ranks. (conversely, it would be impossible for a low rank player to have an alt high rank account – they can only have an alt low rank account, doubling the count of low ranks)

    I find it hard to believe I am in the top 5% of skill in DOTA2, unless the player skill level in HoN is much higher than DOTA2.

    • phantasmal says:

      Yeah, that’s a fair description of the script. Due to time constraints I could only run over about a days worth of games. I wanted to grab some actual game data and Valve’s API was getting slammed heavily at the time so I had to pick between the two whenever I could get through to the servers. Once they bring it up again I’d like to add in more stretches of time to confirm the percentages, and also to see if the percentages stay constant over the months or if they shift.

      As for the discrepancy between Dota 2 vs HoN, there’s two possible explanations I can offer.

      1. HoN’s playerbase is more top heavy when it comes to skill. It’s pretty believable. I don’t know how much of a casual scene HoN still has, but Valve is likely targeting a lot of their invite waves towards newer and less experienced players, and it’s still got a bit of new game kick to it leading to less low-end burnout.

      2. Dota’s matchmaking cannot consistently make games full of players that are exclusively in the top ~2.5% within a reasonable amount of time., so it settles and makes games where, say, each team has 1 to 2 top 2%, and then fills the rest of the teams with 3-10% players. These games still average out to being in the Very High bracket because the pure top 2.5% games are too rare to fill the bracket on their own.

      For what it’s worth, everything I’ve seen points to one of Dota’s big matchmaking improvements being that it can ‘catch’ and promote smurf accounts much more quickly than either HoN or LoL, so you’re likely right that HoN’s brackets are more distorted. Duplicate accounts in general might also be comparatively more rare right now in Dota, which could further distort any comparisons to HoN.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: