Very High Skill Hell is a REAL place where you WILL be sent at the first sign of success.

Some of the more conspiracy-prone Dota discussion areas are pretty skeptical that Dota2’s matchmaking is effective, and in some extreme cases they doubt it even exists. So I did a bit of analysis on the types of games each bracket and can demonstrate that there are definite trends in player performance between the brackets. This in no way proves that Dota 2’s current matchmaking is flawless, but it does suggest that it’s been successful in sorting by player skill level.

One inconvenience you’ll have to forgive, due to technical difficulties these charts are legendless. In all the charts, the red line is the Very High bracket, the green line is the High bracket, and the blue line is the Normal bracket.

One other tiny, almost irrelevant factor: the ends of the distribution tend to be larger as an easy way of handling extreme outliers without warping the graph. As far as I’ve been able to determine, this simplification is entirely inconsequential.

Disclaimers aside, I give you the distribution of games by duration:

The mean match duration for the sample is:

vh = 38.46 minutes
h = 40.22 minutes
n = 44.10 minutes

Normal games, on average, last significantly longer than games in both the High and Very High bracket. No real surprises here. More highly skilled players tend to emphasize push and gank strategies over farm strategies, and they’re more capable of closing games once they have a significant advantage.

Moving on, we have average GPM:

The mean GPM for the sample is:

vh = 344.75
h = 330.19
n = 307.27

Again, no real surprise. From what I’ve seen, last hitting is the skill that is the most consistently improved as you move up in the skill brackets. You often see complaints about low skilled players leaking into the High and Very High brackets. In most cases, this is simply the result of group queuing, but it’s possible that there’s a class of player who wins more often than not in the Normal bracket simply by having superior last hitting mechanics than the competition. This moves them into the higher brackets where their relatively undeveloped game sense gets exposed by the shift towards more aggressive strategies.

Now for XPM:

Mean XPM in the sample:

vh = 445.51
h = 435.67
n = 422.20

This increase is less pronounced than GPM, which again, isn’t very surprising. An XPM increase mostly reflects an enhanced game sense for finding time to farm. This might entail using a TP scroll to a tower that’s about to receive a creep wave. This might come from remembering to stack and pull as a support. This might come from knowing when and how to use neutral creeps to enhance your farm. All these are important skills, but they’re skills that are already reflected in GPM. GPM adds in the mechanical aspect of last hitting, so GPM will be the stat that’s more sensitive to player skill.

Finally, we have Deaths per Minute:

Average Deaths per minute:

vh = 1.72
h = 1.65
n = 1.60

Of all the statistics, Deaths per Minute sees the smallest shift between the Normal and Very High brackets. Does this contradict what I was saying about Very High players being significantly more aggressive? No. Instead what’s happening here is that Very High players are better at playing aggressive, but they’re also significantly better at playing it safe. The skill increases counteract each other, but aggression gradually edges out safety to make for a slight trend towards higher kill scores.

One possible curiosity that stands out is that in this graph, Normal and High have very similar lines. In all the other graphs, High and Very High had the most similar lines and Normal was the outlier. A tentative interpretation is that as a player learns more about the game and moves up in the brackets, they’re more likely to first improve their farming skill and death prevention than they are their aggression and ganking. Exceptions would exist of course, possibly in great numbers, but the cautious farmers would still be the majority in the High bracket and therefore have the largest statistical footprint.

With this out of the way, it’s likely time to move on to one of the more interesting, and possibly most misleading metrics, creep kills per minute.

Advertisements

6 Responses to Very High Skill Hell is a REAL place where you WILL be sent at the first sign of success.

  1. xdv says:

    Why do you call it Very High Skill Hell? It’s probably the only place you can get a decent game of DOTA. I find even High Skill matches quite intolerable sometimes…

    I’ve found the matchmaking works very well, as I’ve played in all sorts of skill brackets (queueing with friends of dramatically different skill levels). I guess people like me are probably breaking the system though… dragging a 10 game player into a high skill game, or myself entering a low skill game and stomping.

    I think “Top GPM” per team would be a good metric to consider as well, as Very High Skill tends to have players who are more willing to allow a carry space to farm, while in Normal it’s every man for himself.

    • phantasmal says:

      The ‘Hell’ is mostly just to drive home the Portal reference. That being said, I have very little sympathy towards people who complain about being stuck in Normal matches. If you deserve to be in higher level matches then Normal matches should be a fun playground where you can try outlandish strategies and still win more than you lose. If you’re don’t belong in Normal than you should be enjoying it while you can. If you think you don’t belong in Normal, but can’t get out of Normal then you really need to reevaluate your play on either a strategic, tactical, or mechanical level, and quite possibly all three.

      Group matchmaking is an interesting conundrum, but it’s unfortunately one I can only offer baseless speculation for, as the stats I have available can’t differentiate between grouped and solo queue players. My suspicion is that Dota has an MMR handicap of some sort against group queue teams, which is sensible and exists in other similar MMR systems. The problem is that some pre-mades outperform this handicap while others underperform it, and the system cannot yet detect which kind of group it’s dealing with. Group queues could also potentially make the system more streaky, but to test for something like that I’d need a better way of finding and testing particular players.

      Top GPM per team is coming. To be more exact, I switched to Creep Kills per Minute for this because teams more explicitly share farm than they do Gold. And yes, Very High skill does have a distinct tendency towards funneling their farm to their 1 and 2 slots.

  2. xdv says:

    Someone once did a large and beautiful looking infographic combining a bunch of information like this for League of Legends. Do you mind if I take a stab at it for this, if it only gets published from your site? My day-job is in future vehicle strategy (finance) and we do a fair bit of work with slides and big data.

  3. xdv says:

    I don’t think there is a handicap for group matchmaking. I think the people at Valve just use a simple rule: if a group of 2 is queued for matchmaking, the opposing team must also contain a group of 2.

    The “unknown factor” – being the teamwork advantage – then gets cancelled out, without Valve needing to be able to calculate this mystery number.

    I’ve found this to be quite consistent, by checking the match histories in DOTABUFF, as you can see the game histories of the players and see who they frequently play with. This is how the guys at HoN handled it in the past, I didn’t play LoL long enough to figure it out.

    In any case this would be a self correcting factor anyway, if someone played in groups a lot their MMR would be artificially inflated to reflect that.

    • phantasmal says:

      Valve definitely attempts to match groups with similar pre-made compositions, but matchmaking will ignore this when waiting for a sufficiently good match would drastically inflate wait times. I know there have been plenty of matches where I could find evidence for a group on one team with no evidence of a group on the other, though they’re definitely less common than the reverse.

      Calling it a handicap might be a loaded phrase, but one of the ways I think Purge and others have described it is that the matchmaking rating for the group is weighted towards the highest individual rating of the group members. My experience suggests that something like this is true because when I’ve been able to watch a streak of games from an individual player near the border of a bracket change, the best predicting factor for a move upwards in the bracket is the existence of a large group queue on either team.

      It also makes sense to do this, to a point. I remember it being said that Riot does something similar with regard to group queue in ranked games (which is capped at a 2 person group) and then said that the average win rate of group queues was 45%. At the same time we know for a fact that there are many people who group in ranked and absolutely crush the expected 45% win rate. The complication we run into is that the teamwork advantage factor that you bring up is not a constant. Some teams are taking the game seriously. Some aren’t. Some even use the fact that they’re in a team as an excuse to take it less seriously than they would normally. “My friends are way more chill than random matchmaking people so I’m going to use group queue to practice Meepo.”

      Another much more specific example, one time I was playing a round of Dota, and things were going well. Everyone worked together and we systematically dismantled the other team. One person suggest that we form a group after the game and queue together and everyone goes along with it. We go into the next game and we get absolutely crushed.

      My suspicion is that matchmaking viewed our second team as much more competent than we really were because we now appeared to be a 5-man premade. In reality our organization hadn’t changed at all, but because we grouped outside of matchmaking the system had to assume we might have a teamwork advantage that we didn’t really possess.

      (For what it’s worth, I’d like to see an option in solo queue that is essentially, “Keep playing with these players without being treated as a premade” as a way to foster more social interaction. Of course I can’t vouch for the long-term practicality of this particular solution, but it’s a direction I’d like to see explored.)

      I guess it also should be said that I’m not really critical of any of this so much as I am sympathetic. Trying to deal fairly with pre-made groups of vastly different skill distribution, organizational complexity, and size while maintaining short queue times is a challenge of nightmarish complexity. I’m sure there are improvements that could be made, but many complainers don’t even begin to appreciate the scope of challenge that is good, casual, group matchmaking.

      As for doing stuff with the data, knock yourself out. I’m not terribly protective, and I know that everything I produce is lacking when it comes to visual kick.

  4. xdv says:

    Could you send me an email at my address so I have a means of getting in touch with you? I assume you can see my email from my comment details. I’ve got a mockup of an infographic but it would need some data you may not have mined yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: