An Extremely Unofficial Dota Matchmaking FAQ part 3

Originally this was going to be just a two part series, but I realized the other day that there was a big subject I skipped over entirely: premade groups.

9. Matchmaking keeps putting me up against premade groups when I’m solo.  How is this fair?

When dealing with group queues, Dota-like matchmaking systems have a mechanism that will effectively inflate the rating of the pre-made group to compensate for their organizational advantage.  Is this fair?  Like everything else in matchmaking not perfectly.  But it attempts to be fair enough.

10. Well I don’t like it, why can’t Valve just implement a solo queue?

Some of the people who play Dota, or who might play in the future, will only get into the game if they can play with friends.  Queuing with strangers can be extremely stressful, sometimes in the long-term, sometimes just when starting out.

If Valve were to make a solo queue, it would effectively be a competing ecosystem to the current queue.  It’s entirely possible that the group queue would die out and with it a lot of potential players.  Please, let us just accept as fact that this is an unacceptable outcome from Valve’s point of view without even having to debate about what it would do to the health of the community as a whole.

The death of the group queue is not guaranteed of course.  League of Legends has accomplished this with its unrestricted Normal matchmaking and its max of 2 people per party Ranked matchmaking.  It’s not entirely out of the realm of possibility that Valve could do something similar.  For example, they could make CM a purely solo queue mode that unlocks once you reach a certain rating.  Whether something like this would work, backfire spectacularly, or be generally alright but inferior to some other option is an insanely difficult question to answer.

So whatever Valve’s intending, it’s a good thing they’re taking their time to test their expectations of how the population might respond.  Once you put out a social system it can be extremely difficult to roll it back, so you want to be confident that the design you end up going with isn’t an unexpected disaster.

11. Does Dota2’s matchmaking handle group matchmaking well?

Possibly not.  If there were only one reoccurring complaint about matchmaking with a shred of legitimacy, it would be group matchmaking.  To be fair though, I’m willing to consider this portion of matchmaking a work-in-progress.

For starters, trying to evenly handicap every premade group is a really tricky problem.  If you’ve played in organized or semi-organized groups in really any game you know that there are always going to be vast differences in general skill level, skill distribution, and overall coordination.  Some groups are a bunch of highly skilled players goofing off.  Some groups are all over the place skillwise but ‘tryharding’ as the parlance goes.  Others are low skilled and possessing terrible coordination, but are deluded enough to believe that neither is the case.

So suppose we put a really basic rule in the system to try to handle group matchmaking.  For the sake of simplicity we’ll only allow fully premade groups, and to find the rating of the group we’ll take the average of the group and multiply it by x.  For any value of x, we could find the collective win rate of all the groups in the system, and then set x so that the group winrate is 49%.  That should be fair right?

Unfortunately, what you’ll most likely see is most of groups at 44-54%, with a bias towards the lower end.  You’ll have a decent number of groups that are much lower.  Finally you’ll have a handful of groups vastly over-performing at +60%.

That won’t do, so we’ll just set x so that the group winrate is 45%.  Now the overperformers are less prevalent, but the underperformers are much more severe and quitting the game because they feel it’s rigged against them, and it arguably is!

Dota’s matchmaking rules are almost certainly a bit more elegant than my average*x, but something similar but less pronounced could still be happening in the current matchmaking.  It would be an interesting study to see the rate of group queues by skill bracket.  I unfortunately cannot accomplish this with my data alone, though it might eventually be possible through a combination of a skill separated sample plus a general match database like Dotabuff.  But given the patterns of complaints about group queues, I wouldn’t be surprised if group queuing is much more prevalent in Very High than it is in Normal.  If Very High is underpopulated right now due to restricted invites, and the matchmaking handicap for groups is too top end lenient, it could lead to a group queue arms race that becomes very frustrating to solo queue into.

But there’s a lot of ways to alleviate this.  More players is the most obvious one.  Having a more populated top end would allow Valve to make matchmaking more restrictive without skyrocketing queue times.  I’d be extremely surprised if the relation between queue length and player activity wasn’t one of the first statistics they looked into regarding matchmaking.

Further under-the-hood changes to matchmaking are also likely coming, and many could even be in place as we speak.  People argue over whether Dota should be considered a beta or full release, but in reality a component like matchmaking is never going to be completely ‘done.’  It’s a long, iterative process that also has to adapt to changes in the underlying playerbase.

But the most intriguing option to me is adding more features that give players a greater option of things to do outside of matchmaking.  After all, Dota is what it is today because of the profession scene that grew out of player-run in-house leagues.  Adding more features that enable players to easily run and expand new amateur leagues would be far more healthy for the game than ingraining a culture of solo-queue stardom.

One possible addition I’d like to see is a sorting method for custom games that arranges players by randomly selecting one of the most matchmaking appropriate compositions available.  It would give in-house games an impartial arbiter for the teams and hopefully cut down on both instances and accusations of teamstacking.  I’d also like to see the player cap raised on teams because in my experience amateur teams in online games are not capable of regularly hitting near-100% attendance.

In the slightly longer run I’d like to see toolsets that make it easy to design, run, and coordinate player run leagues inside the client.  Primarily standard team vs team leagues with a limited variety of options (double elim tournament, divisions, fluid or not fluid ladders, etc.)  Players would hopefully form subcultures around different leagues that could have their own match listing and history like the major tournaments, and with an automated weekly highlight reel.  Then you could have larger invitational tournaments that go out to the champions of a league cluster, and through this you could help encourage the formation of future pro teams.

12. Why doesn’t Valve just implement team matchmaking?

Good matchmaking is dependent on being able to find good opponents.  The max possible opponents for a team system would be the current active players divided by 5, but realistically the actual number would be much smaller than that since many players either do not have the interest or social connections to find a team.

Compounding this, you have to provide the teams that do form good matches or they won’t stick around.  If you form a team that does nothing but lose, there’s a good chance your players will get sick of the extra effort and just go back to solo queue.  In isolation, whatever, but this doesn’t happen in isolation.  Instead we get a domino effect where a portion of the lowest teams quit, which then drives down the win rates of the teams just above them who then quit in droves for the same reason and so on.  If you want a healthy long term environment you have to at least limit this tendency towards an extinction event.

One potential solution, albeit one with a significant downside, would be to activate team matchmaking with limited time windows staggered across week for different regional clusters.  This would hopefully promote enough player density during the limited uptime that match quality would stay high and people would continue queuing.  This density might also be further encouraged with accelerated item drops, but Valve might already be testing the effectiveness of this with the group battle point bonus that’s currently active.

For what it’s worth, I’m of the opinion that any external impetus to queue needs to be universal and not one of those “top x% of players get _____” deals.  The players you need to encourage the most are the ones who know that they won’t be in the top x%.

Now capping everything off with a couple general questions…

13. So why does Dota 2’s matchmaking enforce a 50% win percentage?!

It doesn’t.  Take a look for yourself.  And if they don’t count because they’re too good or group queuing or whatever else I can provide you with a couple of profiles that manage to hit 40% and even much, much lower in over a hundred games.  Not in public though.  No casual-shaming on this blog.

The fact of the matter is that for the vast majority of the playerbase, there will be a rather large group of players who will all go 50/50 when matched against each other.  It sometimes takes the system some time to sort everyone into these groups, but it eventually does a good enough job.

Often you’ll see people say that Dota 2’s matchmaking should be replaced with some system their old hangout in original Dota used, something like you got promoted to a higher league if you managed a 60% win rate and then an even higher league at 70% and players were only matched within their own league and it was fair and worked and wasn’t communism and blah blah blah.

Guess what?  Those players were the big fish in a small pool.  They might still be quite capable of being pretty good players in Dota2, but we no longer need to wait for them to get to a 70% win rate before we realize they shouldn’t be matched against newbies.

14. But matchmaking always gives me all the feeders!

No it doesn’t.  It couldn’t if it wanted to.  So there’s two actual possibilities.

1. You don’t ever feed yourself because you play constant hyper-conservative strategy, so your teammates die more often because they never have anyone backing them up.  Mindlessly safe play does not win high level pub games.  You win by identifying enemy weaknesses and exploiting them, and if you’re consistently doing neither than you are as much of a handicap to your teams as any of the feeders that you identify.

2. You do feed yourself, but it’s different somehow when you feed because you got ganked, or someone didn’t call MIA, or everyone picked carries so you didn’t have support (because you, of course, picked a carry).  “The only justified feed is my feed!”

If you really want to get to high or very high matchmaking, focus on stepping up your game instead of throwing other people under the bus.  I’m 100% serious.  I know a guy who is completely new to Dota 2, is not a great last hitter, and definitely has distinct gaps in his knowledge of the game.  Despite all this, he plays fairly smart, and has managed to get into High matchmaking predominantly through solo queueing.  I’m half convinced that the biggest reason for this is he starts out every game by saying hi to everyone before the pick phase and wishing them a good game.  Setting a positive atmosphere (or at least a not negative atmosphere) goes a hell of a long way towards winning.

To be able to do that you need to accept that most of your teammates will not be idiots.  They will have flaws, just like you have flaws.  Don’t panic if they die to something, and don’t lash out if you die to something and you think they could have somehow prevented it.

Or just get better at last hitting.  That seems to work for a non-trivial number of belligerent manchildren.

15. Bu…bu…but Russians!

No.  Just no.

Also, “it’s not a race it’s a nationality” will never be an interesting or compelling argument.

Now that I’ve finished trolling every Dota2 matchmaking thread in existence, I’d like to welcome you to chime in in the comments section if you have some further insights regarding matchmaking.  There’s a conversation going on here that adds some extra details about the Trueskill system and the patterns of smurf accounts.  I tried to keep things as general as possible, so there’s a lot of details still to be discussed.

That includes pointing out things that may have been wrong, or at least misleading.  If you don’t want to leave a comment you can contact me at dotametrics@gmail.com, and I am willing to revise or correct as proves to be necessary.

Also be on the lookout for another release either tonight or sometime this weekend!  Hopefully a lot of people will have fun with this one.

Advertisements

5 Responses to An Extremely Unofficial Dota Matchmaking FAQ part 3

  1. emdash says:

    Several months ago there was a quote from a Valve employee that indicated that they had studied matchmaking results and found that the best player on a team had more impact on the result of a match than the worst player on the team. Does anything you’ve observed indicate that they have somehow integrated that data into the current matchmaking system?

    • phantasmal says:

      Without having detailed player histories I couldn’t begin to make a call on this issue, and even then it’d be hard because I’d at best have a rough estimate of the individual MMRs. Working through a sample is fairly adequate for an aggregate analysis, but there’s just too many holes to do any kind of an individual player analysis (which includes things like the questions about average player win percentages that have popped up).

      Their sentiment makes sense, and from what I’ve heard the matchmaking for premade groups is weighted towards the player with the highest MMR rather than any kind of a straight average. But I have no information whether this effect holds true for pure solo queue matchmaking.

      • emdash says:

        Thanks, I figured that might be the case. The one thing that worries me is that they may not have accounted for the types of heroes that the best player in MMR terms likes to pick, so a person who frequently picks support, solo queues, and ends up as the highest MMR on a team may be disadvantaged as compared to someone who likes to pick semicarries. In other words, I wonder if they’ve weighted for the influence of hero types on game outcome. Might be a pipedream.

      • phantasmal says:

        It would be an interesting thing to test for. You could just grab all the recent matches of a variety of hard support characters, make a metric like (Support MMR)/(Avg of Team MMR) and see if the win rates consistently go down as the metric goes up. The analysis would likely benefit from having a separation between pre-made groups and entirely solo queue groups as well as being able to filter to different MMR ranges.

        My general feeling though is that people underestimate the potential effect a support can have on the game because expect support play to be passive. Good support play can contribute to winning multiple lanes, maintaining rune control, and finding farm sources that would have otherwise gone to waste. But players (particularly on the lower end of the spectrum, though this never completely goes away) moan about having to play support and then just devote their entire game to a single lane that’s often a lost cause with or without their presence.

        I was watching Merlini the other night, and he had a team with some extremely questionable picks — Phantom Assassin/Slardar/Templar Assassin/Nyx Assassin. Many people faced with this would complain, pick some random traditional support, and then get stuck in the suicide lane with a Phantom Assassin and probably have an extremely frustrating game. He instead went all in, picked Bounty Hunter, bought some tangos and an Orb of Venom, and just camped the enemy Nature’s Prophet’s jungle, killing him 3 or 4 times in the first 10 minutes. It’s certainly an unconventional “support” strategy, but he influenced the game without taking any lane CS and even without taking ambient lane exp. I don’t think you need to not be the highest MMR on your team to exhibit that kind of influence with regularity, but it could certainly be the case that effective support play is extremely rare and that many people make it to the higher brackets without ever having to develop it.

  2. JRR says:

    Reading this makes me think you don’t play dota

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: