Gonna jump right back in to where we left off yesterday.
5. So what’s all this about smurf detection?
The short of it is that there’s many reports of new accounts getting sent into high and even very high games after an extremely short time of being active. Under 5 games in a lot of cases.
Let’s first examine why something like this isn’t usually possible.
Going back to the LoL example, let’s say we have a new ranked player who belongs at 1800, which is pretty close to the platinum ranking. How many games would it take to reach 1800 if we assume +20 per win and -20 per loss?
The answer of course is that it depends on the win rate, but if matchmaking is working properly the win rate shouldn’t be a constant. For the sake of simplicity we’ll claim that the win rate of the player by the time they hit 1800 is 60%, which implies their win rate at 1200 might be something closer to 65% or possibly even higher.
So at a 60% win rate the player should gain 40 rating every 10 games for an average of 4 rating a game. For them to gain 600 rating they would need to play 150 games before they hit their proper rating.
150 games is a long time. At a generous estimate of a half hour a game that’s 75 hours of playtime before you hit your proper rating. This isn’t that far fetched. Microsoft’s description of their Trueskill system claims that in a 4v4 game it can detect each player’s rating in 46 games. But it goes on to say that “The actual number of games per gamer can be up to three times higher depending on several factors such as the variation of the performance per game, the availability of well-matched opponents, the chance of a draw, etc.” Dota can’t have draws, but Dota and it’s genre companions have huge issues with the other factors.
Variation of performance per game is a no-brainer. There are nearly 100 heroes. They have ideal line-ups, and less ideal line-ups. They have vastly different playstyles, and your average player is going to not be equally skilled with all of them. From the perspective of consistency the game is a nightmare, and while this makes for a better game it wreaks hell on effective matchmaking.
But think about the “availability of well-matched opponents.” If it ends up taking nearly a hundred hours for the game to figure out your skill level, then that’s also true for everyone else in every game you get matched with. If the average playtime of the 10 players in the game is under 25 hours, the matchmaking is essentially a weak educated guess. This undermines the effectiveness of the results, which makes the matchmaking take even longer to detect player ratings, which undermines the effectiveness of even more future games, and so on, and so on.
So wouldn’t it be ideal if we had a way to estimate a players skill based on their individual performance without having to even consider the match results at all?
6. But you can’t do that! Stats without context are meaningless!
Not technically a question, but it’s a legitimate complaint. Dota is an extremely complicated game. We have detailed stats, but the value of any particular stat is extremely situational because of all the feedback loops built into the game. Judging players exclusively by CS/min or K+A/D is dumb, and I do not in any way believe this is what Valve is doing.
So look at it a different way. Suppose we have detailed stats of literally millions of games, and we now have a good idea of the relative skill level of some of our players. Let’s make some measurement, say creep score per minute by 5 minutes on a hero by hero basis. If we know that 99.5% of the time, any player achieving 25 CS by 5:00 with Lina happened to have a MMR of at least 1500, and in your first 3 games at 1200 you do this all three times, we could make a pretty good argument that you deserve to be at least at 1500. You might be a 1500 player. You might be a 2000 player. We’re not trying to make an exact prediction of your final rating. We’re just trying to make a prediction of your minimal expected rating.
Is CS/min by 5 minutes an effective metric? I have no idea. It’s just an example of how the system might work. The actual metric or metrics could be literally anything, and would have to be backed by a ton of data, but hypothetically something like this could work and could be a potential explanation for how Valve is rapidly accelerating the MMR of certain players.
Will the system miss players? Sure, but the rate of false negatives isn’t that big of a deal. Those players will move through the lower ranks more quickly by virtue of having less underrated players to randomly get matched against. False positives are, in my opinion, more problematic so I’d expect the system to be very conservative.
Is it fair? Who cares. MMR is a system to make good matches. It’s not a reward scheme, and it doesn’t tell you your value as a player. By keeping it hidden Valve can do whatever it wants to the math in order to create good matches. They can also make hidden adjustments to the system without having players complaining why they’ve suddenly lost 100 rating in the last week. Hidden MMR is for the best, and I hope it continues indefinitely, or at least until team matchmaking is in play.
Whatever the mechanism, a smurf promoter is a good thing for the genre, provided it can be calibrated properly. Now that that’s out of the way, let’s move on to some other subjects
7. Why does Valve keep me at 50% by matching me with worst players as I win?
If you’re in High matchmaking or below, this isn’t what’s happening. I mean what, do you think Valve has a list of players who they’re secretly conspiring to keep out of Very High by handicapping them with 4 of the faceless mass of terrible players they keep on a server somewhere?
Here’s what happens. You enter the game at the starting rating. You win doing whatever because the quality of play at the starting rating is terrible. Eventually your rating rises and so does the ratings of the players on both teams. Your current level of play is no longer high enough to effortlessly carry, so you start looking for reasons why you’re suddenly no longer winning (because it couldn’t be that you’re just not as good as you think you are). Maybe you find your teammates making mistakes –which happens a lot because the people you’re playing with still aren’t very good, but neither are the opponents– or maybe you just do a Dotabuff search and pick some meaningless metrics to explain why your team never really had a chance.
But the real problem is that you’re simply not good enough to effect the outcomes of games at the MMR you’ve reached. Every game feels like part of a random walk, and it probably is. But it’s a random walk because you are missing opportunities to effect the outcome of the game.
“But my opponents always feed!” Maybe they feed because you’re a non-issue because they’re playing 4v5 for the first 30 minutes. You see this a lot with players who crush the lower brackets with something like jungle Naix because literally no one can farm in the lower brackets. But against decent opponents jungle farm isn’t impressive, and staying in the jungle puts your team at a disadvantage that decent opponents will punish. Maybe it looks like your teammates were feeding, but in reality your opponents have just realized that the most consistent way to win in a pub is to force the other team to feed, and until you adapt to that you’re not going to climb any higher.
Now, if you’re in Very High, you might have a case. If you’re at the top of the distribution, you’re there with little company. If matchmaking can’t find appropriate opponents it either has to skyrocket your queue times or reach downward. This should improve as more invites go out and the playerbase expands. Valve has to deal with a tradeoff between tightly matched player ratings and queue times. In the middle of the distribution this isn’t that big of a deal, but at the top end it’s a huge complication that will be alleviated as more players enter the system.
8. How do High and Very High compare to the rating systems in other games like LoL and HoN?
Going to assume that 2.3% of the population is capable of solo queuing into Very High games, and about 16% of the population solo queue into High or Very High rated games. This is my best guess based on the match distributions I have right now.
I found an unofficial measurement of the MMR population for HoN. Based on their data I would estimate that High is roughly equivalent to a 1650 rating in HoN, and Very High is 1800.
LoL is…messier. The only LoL information seems specific to ranked matches, which would leave out the many players who do not play ranked matches (and likely trend to the lower end of the bracket). I previously said that I thought High was equivalent to 1350, but sources have since been conflicting and I no longer have confidence in any of them.