DBR vs Valve

I’ve received a few questions about my thoughts on the upcoming DotaBuff Rating.  Most of my general feelings about personal public matchmaking stats are covered in the recent Insignificance of Pub Stats series, with MMR being discussed primarily in part 3, but DBR was enough of a significant development that it warranted its own post.

I say ‘was’ because apparently Valve has some feelings on the subject that supersede my own.

– Added a privacy setting in the UI so that players can specify whether they want to allow external 3rd party websites to be able to access their match history (defaulted to private).

I don’t know to what extent this will hinder Dotabuff, but in other places I had described DBR as Pandora’s box.  Possibly I got my myths mixed up and it was really Icarus flying too close to the sun.  Time will tell.

As for me, this is potentially going to make future stat aggregation even harder.  Not like it’s been easy these past few months, but I admit I’m a little worried.

In any case, the patch has some other cool changes.  All Random is in, Least Played not removes double the heroes, and Troll Warlord.  Practice vs Bots is also a good quality of life change to the UI for new players.

Edit: I don’t have intimate knowledge of how information gets passed from Valve to Dotabuff so this is speculation, but what I do know is that Dotabuff creates most of their information from match results, so the big question is what happens to a match result when some of the players are set to private and others are set to public.

Option #1: Match information cannot be obtained if any of the players are set to private.  This is the nuclear option and basically threatens Dotabuff’s entire existence.

Option #2: Match information can be obtained as long as any of the players are set to public.  This makes the privacy setting a toothless placebo.  I feel there’s basically no chance Valve would go with this, but I include the option for the sake of completeness.  Though it is possible that Dotabuff is capable of bypassing the privacy setting through their unique access method, which would effectively be this option.

Option #3:All match information can be obtained, but the player name and account ID entries are filtered out for anyone with a private account.  This allows Dotabuff to continue to exist, but to use it for your match history you would have to set your account to public.  I don’t know for certain whether public accounts would still be able to have a DBR under this system, but I would suspect that any subsequent DBR would likely be less accurate.

I’m kinda leaning towards 3 being the most likely outcome, which is actually nice from my end because it wouldn’t inconvenience me at all.  I just want to know the match MMR rating and am perfectly content leaving all of the player information completely anonymous.


Reports from the Dev forum suggest that Valve definitely went with option 3, so stat aggregation and Dotabuff will be able to continue.  Despite my comment, it’s unsure whether the anonymous account IDs are consistent over matches or not.  As was pointed out in the comments, consistent IDs would allow DBR to continue to exist.

Update #2:

Quoting from here:

It remains trivial to associate a player’s match history with their Steam profile

Not sure if this was intentional or not, but it seems that:

– The account_id of a player who has their match history set to private doesn’t change between games.
– You now have a player’s matchmaking history, but don’t know who they are.
– Replay files contain a player’s steam 64 id.
– Assuming you have one replay file in which the player has a unique name and have collected the data for that game, you’ve now linked a steam profile to a “new” account_id.
– You match history is now public again.

If accurate, it sounds like it might be trivial for DotaBuff to bypass the privacy filter entirely.  Though maybe they’ll quietly and voluntarily respect the privacy filter so they can keep their DBR while Valve can maintain a stance in favor of player privacy.

4 Responses to DBR vs Valve

  1. vonmixer says:

    Really enjoy the work you are doing on this blog, very intelligent and insightful stuff. Hoping the new changes don’t screw over this kind of stat analysis.

    • phantasmal says:

      Thanks for the compliments. Hopefully it won’t, but we’ll probably find out more once the patch rolls out officially.

  2. Pickled says:

    Was discussing this earlier, didn’t even consider #1 and #2 to be likely enough to talk about. I split #3 into two disparate options:

    3a) Hidden players get a completely anonymous name and id when set to private (eg. “Anonymous”). Dotabuff continues to exist as it was, except when viewing previous matches you see a lot of “Anonymous” names, and DBR is impossible to calculate/vastly less accurate.

    3b) Hidden players get an anonymous, but unique name and id when private (eg. “Anonymous #123456”.) Dotabuff is essentially the same as it was and DBR is still calculable. People who don’t want to be looked up wouldn’t be identifiable. Potentially identifiable by Dotabuff if you go from public -> private, but I assume they wouldn’t.

    Personally I’m hoping for 3b, I’m interested in seeing the ratings (particularly my team’s and friends) but don’t want to force anyone else to have it.

    (Awesome blog btw, I loved the standard deviation matchmaking brackets, makes perfect sense.)

    • phantasmal says:

      Reports on the Dev forum sounds like your 3b might be what Valve went with, which should keep DBR intact as you point out.

      As for the standard deviation, I liked the explanation but I’ve started to wonder if it’s too cute for its own good. Really depends on whether Very High has remained a similar size in the half a year since the last test.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: