In the past few weeks during my absence, I received a couple of questions about why I haven’t done anything on Legion Commander yet, but believe me, when she dropped the first thing I did the following night was try to start putting together some samples. Unfortunately, I found that my parses were full of a bunch of garbage returns that didn’t represent real matches. It then dawned on me that these garbage matches were actually Wraith Night games, and while not quite as interesting as what I wanted, still might present an opportunity nonetheless. I’ve been going through them these past couple of weeks, and here’s what I’ve found. It’s a little too late for anyone to act on it, but it’s better to not tamper with an experiment in progress.
The simplest and most obvious thing to do was to look at the popularity of different heroes. To do this I created a simple sample of the first 500 games chronologically each day to create something that could at least pass as randomized. Wraith Night returns were also split up by the usual matchmaking groups (Normal/High/Very High).
The two biggest trends here to me are the huge increases in popularity of Windrunner and Shadow Fiend when you leave Normal. Axe and Omniknight see the biggest net declines, but both remain popular heroes in all brackets. Overall we have two tiers of heroes emerging with Axe to Legion Commander representing heroes that see a ~40% pick rate or higher, and Magnus to Crystal Maiden having a ~25% pick rate or lower.
Due to the method I used we can also look at hero pick rates by day. This allows us to see how the discovery process unfolded over the first few days of the modes existence. Let’s look at VH first.
In VH, Windrunner skyrocketed in popularity over that first week. Axe, Shadow Fiend, and Omniknight all also saw some gains, though only Axe was consistent. VH was also the most dramatic of the three brackets in terms of distinct trends so I’ll spare you the extra graphics, but I will say that Windrunner and Omniknight were #1 and #2 in usage growth in both Normal and High.
Of course, week two saw the addition of 5 new heroes in Templar Assassin, Witch Doctor, Queen of Pain, Storm Spirit, and Elder Titan. How did their addition change up usage rates? About how you’d expect
(Click for the Day-by-Day Usage in Week 2 in the Normal Bracket)
The new heroes all enter at around 30-35% and then decline over the week to ~20%, with the lone exception being Templar Assassin who maintained a near 60% usage rate, second only to Axe. With the addition of the new heroes, we now have three emerging tiers.
- Tier 1 – Axe, TA, Windrunner, and Omniknight
- Tier 2 – Drow Ranger, Shadow Fiend, Sven, Queen of Pain, Storm Spirit, Witch Doctor, Legion Commander, Elder Titan
- Tier 3 – Lina, Magnus, Juggernaut, Venomancer, Shadow Shaman, Crystal Maiden, Jakiro, Sand King
Tier 2 is definitely the least stable of the three, as the new heroes could still have been in decline at this point. Of the four, I suspect Storm Spirit and Elder Titan would be the most likely to fall into Tier 3.
But of course these tiers only represent popularity, and many times popularity doesn’t correlate with actual success. This could be especially true in a PvE game like Wraith Night, where people might be more inclined to play something fun over maximizing their team potential. If we want to look at hero strength, we’ll need some other metrics to look at. The obvious choice is just basic win rate, but unfortunately Wraith Night like Dire Tide doesn’t actually record a winner. Dire is always set as the victor (except, inexplicably, in match ID 430018855, the only game in all of my samples to have Radiant_Win set to true).
So I started looking for other predictor stats, but I kept running into difficulties. Take this early scatter plot for example:
Don’t worry too much what this is supposed to represent, just know that the Y-axis is match duration in minutes. Why are there match durations that last up to 5 hours? Unfortunately, by the time I started looking into this the match replays were already unavailable, but these weird returns were annoying. The good news is that whatever was causing these matches was gone in the week two sample. The bad news is that absurd match durations wouldn’t be my only stumbling block.
Here the X-axis represents total team GPM. That line of dots on the far right? Games where every member on the team recorded a 2500 GPM. I don’t know what’s going on in these either, but I don’t trust them messing with my measurements. What I settled on to get around this was just eliminating the top and bottom 10% returns in order to get rid of these irregular matches. I don’t know that it’s the right thing to do, but I was getting frustrated and it did leave me with something a bit more manageable.
So first things first, why did I decide to go with team GPM? My guess (and it really is just a guess; I don’t actually know anything about Wraith Night) is that teams with the highest GPMs would be the ones with the most successful waves and the highest post-wave bonuses. There’s also the fact that being able to win quickly is important if you’re trying to maximize your item returns, so it’s a pretty likely criteria for being considered a good hero.
Second, the interesting thing about this plot is that it seems to create 3 separate sloping bands, one clear one in the 30-50 match duration range, and two muddier ones at 10-20 and 20-30. Is this related to the different difficulty settings in Wraith Night? Haven’t a clue, but whatever the case the slope makes sense. The higher your team GPM the shorter the expected match duration, so in the top band the teams that finished in 30 minutes had ~3000 total GPM and the teams that finished in nearly 50 minutes had a ~2000 total GPM.
Based on this, what I decided to do is look exclusively at the 30-50 band of matches. It was a bit sloppy and some of the lower band of matches may have dribbled in, but whatever. I just want to be finished at this point. Using this set of matches, I found the average team GPM for each hero, and theoretically, the heroes with the highest average team GPM ought to be the most successful. Theoretically. I’m not 100% confident in the results, and would be more confident if I had a way to preserve a larger sample size, or just better structured match data overall. That being said, here’s what I ended up with:
The one consistent thing is that no matter how I slice it, Omniknight and Windrunner always end up on top, and I feel pretty confident that they were easily among the strongest heroes of the mode. Magnus and Witch Doctor might have been sleeper picks, but it’s hard to tell how much of that is genuine and how much of it is just really small samples.
The other thing I’ve taken from this is that if this mode sees continual balance passes for use next year, the top priority, besides adding new heroes, should be to buff underperformers. I can’t easily distinguish between heroes that are actually underpowered and heroes that are just boring, but both issues deserve to be addressed. My best guess for the heroes most deserving these adjustments would be Crystal Maiden, Elder Titan, Lina, Juggernaut, Sand King, and Venomancer, but a better testing environment would likely come up with a far more definitive list.
Anyway, that does it for Wraith Night. It’s late and not especially topical anymore, but I hope it was still interesting. The API is back to normal, so we should hopefully be able to start 2014 off soon with a look at Legion Commander sometime next week.