A Look Back at Wraith Night

In the past few weeks during my absence, I received a couple of questions about why I haven’t done anything on Legion Commander yet, but believe me, when she dropped the first thing I did the following night was try to start putting together some samples.  Unfortunately, I found that my parses were full of a bunch of garbage returns that didn’t represent real matches.  It then dawned on me that these garbage matches were actually Wraith Night games, and while not quite as interesting as what I wanted, still might present an opportunity nonetheless.  I’ve been going through them these past couple of weeks, and here’s what I’ve found.  It’s a little too late for anyone to act on it, but it’s better to not tamper with an experiment in progress.

The simplest and most obvious thing to do was to look at the popularity of different heroes.  To do this I created a simple sample of the first 500 games chronologically each day to create something that could at least pass as randomized.  Wraith Night returns were also split up by the usual matchmaking groups (Normal/High/Very High).

WraithWeek1Overall

The two biggest trends here to me are the huge increases in popularity of Windrunner and Shadow Fiend when you leave Normal.  Axe and Omniknight see the biggest net declines, but both remain popular heroes in all brackets.  Overall we have two tiers of heroes emerging with Axe to Legion Commander representing heroes that see a ~40% pick rate or higher, and Magnus to Crystal Maiden having a ~25% pick rate or lower.

Due to the method I used we can also look at hero pick rates by day.  This allows us to see how the discovery process unfolded over the first few days of the modes existence.  Let’s look at VH first.

WraithWeek1ByDayVH

In VH, Windrunner skyrocketed in popularity over that first week.  Axe, Shadow Fiend, and Omniknight all also saw some gains, though only Axe was consistent.  VH was also the most dramatic of the three brackets in terms of distinct trends so I’ll spare you the extra graphics, but I will say that Windrunner and Omniknight were #1 and #2 in usage growth in both Normal and High.

Of course, week two saw the addition of 5 new heroes in Templar Assassin, Witch Doctor, Queen of Pain, Storm Spirit, and Elder Titan.  How did their addition change up usage rates?  About how you’d expect

WraithWeek2ByDayVH

WraithWeek2ByDayN

(Click for the Day-by-Day Usage in Week 2 in the Normal Bracket)

The new heroes all enter at around 30-35% and then decline over the week to ~20%, with the lone exception being Templar Assassin who maintained a near 60% usage rate, second only to Axe.  With the addition of the new heroes, we now have three emerging tiers.

  • Tier 1 – Axe, TA, Windrunner, and Omniknight
  • Tier 2 – Drow Ranger, Shadow Fiend, Sven, Queen of Pain, Storm Spirit, Witch Doctor, Legion Commander, Elder Titan
  • Tier 3 – Lina, Magnus, Juggernaut, Venomancer, Shadow Shaman, Crystal Maiden, Jakiro, Sand King

Tier 2 is definitely the least stable of the three, as the new heroes could still have been in decline at this point.  Of the four, I suspect Storm Spirit and Elder Titan would be the most likely to fall into Tier 3.

But of course these tiers only represent popularity, and many times popularity doesn’t correlate with actual success.  This could be especially true in a PvE game like Wraith Night, where people might be more inclined to play something fun over maximizing their team potential.  If we want to look at hero strength, we’ll need some other metrics to look at.  The obvious choice is just basic win rate, but unfortunately Wraith Night like Dire Tide doesn’t actually record a winner.  Dire is always set as the victor (except, inexplicably, in match ID 430018855, the only game in all of my samples to have Radiant_Win set to true).

So I started looking for other predictor stats, but I kept running into difficulties.  Take this early scatter plot for example:

WraithBadScatter1

Don’t worry too much what this is supposed to represent, just know that the Y-axis is match duration in minutes.  Why are there match durations that last up to 5 hours?  Unfortunately, by the time I started looking into this the match replays were already unavailable, but these weird returns were annoying.  The good news is that whatever was causing these matches was gone in the week two sample.  The bad news is that absurd match durations wouldn’t be my only stumbling block.

WraithBadScatter2

Here the X-axis represents total team GPM.  That line of dots on the far right?  Games where every member on the team recorded a 2500 GPM.  I don’t know what’s going on in these either, but I don’t trust them messing with my measurements.  What I settled on to get around this was just eliminating the top and bottom 10% returns in order to get rid of these irregular matches.  I don’t know that it’s the right thing to do, but I was getting frustrated and it did leave me with something a bit more manageable.

WraithGPMScatter

So first things first, why did I decide to go with team GPM?  My guess (and it really is just a guess; I don’t actually know anything about Wraith Night) is that teams with the highest GPMs would be the ones with the most successful waves and the highest post-wave bonuses.  There’s also the fact that being able to win quickly is important if you’re trying to maximize your item returns, so it’s a pretty likely criteria for being considered a good hero.

Second, the interesting thing about this plot is that it seems to create 3 separate sloping bands, one clear one in the 30-50 match duration range, and two muddier ones at 10-20 and 20-30.  Is this related to the different difficulty settings in Wraith Night?  Haven’t a clue, but whatever the case the slope makes sense.  The higher your team GPM the shorter the expected match duration, so in the top band the teams that finished in 30 minutes had ~3000 total GPM and the teams that finished in nearly 50 minutes had a ~2000 total GPM.

Based on this, what I decided to do is look exclusively at the 30-50 band of matches.  It was a bit sloppy and some of the lower band of matches may have dribbled in, but whatever.  I just want to be finished at this point.  Using this set of matches, I found the average team GPM for each hero, and theoretically, the heroes with the highest average team GPM ought to be the most successful.  Theoretically.  I’m not 100% confident in the results, and would be more confident if I had a way to preserve a larger sample size, or just better structured match data overall.  That being said, here’s what I ended up with:

WraithAvgGPMVH

WraithAvgGPMN

The one consistent thing is that no matter how I slice it, Omniknight and Windrunner always end up on top, and I feel pretty confident that they were easily among the strongest heroes of the mode.  Magnus and Witch Doctor might have been sleeper picks, but it’s hard to tell how much of that is genuine and how much of it is just really small samples.

The other thing I’ve taken from this is that if this mode sees continual balance passes for use next year, the top priority, besides adding new heroes, should be to buff underperformers.  I can’t easily distinguish between heroes that are actually underpowered and heroes that are just boring, but both issues deserve to be addressed.  My best guess for the heroes most deserving these adjustments would be Crystal Maiden, Elder Titan, Lina, Juggernaut, Sand King, and Venomancer, but a better testing environment would likely come up with a far more definitive list.

Anyway, that does it for Wraith Night.  It’s late and not especially topical anymore, but I hope it was still interesting.  The API is back to normal, so we should hopefully be able to start 2014 off soon with a look at Legion Commander sometime next week.

Advertisements

8 Responses to A Look Back at Wraith Night

  1. rbtessier says:

    The absurd match durations in the early scatter plot may have been due to an expoit.

    Namely there is one where if every user but one left the game during the phase where one chooses to move on to the next difficulty(after selecting to move on), after rejoining, those users who left would save the levels and items from the previous match allowing you to win consistently on difficulty levels up to newgame+++++ and beyond. To get to this level to took about 5-6 hours, so perhaps this is the problem.

    Eventually valve discovered this bug and patched it out.

  2. SoapDealer says:

    No posts for a month and then you do a post on Wraith Night? Bummer

    • phantasmal says:

      For a month Wraith Night was over half of the returns I was getting, which made my typical operations rather inefficient. API appears to be back to normal now.

  3. BLUEPOWERVAN says:

    IMO, the banding you see in Game length/GPM is due to the varying difficulty of rounds. Rather than being progressively harder or easier, some rounds are much harder than others. For instance round 3 or 4 has slardars and slarks which are quite hard as you need to dodge skills to live — This is the first major band of 10 minute games.

    There are a few other difficult rounds, but they are mostly spaced out, which is why you get banding.

    • phantasmal says:

      Particularly hard rounds creating a failure wave is an interesting possibility. It’s not really testable without doing more replay watching than I care to, but it’s a plausible explanation.

  4. zoolooman says:

    Some of the heroes you named in that list (Sand King, Lina) are actually some of the strongest heroes in the mode, but you have to play them in a way that is completely opposite your initial expectations.

    Sand King’s Caustic Finale was a complete trap and should’ve only received one point max. By maxing Sand Storm and rushing a Radiance, you trivialized even the hardest waves by creating a death zone that enemies could be kited through.

    Similarly, Lina was one of the best combo AoE + DPSers because her passive was massively overbuffed, but you had to build a Bloodstone to take advantage of it, which was deeply unpopular because it didn’t look like a DPS item. That said, bloodstone + crit + raw damage essentially made her into the best killer after Drow and WR, who could just stack rapiers.

    • phantasmal says:

      I did mean to address the possibility that just like in regular Dota there might be heroes that are unpopular and unsuccessful, but only because most of the playerbase is unaware of some kind of build or playstyle that can make them successful.

      But that brings up the question of what should a mode like Wraith Night be balanced towards? Unlike regular Dota, I’m not convinced that it should be balanced primarily with top players in mind. It’s a “for fun” mode, so the primary goal should be that as many heroes as possible be fun to play for the average, no-nothing player. If someone like Sand King gets buffed because of this and becomes a must have in ++ groups, well, then I guess Axe/Windrunner/Omniknight have some company. That’s not to say that balance shouldn’t seek to prevent a hero that completely trivializes upper difficulties, just that it doesn’t look like it’s the most urgent balance concern.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: