Is Finery All You Need? A Deeper Look Into Guild Progression.

It started with close DPS checks…

In the first season of The War Within, Blizzard introduced a stacking buff that adds a ~2% percent increase to damage and healing every two weeks, capping out at 20% increased throughput. This change was done to help out lower rank guilds after gear progression had finished.

The guild I play in is a rank ~1000-1500 guild. (CE this tier for sure… but seriously, we are on Queen prog now) We noticed that we were downing bosses much faster than before, as ranked on progstats.io. Our guild rank had moved up this season, but not significantly. On a few of the bosses, we wouldn’t have met the DPS check without the finery buff. At the very least they would have needed much more optimization.

Progstats shows histograms of progression time, which is good for seeing how an individual guild ranks against the crowd. I wanted to see how progression times evolved over guild ranks and kill date. But, honestly, seeing the raw data opened up a whole bunch of other questions. Unlike a typical paper, I’m going to lead with results but for those interested, the methodology and raw charts will be down at the bottom of the article.

Getting the data

For raid information, I scraped the top 2000 guilds of each tier from Raider.io. The time spent on each boss is also aggregated by Raider.io. After doing some basic outlier removal (detailed in methodology), this is what a boss looks like. Note that it is slightly different than what progstats calculates. Progstats tries to estimate the duration of each raid night session while this data is solely time spent in combat.

If you’re a current wow raider, this is surprising, right? At least it was for me. There is a massive amount of variance across all guild ranks and kill dates. Within a few ranks of each other, guilds can take double, or even triple the time to down the same boss. Also the time spent is remarkably flat across guild ranks. We’ll discuss this more in the following section, but for now, let’s focus on the the goal: evaluating finery’s impact.

Is the finery buff working?

In an attempt to make some sense of the data, I took the moving average (shown in red), and tried to fit a line to the raw data (shown in green). From the line, we can determine roughly if a boss took longer or shorter for later guilds to kill. To do this properly, a linear regression was applied to get both the slope and statistical significance.

The following chart shows each boss with an estimate of how much the prog time increased or decreased over the course of the tier. This is calculated by using the regression results (the green line, roughly) and a picking a start and end date. The start date is chosen from the 5% percentile and the end date from the 95 percentile. This is done because bosses get killed in different time periods. Typically the first few bosses get cleared quickly with the vast majority of guilds clearing in 1-2 months. Later bosses are more spread out with the bulk of guilds usually taking around 3-4 months.

The percentage shown is relative to the median kill time. A positive slope, (shown in red, example) indicates that a boss took longer for guilds that killed it later, and vice versa (shown in green, example). A yellow indicator is shown when the statistical significance was too low (p>0.05).

For a concrete example: a theoretical boss takes 9 hours to kill in the first week. It goes up to 11 hours to kill for guilds that kill at the end of the tier. The median kill time is 10 hours. The delta for that boss would be +2 (9 -> 11 hours). The percentage would be 20% (2 hours / 10 hour median). This boss enrty would look like this:

• Example Boss 1.00 (20.00%)

Here’s the data in all its glory:

I don’t think the results are conclusive. There aren’t any immediately discernable patterns, neither in direction or magnitude. I can see two possible reasons for this, but there could be more:

1. Gear and Finery have little impact on progression time. The main requirement to down a boss is to simply put in enough hours to learn the mechanics. The difference in rank is mostly determined by number of hours a guild raids per week.

2. Gear and Finery have impact, but are offset by the steadily lowering skill of lower rank guilds. The time spent has actually been carefully balanced so that a high and low rank guild spend the same time progging at different power levels.

Looking at guilds

To test this, I dug into some individual guilds along Kyveza progression to see if some patterns emerged. For the guilds who killed early, the ones that killed with more prog time, as would be expected, had longer schedules. Many 3-4 day guilds. The ones with fast prog times had lighter, but still relatively heavy schedules. I saw one outlier guild, with extremely low prog times and a 6h weekly schedule. Props to them!

On the other end, guilds that killed in late Dec I saw little difference in schedules from the ones that killed with low and high prog time. Most of the ones I surveyed were 6h guilds. The difference in pull time must have arisen from other factors, but I couldn’t dig enough to get a definitive answer or pattern. These could include taking weeks off for vacation, failed reclears, or reclearing more before locking. While this could be investigated, this project has taken enough time so I’ll leave the thread here for now.

Looking at raids as a whole

Switching gears to comparing raids, Nerub is scaling less than Aberrus and Amidrassil, but VOTI had a negative slope. This is, ironically, inverse to expectations as most people consider Aberrus the easiest raid of the tier. Maybe it’s related to nerfs? Pre-nerf Ansurek and TSwift were nightmares to prog and were maybe brought down when Blizzard made mechanics easier to solve? Maybe more teams jumped into Aberrus, but bounced off the others? It’s hard to tell.

In theory, these numbers have the potential to add up. Lets take a guild on late Aberrus prog. On the tail end, they are looking at 15-16 extra hours of raid compared to a better guild. That probably translates to 2-3 extra weeks, maybe 3-4 weeks due to breaks and downtime between pulls. But without a conclusive pattern it’s hard to put a pin in things. S2 of TWW is slated to have a similar stacking buff. With more data from Nerub-ar and the following raids, we might collect some conclusive evidence of the effect of the stacking buff.

From a design perspective, I believe that a negative slope on bosses is a good on-ramp for newer players, either through gearing or the stacking buff. It’s a natural tradeoff. People who wish to push higher ranks should spend more time progressing. On the other side, new players can get to experience the raid more efficiently and with lighter raid schedules. All in all, I think current balancing is ok. While it could be better, we need more time with the finery buff to truly analyze its impact.

Back to those graphs, some extra tidbits

Here are links to full images of Vault, Arberrus, Amidrassil and Nerub-ar. Note that both the time and date axis ranges vary depending on the data.

As mentioned before, the variance between guilds who kill on similar dates is astounding. At a glance, most people look at to guild rank as an indicator of a guild’s performance, but from these graphs it’s clear that within a rank range, efficiency is another factor to look for when choosing a guild.

What can we conclude on this front? For a guild that wants to push guild rank, the best way it seems to do so is to increase the number of hours per week, assuming people’s mentals hold up. The sooner you can reach that average threshold for each boss the better the chances are at acquiring higher guild rank. For someone applying, look for guild rank, but also look for hours per week as there appears to be a wide range of guilds at every guild rank.

As a side note: Difficult bosses at the beginning of the tier highlight the difference between RWF, top 10, and the rest of the community. The rank 5-10 guilds spend significantly more time on bosses than RWF. The kills are much more sparse for the first few months until the rest of the high-end community reaches the bosses.

Methodology

If you want to play with the data yourself, a .zip is here with all the ranks and times in .json format. The ranks folder contains the guild raid rank and metadata, as that is what raider.io provides. The times folder contains the time to kill for each guild. The names all follow raiderio’s slug format.

Data Sources

The raid ranks are extracted from Raider.io’s public API. Unfortunately the rank for each boss isn’t available from their public api so we have to use the raid rank as a surrogate.

The boss time data isn’t documented in their public API but can be found by using their site and monitoring network traffic. I’m not going to include it here, since it’s not in their docs. If you have the capability to find it yourself, please use it responsibility.

Outlier Removal

For the slope analysis, outliers were drawing the regression away from the main trends of the majority of the guilds. I had to remove outliers differently for easy, early bosses and the later, more difficult bosses. The main graphs above are all shown with outlier removal.

For the early bosses, without outlier removal, rekills from newly formed guilds, (or guilds entering late), would skew the data away from the trend. I ended up going into each boss, and manually filtering out the guild kill times. Usually removing the last 200-400 of the top 2000 guilds would remove the long tail of late kills.

For the late bosses, in an attempt to make data cleaner, kills that took under a hour to down were removed. This data was likely generated by experienced guilds or players reforming in the middle of the season. In the end though, this didn’t affect the slope of the bosses too much.

The top end outliers could be removed too, but wasn’t due to time. I suspect this is from logs not being pieced together correctly. Maybe a guild with multiple CE teams who were progging at the same time, or a separate reclear team. By and large, these weren’t affecting the slopes so were left in.