DPS Error Bars

[Kinerd]

I started talking about this in Shade's thread, but it makes more sense as its own thread. I also made some math errors which are corrected here.

I took the case of a Barbarian 20 with an Epic SoS and all the other gear as described in the spreadsheet and split out those parts that were flat (such as +10 from weapon) and those that were distributed (such as 5d6 base damage). Then I took the standard deviation of each distribution and added them in quadrature to get a plus/minus for each attack roll. [[First note: I skipped a step by not doing a distribution for chance of glancing blow, which is to say relative frequency of attack animation, but instead treating it as a flat .75, which seems fine to me. Second: I didn't get the same final result for DPS as the spreadsheet, but I was only off by 5.94, so I'm not super concerned about that either.]] The final result was 500.9 +/- 5.2, so from damage dice fluctuations alone we would see about 1% variation in 95% of measurements.

Next, I took the data Vanshilar listed here, multiplied the occurrences of each attack roll within each set by its associated damage, converted to DPS, and got 494.4 +/- 44.8. [[This makes sense, because the average within Vanshilar's data was 10.4, slightly lower than the perfect 10.5.]] Thus, from attack dice fluctuations alone we would see about 9.1% variation in 95% of measurements.

.

I performed essentially the same procedure with the Fighter build listed in the default spreadsheet. Because we're adding error in quadrature, the damage dice variation (0.5%) provides practically no contribution to overall variation. The value I calculated over a perfect 1d20 distribution was 528.4, off from the spreadsheet by 11.4, and the values I calculated over the observed 1d20 distributions satisfied 525.6 +/- 45.6, for 8.7% variation. This figure is slightly less than the Barbarian's, which makes sense because of the way Frenzied Berserker crit profiles look next to Kensei III.

.

I plan to continue this process for the rest of the builds described in the spreadsheet, and will try to condense it into table form for the next post.

[Chai]

Funny, I was using the same type of principle, to figure out this same type of variance, only in application testing, because this is how we process phase carriers in the audio world, where we can be precise to 1/92kps. When I brought this up however, I made an incorrect reference to the law of large numbers being the law of averages, and the absolutists trolled me on the basis that if I was incorrect there I must be incorrect everywhere else, because that is simply how it works when telling people their math is off. The variance is due in part to the exact same principle, that the range is too large that we do not fight mobs from start to finish for a long enough time to be as precise as we would like.

Ahhh frequency theory, how do I love thee...

Your neg rep fails. Keep trolling it though, as it will call attention to exactly what needs to be pointed out. People dont get this mad when youre wrong. They target you when you are right.

[quityourjobs]

+1 for math.

[voodoogroves]

trolled

http://www.youtube.com/watch?v=G2y8Sx4B2Sk

[voodoogroves]

Classy. May not have caught that had I not been watching.

[Kinerd]

While working out a TWF example, it occurred to me that the treatment of glancing blows ought to have included an uncertainty due to effective process frequency. Even though glancing blows are 100% or 0% on any given swing, a DPS calculation is treating an average second, so really this is no different than treating the d20 for attack roll by making an average roll with deviations for crit process or miss process chance. With that in mind, the tables I'm going to make will have three sources of deviation, as follows:

Build	dmg	proc	atk	total
Brb THF	1.03%	4.97%	8.95%	10.29%
Ftr THF	0.52%	3.23%	8.62%	9.22%
Ftr TWF	1.28%	20.62%	6.97%	21.81%

How this works is that in each case, the other two sources of uncertainty are controlled to see how much deviation is due to just one, so for the "dmg" column I set the proc uncertainty to 0 (exactly X% of main hand attacks generate off hand attacks, the number of two handed attacks is exactly divisible by 4, magical effects occur on exactly X% of glancing blows) and the atk uncertainty to 0 (every 1d20 is perfectly distributed), and see how the fluctuations in damage dice change DPS. For the "proc" column, I instead control the damage and attack dice, and so on. I'm realizing as I write this that I didn't account for double-strike yet, so the fighter numbers posted above need to be revised.

In any event, I feel like the methodology is working well. The process uncertainty is much higher for Ftr TWF than Ftr THF because off-hand attacks represent so much more DPS than glancing blows. The damage uncertainty is measurably higher for Brb THF than Ftr THF because Barbarians get extra DPS from Vicious and Greater Vicious (dice-based) while Fighters get it from Kensei and Weapon Mastery (flat). The attack uncertainty is lower for TWF because we're adding two 1d20 distributions instead of using just one, so our relative deviation will be smaller.

[Kinerd]

It occurred to me that the uncertainties I've been talking about are not properly described. Over a long enough period, they should reduce to zero, but the ones I've cited are not functions of time. To rectify this, I'm going to talk about periods of 185 swings.

The uncertainty due to damage dice is over 20 swings, so over 185 swings this would obviously be much, much smaller (as a percentage). Because the uncertainty due to damage dice was already very small and contributed even less to total uncertainty, I didn't care to re-evaluate it for the listed cases. The interesting case is process for TWF. I had been doing the deviation between 80% mainhand+offhand and 20% mainhand only swings, which in retrospect doesn't make any sense. It makes a lot more sense to do a binomial distribution over 185 swings and see the deviations in number of offhand processes. (I could also do what Vanshilar did and count the number of processes the Turbine PRNG generates over a certain number of trials, but this way is way easier.) Unfortunately I'm using Excel, which will only do factorials up to 170!, so instead of using a factorial approximation I looked for a pattern in standard deviations of binomial distributions from 155 to 170, and found one, giving me an estimate of an 80% process over 185 swings as 148 + 10 - 11; that is, I would expect 95.4% of trials to be between 158 and 137 off-hand processes.

Hence, all I have to do is calculate the DPS values for 158, 148, and 137 off-hand processes and I'll have an uncertainty due to process dice, and I get 528.1 + 14.7 - 16.2, or 5.9% average deviation, giving a total deviation of 9.1% for Ftr TWF.

Note 1: To test the hypothesis that uncertainty due to damage dice is irrelevant in this situation, I put in 1.28% (which it would certainly be no more than and would be much, much less than) and got a total deviation of 9.2%.
Note 2: It occurred to me after that it probably would have been a lot easier just to cut Vanshilar's data into sections of 170... but that wouldn't have been any fun.