The Force Benchmark

[cforce]

Folks, mainly just posting this here as a reference which I often use in my builds threads or discussions of DPS. While in-game testing always trumps calculations, not all of us have time to level and equip every possible build idea. I like to get an estimate of DPS potential before wasting any time actually leveling a build up -- if it's gonna end up being inferior to some other build option, I'd rather know that up front!

I've seen lots of folks post absolute numbers from the calculations, but there are a lot of problems with that approach. It might be interesting to compare my build to Blitz, but a calculation done a year ago is out of date! But re-modeling builds every time you want to do a comparison is too error-prone.

To do any build comparison properly, you need an apples-to-apples comparison with the same set of assumptions for each build. These assumptions need to be updated as the game updates! Over time, using A_O's DPS calculator as a base (thanks again!), I've instrumented a lot of different builds for comparison, wiring the spreahsheet up to make the same adjustments to every single build column simultaneously, along a few different variables: gear assumptions, buff assumptions, fortification, DR, glancing blow targets, sneak attack ratio -- these can all be toggled in one spot, and affect every build I've got listed in the sheet.

I've posted my work here. Others are free to grab it and try to figure it out, although be forewarned that I'm not going to put much effort into trying to explain how the whole thing is wired up. Let the downloader beware! The basics are: the blue cells in the first build column change all builds simultaneously, and the graph in the middle of the spreadsheet represents many of what I consider the important builds to keep tabs on -- a mixture of forum builds and my own. That being said, there's analysis of a *lot* of the popular forum builds in there.

So, here's the set of assumptions *I* use for DPS comparison. You're free to have your own opinon on what's a better comparison -- this is just where I set the bar for my own personal purposes.

Gear: All builds get the best possible non-Epic gear and weapons. Yes, this means THF builds aren't being judged using eSoS -- they're using Lit II's or Holy/Silver/GEOB, just like the rest of the rabble. Some specifics which are assumed:
- Backstabbing+5
- Madstone Boots
- Best possible ToD sets for DPS (including unarmed damage, where appropriate)
- Non-epic Bloodstones or another Seeker+6 source*
- Lit II's for no-DR-bypass (or Holy/Greater Bane for unarmed)
- Holy/<metal>/Greater Bane for DR bypass (or Metalline/PG for unarmed)*
- Divine Power clickies as necessary for BAB augmentation to 20.
- No Yugo pots, guild ship buffs
* - The two notable exceptions I make to the general rule: I'm not assuming access to metal-threaded handwraps; I consider the availability of those to be "0" for all practical purposes. Also, I never upgraded everyone with +2 exceptional seeker after Crystal Cove. The latter is just me being too lazy to factor in a number for everyone that's only going to have a small impact.

Buffs: Best possible raid buffs assumed. Maxed warchanter song, haste, rage, prayer

Attack conditions: Over 5 minutes of constant attacking. "THF Twitch" is theoretically allowed, although isn't typically selected because of the glancing blow assumption, below. The HOrc/Greataxe specific cleave tricks aren't in there -- I'm interpretting those as a bug, and wouldn't ever plan future builds based off of them.

Mob: I assume a mystical mob that doesn't actually exist, but averages attributes which impact DPS across several possible scenarios. Specifically:

1. Metal type required: 50% of the time
Practically, this means I'm assuming a Lit II half time and a Holy Burst/<Metal>/Greater Bane half time.

2. Sneak Attack: 50% of the time
Full-time sneak attack isn't interesting, as it doesn't reflect a variety of important scenarios, but neither is disregarding sneak attack entirely. 50% sneak attack is a ballpark number, but it's the one I choose for how much value I put on SA.

3. Glancing Blows: THF, on average, have 1.5 mobs in Glancing Blow range
Again, this is really a matter of personal preference. While THF grows in power the bigger th mass of mobs is, there are lots of situations where only the single-target damage matters. "+0.5 mobs" is about as much credit as I feel comfortable giving THF without minimizing the importance of boss fights.

4. Fortification: 25%
Even going to 25% is probably over-representing the presence of fortification -- but I consider performance in the face of Heavy Fort more important than performance in the face of 0-fort, so I crank the "fortification factor" up a bit higher than where its average probably lies (I'd guess more like around 10%.) Note that this also acts as a damper on sneak attack damage -- the amount that makes it through *both* the "50%" filter and the fortification ends up only being 37.5% of full SA damage.

5. DR: 5/-
Since I assume appropriate metal types, above, this nod really reflects mobs with non-bypassable DR. If you look at an "average non-bypassable DR", this is obviously a little high -- there aren't too many mobs with non-bypassable DR. But, as with Fortification, I consider the DPS performance in the face of DR a little more important, so I upped this factor to at least 5.

************

What this all boils down to, for my purposes, is a single number, the Force Benchmark. I definite it as <<DPS of this Build>>/<<DPS of Top Build>>, for the given set of assumptions. You can think of it as a percentage -- given any set of assumptions, exactly 1 build scores "100%", and all others get a number which is the percentage of the top build DPS.

You can download the spreadsheet here:

https://docs.google.com/leaf?id=0B8T...yZGE2&hl=en_US

Note that it's a OpenOffice workbook, not a Microsoft Office workbook. Yeah, I know that means you probably need to download OpenOffice to open it. Unfortunately, a number of features don't survive the migration to Excel, so I've never migrated away from the original format I lifted from A_O years ago. (Thanks!)

The usual disclaimers go with this, as they do with any attempt to model the game:

1. In-game observations (when taken in a controlled manner) always trump modeling. Like any other calculator, this is a tool to estimate. Is it likely off by a few percentage points? Probably. Should you dismiss a 20% estimated difference as "what the math guys say"? Probably not.

2. Mistakes -- I've debugged the sheet quite a bit, but I'm certain there are some discrepancies left that need to be addressed. If you see something wrong, please, say something.

Enjoy, and as always, feedback is welcome!

[Luckness]

This is really good work. It's just the kind of thing that allows objective comparison of melee builds, a comprehensive evaluation of how the numbers come together to calculate DPS.

I understand what you are doing in terms of averaging the value of performance in various situations, from fortification to multiple enemies to DR. Further, I like the fact that it should not be too difficult to tweak the math for different purposes, whether that is to calculate using somebody else's assumptions on the relative weight of these situations, or simply to present multiple numbers for each build in several actual situations (sneak attack or none, fortification or none, external buffs or not, and so on). The latter would allow there to be multiple "max DPS" builds, each being max for their own situation, and it would really give clarity on which builds shine where.

Thanks for your efforts. I had noticed your past posts on similar topics, and I'm glad you're still applying yourself to the subject.

[EllisDee37]

You can download the spreadsheet here:

https://docs.google.com/leaf?id=0B8T...yZGE2&hl=en_US

Note that it's a OpenOffice workbook, not a Microsoft Office workbook. Yeah, I know that means you probably need to download OpenOffice to open it. Unfortunately, a number of features don't survive the migration to Excel, so I've never migrated away from the original format I lifted from A_O years ago. (Thanks!)

Bummer, as downloading and installing OpenOffice is a nonstarter for me.

Is there a way that those of us who won't download OpenOffice could get a sample of the data? I read the intro with rapt attention and then got hugely bummed that I get no data at all. heh.

[cforce]

Bummer, as downloading and installing OpenOffice is a nonstarter for me.

Is there a way that those of us who won't download OpenOffice could get a sample of the data? I read the intro with rapt attention and then got hugely bummed that I get no data at all. heh.

Bear in mind that I haven't QA'ed this excel port at all, but try this:

https://docs.google.com/leaf?id=0B8T...zMTg1&hl=en_US

None of the "instrumentation" survives the port to Excel -- it's no longer possible to toggle the different settings, but on quick inspection, it appears that the numbers for the existing set of assumptions made it through the port intact.

[EllisDee37]

Bear in mind that I haven't QA'ed this excel port at all, but try this:

https://docs.google.com/leaf?id=0B8T...zMTg1&hl=en_US

None of the "instrumentation" survives the port to Excel -- it's no longer possible to toggle the different settings, but on quick inspection, it appears that the numbers for the existing set of assumptions made it through the port intact.

Very cool, thanks much!

[TheDjinnFor]

I like the thought, cforce, but to be honest I prefer my own (work-in-progress) dps comparison tool. I understand that what you're trying to do here is give a benchmark to compare all DPS builds under one big umbrella rating score (i.e. how much it compares to the top DPS build), but I don't think your single score does it very effectively considering how much it averages things out and how subjective all of the factors you chose were.

I'm working on a C++-based one that will allow you to customize a build from scratch & compare it side-by-side with any other build, custom-make any weapon & gear layout you want (& compare gear-specific stats like healing amp and hate generation), custom-make any enemy you want the character and his gear to fight, and specify all of the unique situations that they'll be in. You'll even be able to create your own effects that aren't even in the game yet or are added in later.

I think a multitude of scores that represent a characters DPS in all sorts of situations would be more appropriate and helpful. Sure, it's a lot harder for you to compare two builds objectively, but I think DPS comparison is ultimately very subjective anyways.

[cforce]

I like the thought, cforce, but to be honest I prefer my own (work-in-progress) dps comparison tool. I understand that what you're trying to do here is give a benchmark to compare all DPS builds under one big umbrella rating score (i.e. how much it compares to the top DPS build), but I don't think your single score does it very effectively considering how much it averages things out and how subjective all of the factors you chose were.

Certainly! To be more blunt -- this method is a great way to figure out how I would rate the DPS value of a build. It's a very poor tool to try to come up with some sort of absolute, objective rating. Of course, I'd argue that the latter is impossible, as it sounds like you do, too.

I'm working on a C++-based one that will allow you to customize a build from scratch & compare it side-by-side with any other build, custom-make any weapon & gear layout you want (& compare gear-specific stats like healing amp and hate generation), custom-make any enemy you want the character and his gear to fight, and specify all of the unique situations that they'll be in. You'll even be able to create your own effects that aren't even in the game yet or are added in later.

I think a multitude of scores that represent a characters DPS in all sorts of situations would be more appropriate and helpful. Sure, it's a lot harder for you to compare two builds objectively, but I think DPS comparison is ultimately very subjective anyways.

That sounds like a pretty cool tool! I'll be the first in line to play around with it when you're done!

That being said, it sounds like the two have entirely different goals? The goal of the tool you describe is to get an absolute, objective comparison between two or more scenarios.

My primary goal is to get a bird's eye view of 20-40 builds all at the same time -- and to see where a proposed build fits in that picture. With regard to evaluating multiple different scenarios, rather than averaging, I'd say this: it's easy to compare a couple of builds across lots of scenarios, and it's easy to compare lots of builds across a couple scenarios, but attempting to compare lots of builds across lots of scenarios, you run into a data visualization problem. There are just too many numbers in the matrix to process to see anything interesting! I mean, sure, you can print those numbers out, or do a gigantic series of charts, but it's tough to glean insight from that without hours of analysis -- which I don't want to do when I just want a quick, "about how good is the DPS on this build" gut check .

[maddmatt70]

Not sure about seeker +6 anymore. What I am getting at is the lenses of Opportunity +4 seeker and +5 backstab damage is a very nice double slot item which does a nice damage per slot ratio. Since autocrits have changed this item has great potential in the future. You could have something like gem of many facets, 2 tower set, epic claw bracer (assumes gem of many facets is altered to work with both pieces), epic charged gauntlet, epic helm of frost, epic envenom, madstone boots, epic red dragon armor.