Attention Data Miners - Critical Hit Research

Good day,

I am looking for help beating up low level monsters to mine the data necessary to firm up the critical hit equation. Unlike the other equations, this is a statistical process so I need a lot of trials to get half decent confidence intervals.

The sweet spot for getting results quickly involves using a level 20 character to beat up level 8-10 mobs using 9-12 weapon skill. This is the region of the equation where the critical hit rate is the most sensitive, so we can get usable results relatively quickly (unlike level 20 on 20 attacks, which require 25,000 or so attacks to get a usable confidence interval).

If you would like to help, just go hit some mobs and report the results here. I need to know the level of the mob you hit, the number of crits and the number of non-crits, and the attribute level of your weapon skill. Please use a level 20 character.

Also, if you know of a better spot to do this than outside of Yak's Bend, please let me know.

Would you want the weapon to be a specific damage and requirement, ie req9 15-22 sword, as opposed to req13 14-21 sword?

Weapon doesn't matter. I suggest low damage so you can get more hits on a guy before he dies. Weapon req and damage do not affect crit rate at all. What's most important is to make sure you're hitting guys who don't have armor stances, RoFs flying around, or things that give you false negatives.

I'm using the level 10 minotaurs outside of Ice Tooth for my data. Here's what I have so far:

Req 10 Vampiric Hammer, no +15% mods.

12 Hammer Mastery:
289 Crits, 106 Non-crits. 73.1646% crit.

11 Hammer Mastery:
264 Crits, 131 Non-crits. 66.8354% crit.

10 Hammer Mastery:
255 Crits, 140 Non-crits. 64.5570% crit.

Victo's Maul, 15^50.
9 Hammer Mastery:
199 Crits, 196 Non-crits. 50.3797% crit.

Kind of surprised there's such a sharp drop between 10 and 9, to be honest.

@Yunsoo: That caught me on my first run. I was wondering why I was C-spacing for 60 when Wild Blow did 46.

Would something like a candy cane sword be easier to use because the damage is exactly the same unless a crit?

Couldn't you just test on the Isle of the Nameless or am I missing somethign? I would suggest making videos too of everything if you have the space.

I believe Ensign wanted to avoid testing on level 20 monsters so that he can reduce the amount of data he needs in order to get a result that falls within the 95% confidence interval...

The reasoning goes like this.

Beating on level 20 barrels in Isle of the Nameless gives you data, but you need a lot of it because things change so little. For example, is the crit rate at 12 weapon mastery and same level 17% or 18%, or something in between? You would need tens of thousands of data points to make that distinction; and even if you did, what does it mean to anything else?

If you're hitting level 8-10 targets with 9-12 weapon mastery, the disparities are large enough that you'll be getting crit rates in the 60-90% range for most of those, and each level difference in weapon or mob level will give very big differences in crit rate. For example, the difference between hitting a level 9 monster vs a level 10 monster with 12 weapon mastery is over 10%. Contrast that with hitting a barrel, where the difference between 11 weapon mastery and 12 weapon mastery is 1-2%. Because the differences between each level / weapon mastery point are so large, we can start to draw conclusions from relatively few samples, which is what makes this experiment realistic.


Yeah, pretty much; when you're crit rates are low and don't change much with attribute investment, you need really tight 95% confidence intervals (<1% crit rate), but when your crit rates are really high (as close to 100% as I can get them and scaling fast) then you can draw useful conclusions even with 3-4% on your confidence intervals.

I lack the time to whack a bunch of monsters, but I will contribute something to help with the testing: I'm reposting my critical hit mod. Using this mod, critical hits should look like the bottom picture here -- hard to miss.

Ooh, nice animation. I may just try that if I ever want to see them. I believe there was another animation that made it more obvious too.

=IF(((0.05*2^((((8*LevelA)+(4*WeaponSkill)+6*MIN(W eaponSkill,(LevelA+4)/2)-(15*LevelD)-100))/40))*(1-(WeaponSkill*0.01)))+WeaponSkill*0.01-WeaponCritChance>0,((0.05*2^((((8*LevelA)+(4*Weapo nSkill)+6*MIN(WeaponSkill,(LevelA+4)/2)-(15*LevelD)-100))/40))*(1-(WeaponSkill*0.01)))+WeaponSkill*0.01-WeaponCritChance,0)

?

Churrir Fields (next to Kamadan) has Lvl 5 practice targets that could help.

Good luck with the research, but I suggest you change the topic title so more people realize what it is for. When I read the topic title, I thought of the 3 stooges.

http://guildwars.wikia.com/wiki/Data_Mining

To prevent having people expecting info/help on that quest, a new topic title may be useful.

This has been posted before. Plug some numbers in and you can see that it's obviously incorrect.

There isn't a sharp drop, the error bars are just large. Thanks for the help M'Aiq

No problem. I was kind of wondering what the deal was with 10 Hammer Mastery, if my warrior was just "feeling it" or what. Think I should give it a second run? I'm not sure if 395 hits per skill level is enough.

There's a 95% chance that the critical hit rate with 10 weapon mastery against level 10 targets is between 59.74% and 69.37% based on that data.

I'm combining it with some of my own data to get the bounds down. Even wide error bars are useful though, as I'm trying to fit some tight data points (the 12 WS / 10 target) and the looser ones still provide a sanity check.

Have you got any idea what the difference in critical hit chance between 14 and 16 mastery is with level 20 vs. 20 ?\

Also are you going to adapt your data to lvl 20s?

I am liking this idea, its actually gettin us all together well saying that im not able to get on gw atm. I suppose you could probably gave asking them on wiki, they might be kind enough?
Nice plan on the mod too forgot about using that one

The difference in critical hit chance between 14 and 16 mastery, 20 vs. 20, is something like 2.5% - 3%. I haven't seen any experiments that show the difference with reasonable error bars.

I'm doing experiments on low level monsters, but ultimately any model based on that needs to predict 20 vs 20 results as well. 20 vs 20 numbers are more of a sanity check; your model needs to fit them, but realistically you can't get enough data to differentiate different hit % models from 20 vs 20 alone.

Actually, my alliance prompted me to take another look at this question. I think maybe I've solved it.

Assume:
1. Izzy made one typo. "0.05" should be "0.5"
2. WeaponCritChance is 0 for martial weapons and some value between -1 and 1 for caster weapons. (I don't really care about crit chance on caster weapons, so I'm not going to worry about figuring this out.)

Suddenly we get a very sane looking progression for lvl20-vs-lvl20:
Compare that to the observed and linear fit tables on the unoffical wiki. Particularly to the 12 mastery value.

It also predicts 0.635143202 for 10 mastery on a lvl 10 target. (See 4 posts up.)

Let me try to put the formula into a more comprehensible form:

BaseCritChance = A + B - C, where
A = 1%*Mastery.
B = (100% - (1%*Mastery)) * 0.5 * 2^((8*AttackerLvl + 4*Mastery + 6*D - 15*DefenderLvl - 100)/40), where D is the lesser of {Mastery OR (AttackerLvl + 4)/2}.
C = WeaponCritChance, which is zero for martial weapons.

Note that, for lvl20 vs lvl20, A becomes the largest value of generalized equation. This jives with Izzy's comment that "If I recall the base to the formula is basically 1% * Weapon skill for same level characters."

One-line version (martial weapons):
BaseCritChance = (0.01*Mastery) + ((1 - (0.01*Mastery)) * 0.5 * 2^(((8*AttackerLvl) + (4*Mastery) + (6 * Min{Mastery, ((AttackerLvl + 4)/2)}) - (15* DefenderLvl) - 100) / 40))

Now, there's a couple problems left:

First, for really low monster levels, you can go over 100% crit chance according to the formula. Does that match observations? If it's possible to fail to crit against really low level monsters, then the possibilities are that (a) I'm totally wrong with the formula here, or (b) there's a cap on crit chance at some high value like 90%.

Second, how do the critical strikes attribute and assassin skills that improve crit chance interact? (Commentators differ as to whether they add, multiply, or some add and some multiply.)

I don't know what stats you are using, but that just can't be right, 25,000 is a monstrously high value to achieve this.

That gives a pretty good fit of the level 20 data, so it's a good guess at least; it wouldn't surprise me if that's how it actually works at low level disparities.


Any fit you make of the data at low and even moderate level disparities is going to give you a crit chance over 100% at really high disparities. But you never hit 100% crit rate in the actual game. At some point the function starts to slow down and looks like it approaches 100% asymptotically.

I've attached raw data below; I didn't quite get enough data on the level 3's that their 95% CI's don't overlap anymore, but it's very likely from the data so far that the function just starts to slow down at high crit percentages to keep you from hitting 100%.


Clvl = 20; Tlvl = 9; Wskill = 12
142 Crit, 25 Nor; 85.03%

Clvl = 20; Tlvl = 10; Wskill = 12
549 Crit, 187 Nor; 74.59%

Clvl = 20; Tlvl = 10; Wskill = 11
264 Crit, 131 Nor; 66.84%

Clvl = 20; Tlvl = 10; Wskill = 10
255 Crit, 140 Nor; 64.56%

Clvl = 20; Tlvl = 10; Wskill = 9
199 Crit, 196 Nor; 50.38%

Clvl = 20; Tlvl = 3; Wskill = 12
285 Crit, 12 Nor; 95.96%

Clvl = 20; Tlvl = 3; Wskill = 9
223 Crit, 22 Nor; 91.02%


Compared to getting the raw critical hit formula, those interactions should be very easy to test, since all the numbers involved are relatively large and you'd be trying to distinguish very noticeable differences (several percentage points).


You're right, it is, but it's also the number you need. We know that the critical hit % chance at 20v20, 12 weapon mastery is somewhere in the 17%-18% range; in order for data for that data point to be useful, it needs to distinguish between something that happens 17% of the time and 18% of the time, conclusively. Getting that kind of resolution on a randomly distributed event does require a monstrous amount of data, unfortunately.

Yes, the observed not-100%-crit results at big disparities is the only thing that makes me doubt the Izzy-made-1-typo-in-an-otherwise-correct-formula theory. Either the formula is entirely wrong or there's a second effect being applied in big-disparity cases.

[edit: I just made a quick comparison of predicted versus actual values for your data. The only one the formula gets correct is the lvl20-vs-lvl10 @ 10 mastery case. Assuming that formula is correct in general, I'm going to speculate that diminishing returns kicks in above 66.6...%]

I gave the practice dummy a few hundred whacks last night. Sample size was too small for conclusive results in some cases, but everything tended towards the conclusion that each source of critical hits gets an independent roll. So probably
Crit% = 1 - ((1- Base%) * (1 - CritStrikes%) * (1 - CritEye%) * (1 - WotA_or_WotM%))

i want 2 help but wat kind of animals do we need 2 fite

I find Chthon's results interesting (they must have been a hassle to come up with; even I got confused looking at it), because they seem to confirm that you have a lower crit chance against higher-level foes (something many have stated, but until now I have seen no evidence for).

However, it's worth noting that when I plugged in some numbers for 14 weapon mastery and lvl 20's, the result was 0.201742537, not the .192 from the wiki. This may be within the margin of error of the experimentation, however. I don't know.

There's a lot of weird stuff going on in the underlying computations in game; for example, 25% armor penetration takes 16 armor off of a 60 armor target. If something is acting really weird, the code for it is also probably really weird. This is really inconvenient as you can usually anthropomorphize equations in games, and in some cases that just doesn't work in GW.

I know there isn't a hard cap on the crit%; all the data implies it approaches 100% asymptotically. But there isn't anything else in GW that does that as far as I know. So it's exponential at low crit% but diminishes asymptotically at high %...It's complicated enough that it's pretty clear I'm not going to be able to get a very satisfying answer with 2% error bounds, which is what I was hoping for.

The result from critical strikes and friends, at least, makes sense and is easy to put into an equation.


I remember Kuntz doing some massive experimentation with a bot years ago, but we can't find the data from that unfortunately; I don't know of any experiments out there that have gotten the 95% error bars below 1%, so both of those numbers are perfectly reasonable given what's known.