The required number of runs depends heavily on the underlying (unknown) baseline drop rates. (BTW software development for sample size analysis happens to be one of my day jobs, lol).
I'd be happy to analyze any data that anyone submits to me. You'd just need to define your criterion in advance (e.g., drop rates of a specific item or class of items under two different conditions, such as n1 teammates vs. n2 teammates) and try to make sure that no other factors vary. Alternate or randomize the two situations to account for possible time effects. Please PM me with any data.
For example, let's say you're measuring the drop rate difference of festival items when soloing vs. with 7 leechers. Then depending on the approximate drop rate when soloing (Ref Proportion) and the underlying difference (Proportion Diff)), you'll need approximately this many total runs (if split evenly between both situations) to have a 95% chance of establishing a statistically significant result at the .05 level:
Code:
Ref ----------------- Proportion Diff ------------------
Proportion 0.005 0.010 0.050 0.100 0.250 0.500
---------- ------- ------- ------- ------- ------- -------
0.01 21370 6392 580 240 76 28
0.05 86130 22494 1196 386 96 32
0.10 159288 40676 1890 548 116 34
0.15 223788 56692 2496 688 134 36
0.20 279630 70546 3014 808 148 38
0.25 326814 82234 3448 904 158 38
If any of you happen to have SAS installed where you work or go to school, here's a program that you can modify and run for different situations:
proc power;
ods output output=p;
twosamplefreq test=pchi sides=1
proportiondiff = .005 .01 .05 .1 .25 .5
refproportion = .01 .05 .1 .15 .2 .25
alpha = .05
power = .95
ntotal = .;
run;
%powtable(data=p, entries=n, rows=refproportion, cols=proportiondiff);