Implementing Binomial Hypothesis Testing significance tests in Power BI (DAX) - powerbi

Question

This is partly a theory question, and partly an implementation question. My stats is a little rusty...
I am developing a report that is attempting to determine if the difference in occurances between a reference group and a selected group are statistically significant.
So, for example, if something occurs in X of n tests for one group, is it statistically significant than if it 'normally' occurs at a rate of Y of m tests for a different (control) group.
So, my H0 is that the rate is Y of m, per the control group
h1 is that it is not the same as the control group. (ideally, I'd like to use a 1-tailed test, depending if the observed occurrence is greater or less than the control, but my current implementation is 2 tailed)
I'd be comfortable with a CI of 80%.
I've got (slightly pseudocode here):
Zscore =
VAR pControl = DIVIDE(COUNT([Control occurrences]), COUNT([Control Tests])) RETURN
VAR pTest = DIVIDE(COUNT([Test occurrences]), COUNT([Test Tests])) RETURN
VAR controlStandardError =
SQRT(
DIVIDE(
(pControl * (1-pControl)
, COUNT([Control Tests])
)
) RETURN
VAR testStandardError =
SQRT(
DIVIDE(
(pTest* (1-pTest)
, COUNT([Test Tests])
)
) RETURN
DIVIDE(
(pTest - pControl)
, SQRT(POWER(testStandardError, 2) + POWER(controlStandardError, 2)
)
I'm then calculating:
p-Value =
VAR pControl = DIVIDE(COUNT([Control occurrences]), COUNT([Control Tests])) RETURN
IF(pControl > 0,
1 - ABS(NORM.DIST(Zscore, 0, 1, TRUE)
)
I am then displaying in a table each of my non-null hypotheses and filtering the table such that p-Value is less than 0.1. (2-tailed 80%)
am I on the right track here? Or have I completely bungled the theory on this one?

Answer 1

Theory and example tables - Right-tailed (μ > μ₀)
DAX
ControlGroup
XControl = COUNTROWS(FILTER(ControlGroup,ControlGroup[Outcome]=1))
NControl = COUNTROWS(ControlGroup)
pControl = DIVIDE([XControl],[NControl])
TreatmentGroup
XTreatment = COUNTROWS(FILTER(TreatmentGroup,TreatmentGroup[Outcome]=1))
NTreatment = COUNTROWS(TreatmentGroup)
pTreatment = DIVIDE([XTreatment],[NTreatment])
Test Parameters
PooledProportion =
DIVIDE(
[XTreatment]+[XControl],
[NTreatment]+[NControl]
)
ZCritivalValue = NORM.S.INV(0.90)
ZValue = DIVIDE(
[pTreatment]-[pControl],
SQRT(
[PooledProportion]*(1-[PooledProportion])*((1/[NTreatment])+(1/[NControl]))
)
)
Visualization (example)