I told a guy that a wide variance in data essentially means that results were random then he proceeded to explain p values and I’m like “yeah I’m sure the random values values came from nature”.
You can the issue with A-B comparison is that is kinda expected for one group to be higher on average simple based on the data points you selected.
If the averages are kinda similar and variance are high in both group A and B then I’d say both groups are statistically the same even if the statistical values are different
So you have two groups of ten experiments, mean if group A is 100, mean of group B is 105, variance is 25 (for both groups). Obviously we are not confident that these groups differ.
Now suppose we repeat the experiment two billion times. The group A average is now 99, and the group B average is now 103. The variance is still 25. Are you still not confident that the groups are different?
I told a guy that a wide variance in data essentially means that results were random then he proceeded to explain p values and I’m like “yeah I’m sure the random values values came from nature”.
Moral is p values kinda worth less than variance
Are you saying you can’t determine a difference in aggregate statistics by performing more trials if the variance is high?
You can the issue with A-B comparison is that is kinda expected for one group to be higher on average simple based on the data points you selected.
If the averages are kinda similar and variance are high in both group A and B then I’d say both groups are statistically the same even if the statistical values are different
So you have two groups of ten experiments, mean if group A is 100, mean of group B is 105, variance is 25 (for both groups). Obviously we are not confident that these groups differ.
Now suppose we repeat the experiment two billion times. The group A average is now 99, and the group B average is now 103. The variance is still 25. Are you still not confident that the groups are different?