API

vstats

A small Python package providing tools for statistics. An accompanying package for visualization is vplotly.

vstats.get_est_welchs_test_properties(n: ~typing.List[int], share_n_group_1: float, alpha_test: float, rv_1: ~typing.Union[~scipy.stats._distn_infrastructure.rv_discrete, ~scipy.stats._distn_infrastructure.rv_continuous], rv_2: ~typing.Union[~scipy.stats._distn_infrastructure.rv_discrete, ~scipy.stats._distn_infrastructure.rv_continuous], conf_level_rejection_prob: float, n_simulation: int, fix_degenerated_samples: bool = False, rng: ~typing.Union[~random.Random, ~numpy.random._generator.Generator] = Generator(PCG64) at 0x7F0477C0F660) → DataFrame

Estimates properties of a two-sided Welch’s test for two independent samples for a list of given total sample sizes and for given random variables from scipy.stats generating the samples. The estimation also includes confidence intervals of the true value of the properties. Currently only rejection probabilities of the test are estimated. Exact confidence intervals for the true rejection probabilities are calculated as in Theorem 8.11 in Georgii (2008, p. 231). For details about Welch’s test and its assumptions see Loveland (2011, p. 48, 101) and section 10.2 in Devore (2018).

Parameters:

n (list of int) – List of total sample sizes to estimate the properties of the test for. The total sample size is the sum of the sample size of group 1 and the sample size of group 2.
share_n_group_1 (float) – Desired share of the sample size of group 1 in relation to the total sample size n of both groups. Note that the actual sample size n_1 of group 1 is rounded up to the nearest integer, so that the desired share is not always attained exactly. The sample size of group 2 is n_2 = n - n_1.
alpha_test (float) – Nominal significance level of the test applied.
rv_1 (random variable from scipy.stats) – Random variable used to generate the samples for group 1. Use scipy.stats.norm to fulfill the assumptions of Welch’s test. Use other random variables from scipy.stats to test the effect of violations of the assumptions.
rv_2 (random variable from scipy.stats) – Random variable used to generate the samples for group 2. Same usage as for rv_1 above.
conf_level_rejection_prob (float) – Confidence level of the confidence intervals generated for the true rejection probabilities.
n_simulation (int) – Number of tests to perform for estimating properties of the test for a given total sample size.
fix_degenerated_samples (bool, optional) – If True, a more diverse sample is drawn for a group if its sample contains only a single value. The frequency with which this happens is reflected in the columns share_fixed_samples_group_* in the output, which are only included in the output if fix_degenerated_samples is True. If a fix occurs too frequently, the corresponding results are not reliable. This parameter is useful when discrete distributions are used to generate the samples in order to evaluate the performance of the test under violated assumptions, - in this case set fix_degenerated_samples to True. By default False.
rng (Instance or derivate of random.Random or numpy.random.Generator, optional) – Pseudo-random number generator that is used as random_state of the random variable rv. We recommend using numpy.random.default_rng with a seed generated via numpy.random.SeedSequence().entropy. Note that if this is done, the results are only guaranteed to be reproducible with the same version of Numpy with which they were generated. By default numpy.random.default_rng().

Returns:

Result of the estimation of properties of Welch’s test with the columns:

ninteger: Total sample size for which properties of the test are estimated. The total sample size is the sum of the sample size of group 1 and the sample size of group 2.
n_1integer: Sample size of group 1.
n_2integer: Sample size of group 2.
rejection_prob_estfloating point number: Estimate of the rejection probability, i.e. of the probability that the null hypothesis is rejected by the test.
ll_conf_int_rejection_probfloating point number: Lower limit of the exact confidence interval for the true rejection probability.
ul_conf_int_rejection_probfloating point number: Upper limit of the exact confidence interval for the true rejection probability.
share_fixed_samples_group_1floating point number: Share of fixed samples of group 1 with respect to all n_simulation samples drawn to estimate properties of the test for a given total sample size. Fixed samples are samples that initially contained only a single value and were replaced by a more diverse sample obtained by drawing again until a more diverse sample occurred. If this share is too large, the estimates of the properties are not reliable. This column is only included, if fix_degenerated_samples is True.
share_fixed_samples_group_2: floating point number: Share of fixed samples of group 2 with respect to all n_simulation samples drawn to estimate properties of the test for a given sample size. Details are analogous to those of share_fixed_samples_group_1. This column is only included, if fix_degenerated_samples is True.

Return type:

pandas.DataFrame

Examples

>>> import numpy as np
>>> from scipy import stats
>>> import vstats
>>>
>>> min_abs_effect = 3  # minimal absolute effect to detect
>>>
>>> vstats.get_est_welchs_test_properties(
>>>     n=[20, 40, 80, 160, 320, 640],
>>>     share_n_group_1=0.3,
>>>     alpha_test=0.05,
>>>     rv_1=stats.norm(loc=min_abs_effect, scale=8),
>>>     rv_2=stats.norm(loc=0, scale=10),
>>>     conf_level_rejection_prob=0.95,
>>>     n_simulation=10000,
>>>     rng=np.random.default_rng(seed=165372562462609342146380204649518102930)
>>> )
...      n  n_1  n_2  rejection_prob_est  ll_conf_int_rejection_prob  ul_conf_int_rejection_prob
... 0   20    6   14              0.0998                    0.093992                    0.105842
... 1   40   12   28              0.1586                    0.151490                    0.165909
... 2   80   24   56              0.2784                    0.269631                    0.287298
... 3  160   48  112              0.5207                    0.510854                    0.530534
... 4  320   96  224              0.8081                    0.800242                    0.815778
... 5  640  192  448              0.9767                    0.973551                    0.979567

For an example on how to display this result, see the docstring of the function create_figure_for_rejection_probabilities of the package vplotly.

References

Devore, Jay L. & Berk, Kenneth N. (2018). Modern Mathematical Statistics with Applications. 2nd ed. New York, Dordrecht, Heidelberg, London: Springer. ISBN: 978-1-4614-0390-6.

Loveland, Jennifer L. (2011). Mathematical Justification of Introductory Hypothesis Tests and Development of Reference Materials. All Graduate Plan B and other Reports. 14. https://digitalcommons.usu.edu/gradreports/14

Georgii, Hans-Otto (2008). Stochastics: Introduction to Probability and Statistics. Berlin, New York: De Gruyter. https://doi.org/10.1515/9783110206760

vstats.welchs_test(x_1: List[float], x_2: List[float], alpha: float, conf_level_effect: Optional[float] = None) → Dict

Two-sided Welch’s test for two independent samples. Null hypothesis: The expected values of the population distributions underlying the samples x_1 and x_2 are equal. Alternative hypothesis: The expected values of the population distributions underlying the samples x_1 and x_2 are not equal. For details about Welch’s test and its assumptions see Loveland (2011, p. 48, 101) and section 10.2 in Devore (2018).