mxnet.test_utils.chi_square_check¶

mxnet.test_utils.chi_square_check(generator, buckets, probs, nsamples=1000000)[source]¶

Run the chi-square test for the generator. The generator can be both continuous and discrete.

If the generator is continuous, the buckets should contain tuples of (range_min, range_max) and the probs should be the corresponding ideal probability within the specific ranges. Otherwise, the buckets should contain all the possible values generated over the discrete distribution and the probs should be groud-truth probability.

Usually the user is required to specify the probs parameter.

After obtaining the p value, we could further use the standard p > 0.05 (alpha) threshold to get the final result.

Examples:

buckets, probs = gen_buckets_probs_with_ppf(lambda x: ss.norm.ppf(x, 0, 1), 5)
generator = lambda x: np.random.normal(0, 1.0, size=x)
p = chi_square_check(generator=generator, buckets=buckets, probs=probs)
assert(p > 0.05)

Parameters

generator (function) – A function that is assumed to generate i.i.d samples from a specific distribution. generator(N) should generate N random samples.
buckets (list of tuple or list of number) – The buckets to run the chi-square the test. Make sure that the buckets cover the whole range of the distribution. Also, the buckets must be in ascending order and have no intersection
probs (list or tuple) – The ground-truth probability of the random value fall in a specific bucket.
nsamples (int) – The number of samples to generate for the testing

Returns

p (float) – p value that the generator has the expected distribution. A higher value indicates a larger confidence
obs_freq (list) – Observed frequency of buckets
expected_freq (list) – The expected (ground-truth) frequency of the buckets