____                        
                           |\   ` //  /\\   /\\   /\\       
                            \\   //  || || || || || ||      
         /'\\ \\/\\  /'\\  / \\  \\  || || || || || ||      
        || || || || || || || ||   )) || || || || || ||      
        || || || || || || || ||  //  || || || || || ||      
        \\,/  \\ \\ \\,/   \\/  /'    \\/   \\/   \\/       
                                                             

    Rust port of Paul Uszak's new randomness testing suite for TRNG Makers 
    generating in the sub 1 MB space. Successor to John Walker’s venerable `ent`.

    Pipe binary entropy data into the `onod3000` command line tool or call it
    with `-f <filename>` to load entropy from a file. Add `--depth-off` to
    skip the survival depth.

Onod3000 provides multiple randomness tests, each outputting a statistic, a z-score and a p-value to assess the conformity of the data to random behavior. On top of that it rates a whole sample with one calibrated score, see Scoring.

Testing Methodology

• Null Hypothesis: The data conforms to random behavior.
• P-Value Interpretation: Low p-values (< 0.01) indicate a significant deviation from randomness.
• Pass: a test passes when its calibrated p-value (Cal-P, see Scoring) is at least 0.01. Samples under 256 bytes have no calibration, so there the test's own p-value is used.

Even a perfect source fails each test 1% of the time, so with 15 tests about 1 in 7 good samples shows at least one ❌. Use the score for an overall verdict.

All tests have been carefully ported from Paul Uszak's Java code (ent3000-0.6.0-beta). Java libraries were substituted with Rust crate equivalents when possible. The suite includes the following tests:

• Monobit Test: Evaluates the balance of 0s and 1s.
• Chi-Square Tests: Tests the uniformity of bits and bytes.
• MeanByte Test: Checks if the mean byte value aligns with expected randomness.
• Kolmogorov-Smirnov (KS) Test: Assesses uniformity of data distribution.
• Pi Test: Uses Monte Carlo methods to approximate π.
• Shells Test: Analyzes distances in 3D space.
• Gaps Test: Measures gaps between occurrences of a specific value.
• Avalanche Test: Analyzes bit-level changes in data chunks.
• Runs and RunUps Tests: Checks for sequential patterns in data.
• Prediction Test: Assesses the predictability of next bits.
• UnCorrelation Test: Evaluates correlation between shifted data.

More details on some individual tests available here

Sample Output

cat /dev/random | head -c 259200 | onod3000

Testing 259200 bytes from stdin.
----------------------------------------------------------------
Randomness Test           Value   Z-Score   P-Value   Cal-P Pass
----------------------------------------------------------------
Shannon                   7.999   -0.3799   0.7040  0.6693   ✅
Monobit                   0.500    0.6722   0.5014  0.5007   ✅
ChiBit                   12.457    1.1141   0.1320  0.1320   ✅
ChiByte                 244.810   -0.4512   0.6656  0.6630   ✅
MeanByte                127.442   -0.3986   0.6902  0.6934   ✅
Compression               1.000    0.0868   0.9308  1.0000   ✅
Kolm.-Smirnov             0.001    0.4724   0.9789  0.9604   ✅
Pi                        3.138   -1.0970   0.2726  0.2602   ✅
Shells                   24.512   -1.1506   0.9068  0.8923   ✅
Gaps                      5.994   -0.7085   0.7405  0.6891   ✅
Avalanche                80.135    0.0214   0.9829  0.0848   ✅
Runs                 129129.000    0.1080   0.9140  0.9160   ✅
RunUps                    2.604    1.1345   0.1066  0.1073   ✅
Prediction           259198.000    0.3321   0.2254  0.2172   ✅
UnCorrelation             0.000    0.0144   0.9885  0.9842   ✅
----------------------------------------------------------------
15/15 tests passed.
----------------------------------------------------------------
Score 0.06 (pass) · weakest: Avalanche p=0.085 · Depth 21.0+ bits
----------------------------------------------------------------

Scoring

Besides the individual tests, onod3000 rates a whole sample with one score and a survival depth. Both need at least 256 bytes.

Calibrated p-values

Some of the tests' own p-values are wrong on random data. They rely on approximations that only hold for large samples, or on constants that don't match the real spread:

• Shannon: compares against exactly 8 bits, but measured entropy comes out lower on finite samples.
• Gaps: ignores repeats past the end of the sample.
• Compression: its standard deviation is made up.
• Avalanche: divides by the spread of a single window instead of the spread of the mean.

So on truly random data Shannon fails every sample up to 4 KB, Gaps fails every sample up to 2 KB, and Avalanche never fails at all.

The score therefore doesn't use those p-values. examples/calibrate.rs ran every test on 20,000 random samples at each size from 256 B to 1 MB (powers of two) and stored where each test's z-score landed. The calibrated p-value (Cal-P) says how unusual a z-score is compared with those random runs:

• Sizes in between are interpolated, and sizes above 1 MB are extrapolated, both linearly in 1/√n. That is how Shannon's bias and Avalanche's spread change with size, and it leaves the other tests unchanged.
• Compression's z-score on random data is a fixed value for each length (deflate's overhead), so it is compared against a reference random sample of the same length instead.

The Z-Score and P-Value columns are unchanged and still match the Java original.

Score

The 15 calibrated p-values are combined with the Cauchy combination test (Liu & Xie 2020). Its null distribution was also measured by simulation, so the combined p-value stays correct even though several tests look at the same thing (Shannon, ChiByte, KS and MeanByte all come from the byte counts).

The score is -log10(combined p), capped at 12:

Score	Verdict	Meaning
< 2	pass	nothing unusual
2 – 4	suspect	as unusual as 1 in 100 to 1 in 10,000 random samples
≥ 4	fail	rarer than 1 in 10,000 random samples

A perfect source does not score 0. On random data the combined p-value is uniform, so 1% of good samples score 2 or more and 0.01% score 4 or more. A source that keeps landing in suspect is the real warning sign. The weakest test (lowest calibrated p-value) is reported with the score.

Accuracy

How often fresh random data (not used to build the tables) gets a combined p-value below each threshold. For a correct calibration that is exactly the threshold: 5%, 1% and 0.01%. Values a little under target mean the score errs toward passing good data.

Sample size	Random samples checked	p < 0.05	p < 0.01 (score ≥ 2, suspect)	p < 0.0001 (score ≥ 4, fail)
256 B	2,000,000	4.98%	0.88%	0.0046%
1,000 B	20,000	4.77%	0.95%	–
1 KB	1,000,000	4.84%	0.88%	0.0025%
4 KB	1,000,000	4.62%	0.84%	0.0022%
10,000 B	20,000	4.88%	0.90%	–
100,000 B	20,000	5.23%	1.06%	–
259,200 B	2,000	4.60%	0.85%	–
1 MB	2,000	5.25%	1.10%	–
1,500,000 B	2,000	4.95%	1.05%	–
4,000,000 B	2,000	4.50%	0.65%	–

Sizes that are not a power of two use interpolation, and sizes above 1 MB use extrapolation. "–" means too few samples to measure a 0.01% rate. Reproduce with cargo run --release --example calibrate -- --check N --sizes A,B,....

Survival depth

Ported from Comscire's QNGmeter: the tests are run on the first 256 B, 512 B, 1 KB and so on up to the whole sample. The depth is log2 of the number of bits in the longest prefix that passed, where all shorter prefixes passed too. A prefix fails when its combined p-value drops below 10⁻⁴, QNGmeter's freeze level.

• + means the whole sample survived.
• A depth below the sample size shows roughly how much data it takes for a flaw to show up.
• Computing the depth takes up to twice as long. --depth-off skips it.

Rebuilding the calibration

cargo run --release --example calibrate                  # simulates and writes src/calibration/table.rs, ~40 min on 14 cores
cargo run --release --example calibrate -- --from-dump target/calibration-runs.bin   # rebuilds the table from saved runs in seconds
cargo run --release --example calibrate -- --check 2000  # checks the compiled-in table on fresh data

Rust Library

use onod3000::Onod;

fn main() {
    let data = vec![...]; // Your binary data
    let (_, _, p_value) = Onod::monobit(&data);
    println!("Monobit Test P-Value: {:.4}", p_value);

    // One test with its calibrated p-value: (statistic, z, p, calibrated p)
    let (_, _, _, p_cal) = Onod::calibrated("gaps", &data);
    println!("Gaps calibrated P-Value: {:?}", p_cal);

    // The whole sample; `true` also computes the survival depth
    let score = Onod::score(&data, true);
    println!("Score {:.2} ({}), depth {:.1} bits", score.score, score.verdict.as_str(), score.depth_bits);
}

Bindings

The library ships with FFI and optional Python bindings so it can be used as C or Python library.

C

Link against the static or dynamic library and include include/onod3000.h:

double result[3];                               // statistic, z-score, p-value
onod_run("monobit", data, len, result);

OnodScore score;
if (onod_score(data, len, true, &score))        // true: also compute the survival depth
    printf("score %.2f, weakest %s\n", score.score,
           score.weakest >= 0 ? onod_test_name(score.weakest) : "-");

OnodScore.verdict is 0 pass, 1 suspect, 2 fail or 3 insufficient data. p_cal holds the calibrated p-value per test, in onod_test_name order.

Python

Install using maturin:

maturin build --release --features python

from onod import Onod

stat, z, p = Onod.run("monobit", data)
score = Onod.score(data)            # Onod.score(data, depth=False) skips the survival depth
print(score["score"], score["verdict"], score["weakest"])

Comparison

To test the implementation three files of binary random data have been analyzed using the original Java program (version 0.6.0-beta) and this Rust port, expecting the same p-values. The test files can be found in the test folder.

Each table has two parts:

• Java ent3000 / Rust onod3000: each test's own p-value in both programs. These should match. Java vs Rust notes where they don't.
• Rust calibrated: the p-value this port uses for the Pass mark and the score (see Calibrated p-values). It is the same z-score, compared with 20,000 random samples of the same size instead of a textbook distribution. The Java program has no equivalent. It differs from the raw value most where the raw value is wrong for the sample size, like Shannon and Gaps on the 1 KB file.

Below each table is the Score line that onod3000 prints for that file.

Testing 1024 bytes from test1.bin (/dev/random)

Test	Java ent3000	Rust onod3000	Java vs Rust	Rust calibrated
Shannon	N/A	0.0000	Not in Java	0.0379
Monobit	0.6746	0.6746		0.6676
ChiBit	0.9299	0.9299		0.9313
ChiByte	0.9819	0.9819		0.9812
MeanByte	0.7964	0.7964		0.8037
Compression	N/A	0.0146	Not in Java	1.0000
KS	0.5313	0.9998	Different KS method	0.9986
Pi	0.3784	0.3784		0.3760
Shells	0.7559	0.7559		0.8171
Gaps	0.0000	0.0000		0.5107
Avalanche	0.8794	0.8794		0.4529
Runs	0.0850	0.0850		0.0886
RunUps	0.6537	0.6537		0.6972
Prediction	0.6164	0.6164		0.6305
UnCorrelation	0.0921	0.0918	Close enough	0.1029

Score 0.02 (pass) · weakest: Shannon p=0.038 · Depth 13.0+ bits

Feel free to provide more results, and I'll continue updating the table!

Testing 259,200 bytes from test2.bin (/dev/random)

Test	Java ent3000	Rust onod3000	Java vs Rust	Rust calibrated
Shannon	N/A	0.6853	Not in Java	0.7644
Monobit	0.2103	0.2103		0.2096
ChiBit	0.5590	0.5590		0.5617
ChiByte	0.3748	0.3748		0.3722
MeanByte	0.0183	0.0183		0.0195
Compression	N/A	0.9308	Not in Java	1.0000
KS	0.2900	0.1158	Different KS method	0.0975
Pi	0.9765	0.9765		0.9700
Shells	0.7235	0.7235		0.7126
Gaps	0.6425	0.6425		0.5852
Avalanche	0.9995	0.9995		0.9539
Runs	0.3310	0.3310		0.3271
RunUps	0.9393	0.9393		0.9450
Prediction	0.1519	0.1519		0.1486
UnCorrelation	0.4497	0.4497		0.4519

Score 0.29 (pass) · weakest: MeanByte p=0.020 · Depth 21.0+ bits

Testing 259,200 bytes from test3.bin (Hardware QRNG)

Test	Java ent3000	Rust onod3000	Java vs Rust	Rust calibrated
Shannon	N/A	0.6978	Not in Java	0.8522
Monobit	0.4926	0.4926		0.4922
ChiBit	0.8428	0.8428		0.8436
ChiByte	0.5785	0.5785		0.5766
MeanByte	0.9601	0.9601		0.9583
Compression	N/A	0.9308	Not in Java	1.0000
KS	0.7395	0.7777	Different KS method	0.7233
Pi	0.2806	0.2806		0.2718
Shells	0.7711	0.7711		0.7504
Gaps	0.1937	0.1937		0.1532
Avalanche	0.9932	0.9932		0.4964
Runs	0.2500	0.2500		0.2491
RunUps	0.9710	0.9710		0.9775
Prediction	0.7173	0.7173		0.7221
UnCorrelation	0.4674	0.4674		0.4686

Score 0.07 (pass) · weakest: Gaps p=0.153 · Depth 21.0+ bits

Conclusions

Although the implementation of the randomness tests closely follows the original Java logic, the Kolmogorov-Smirnov test deliberately differs. The Java version compares the data against a pseudo-random uniform sample, which makes the p-value change from run to run for the same input. This port instead runs a one-sample KS test of the byte values against the exact discrete uniform distribution over 0..255, with p-values from the asymptotic Kolmogorov distribution (Stephens' small-sample correction). The result is deterministic and calibrated: on /dev/urandom it fails about 1% of the time at alpha = 0.01, slightly less on small inputs because the discrete null makes the test a little conservative.

All other differences are generally negligible for practical purposes and do not affect the overall functionality or statistical significance of the tests.

The raw Shannon and Gaps p-values of 0.0000 on the 1 KB file are not a flaw in the data. Both tests reject almost every random sample that small (see Calibrated p-values), in the Java original too for Gaps. Calibrated, they come out at 0.0379 and 0.5107, and all three files pass.

License

This project is licensed under the Apache License 2.0. See the LICENSE and NOTICE files for details, and THIRD_PARTY_NOTICES.md for the licenses of the dependencies. Releases up to 0.1.0 were licensed under the MIT License.

The ported tests in src/uniformity/ are based on Paul Uszak's Java original, which is MIT-licensed. Its notice is kept in each of those files and below.

License of Original Java Implementation

Copyright (c) 2023 Paul Uszak.    

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.