Kolmogorov–Smirnov Test Calculator

Introduction: Understanding the Kolmogorov–Smirnov Test

The Kolmogorov–Smirnov (KS) test is a versatile nonparametric method for comparing empirical distributions. It gauges whether two samples originate from the same continuous distribution without assuming any particular functional form. Unlike parametric tests that rely on specific distributional assumptions, the KS test operates solely on the ordering of data values, making it especially useful when little is known about the underlying populations. It examines the maximal vertical distance between the empirical cumulative distribution functions (ECDFs) of the samples and uses that distance to infer whether the observed discrepancy is likely under the null hypothesis of identical distributions.

To appreciate the logic behind the KS test, imagine plotting the ECDF of each sample on the same axes. These step functions start at zero and jump upward at each observed data point until they reach one. If both samples arise from the same population, the two ECDFs should trace similar paths, though random sampling variation will cause slight deviations. The KS statistic, denoted $D$, captures the largest of those deviations. A large $D$ suggests that one sample tends to accumulate probability mass faster than the other in some region, indicating a systematic difference between the distributions.

The intuition for the test dates back to work by Andrey Kolmogorov and later Nikolai Smirnov in the mid‑20th century. They derived the exact distribution of the test statistic under the null hypothesis, revealing a surprising universality: after appropriate scaling, the distribution of $D$ depends only on the combined sample sizes, not on the particular form of the common distribution. This property allows the KS test to function as a distribution‑free goodness‑of‑fit tool, applicable to both theoretical comparisons and practical data analysis.

Computation of the KS Statistic

Suppose we have two independent samples $X_1,X_2,...,X_n$ and $Y_1,Y_2,...,Y_m$. Their ECDFs are defined by

$F_n\left(x\right)=\frac{{\#}_{X\le x}}{n}$ and $G_m\left(x\right)=\frac{{\#}_{Y\le x}}{m}$.

The KS statistic is then $D={sup}^x\left|F_n\right(x)-G_m(x\left)\right|$. Our calculator constructs both ECDFs by sorting the samples and scanning through unique data values, updating cumulative counts at each step. This algorithm runs in linear time after the initial sort and yields the exact $D$.

Approximating the p‑Value

The null distribution of the two-sample KS statistic can be expressed through the complementary cumulative distribution function of the Kolmogorov distribution. When sample sizes are moderate, a useful approximation for the p‑value is

$p=2{\sum }_j^{=}{-}^{j-1}{e}^{-2j^2{\Lambda }^2}$ where $\Lambda =(\sqrt{\frac{\mathrm{nm}}{n+m}}+0.12+\frac{0.11}{\sqrt{\frac{\mathrm{nm}}{n+m}}})D$.

By summing terms in this alternating series until they become negligibly small, the calculator estimates the tail probability that a difference at least as large as the observed $D$ would arise under identical parent distributions. Small p‑values—typically below 0.05—indicate that such a difference is unlikely by chance alone, leading to rejection of the null hypothesis.

A worked example: two machining lines

Consider two manufacturing processes that produce slightly different measurements for a critical component. Suppose sample A consists of the diameters [5.01, 4.98, 5.03, 5.00, 5.02] millimeters, while sample B produces [4.97, 4.99, 5.04, 5.01, 4.96]. When these values are entered into the calculator, the sorted ECDFs reveal a maximum deviation of $D=0.4$. With both samples containing five observations, the approximation above yields a p‑value around 0.70. This high p‑value implies that the observed difference could easily result from random variation, so we fail to reject the hypothesis that both processes share the same distribution.

Formula: Algorithmic Steps Summarized

Interpreting Results

It is important to remember that the KS test evaluates differences in cumulative distributions, not just in means or variances. As a result, it is sensitive to shifts in location, changes in dispersion, or any other distributional discrepancy. However, it is less powerful than specialized tests if a specific alternative hypothesis is known. For instance, if you expect only a difference in means, a t‑test may detect smaller shifts than the KS test, because the latter accounts for all possible differences.

The two-sample KS test is symmetric: swapping the sample labels yields the same $D$ and p‑value. It also handles samples of different sizes with ease. When sample sizes are very small, p‑value approximations may be less accurate, so exact tables or permutation tests provide better control of Type I error. For large samples, the asymptotic formula used here performs well.

Beyond hypothesis testing, the KS statistic offers a convenient measure of distance between distributions. In machine learning and data mining, it can quantify how well synthetic data mimic real-world observations. In finance, analysts compare historical return distributions across market regimes using KS distances, highlighting structural changes that may affect risk assessments.

Although the classic KS test applies to continuous data, variants exist for discrete distributions and for one-sample goodness-of-fit assessments. The one-sample version compares the empirical distribution to a fully specified theoretical CDF. Extensions also handle cases with tied observations by applying mid‑rank adjustments or by adding small random jitter.

The KS test's reliance on empirical distributions means it does not rely on parametric assumptions, but it still assumes independent observations. Correlated data, such as time series, can inflate the probability of false positives because neighboring values influence each other. When analyzing such data, consider block bootstrap methods or other techniques that account for dependence.

Historically, the test gained prominence in the mid-20th century as statisticians sought methods that were robust to unknown distribution shapes. Its use has expanded with modern computing power, enabling practitioners to evaluate complex models by comparing simulated and observed data. The KS statistic also underlies metrics such as the Earth mover's distance and the Cramér–von Mises criterion, situating it within a broader family of distributional comparisons.

To deepen your understanding, experiment with the calculator by entering samples drawn from known distributions. Try comparing normal and uniform samples, or two normal samples with different means. Observe how the KS statistic reacts to shifts, spread changes, and outliers. Such experimentation builds intuition for how cumulative differences manifest in practice.

Because every computation occurs in your browser, no data are transmitted elsewhere. You can save this page and use it offline to analyze sensitive datasets securely. The algorithm scales well for small to moderate sample sizes, though extremely large datasets may incur noticeable sorting time. Nonetheless, for educational purposes and quick analyses, the KS test's simplicity and generality make it an invaluable addition to any mathematical toolkit.

Entering your two samples

1. Paste your first group of measurements into Sample 1 as numbers separated by commas, for example 5.01, 4.98, 5.03. Spaces around the commas are ignored, and the two samples do not need to contain the same number of values.
2. Paste your second group into Sample 2 the same way. Make sure both samples are recorded in the same units — comparing centimetres against inches will inflate $D$ even when the underlying processes are identical.
3. Press Compute KS Test. The calculator sorts each sample, walks the merged list of values to find the largest gap between the two empirical CDFs, and reports that gap as $D$ alongside the approximate p‑value and both sample sizes.

What the test can and cannot tell you

A small p‑value means the two samples are unlikely to have come from one shared distribution, but it does not tell you how they differ — a shift in centre, a change in spread, or a difference in shape all register the same way. When you already suspect a specific difference, such as a change in the mean, a targeted test like a t‑test will usually detect it with fewer observations. The p‑value here comes from the asymptotic Kolmogorov series, which is accurate for moderate and large samples but grows rough when either sample drops to a handful of points; in that regime an exact permutation test is more trustworthy. The test also assumes the observations within each sample are independent, so autocorrelated data such as consecutive readings from a time series can produce false positives. Finally, the classic form assumes continuous data: heavy ties from rounded or discrete values make the statistic conservative, and jittering or a tie-aware variant is worth considering. All computation stays in your browser, so you can analyze sensitive measurements without sending them anywhere.