SQL Statistical Aggregate Functions
Describe the shape of a column: its middle, its spread, its most common value, and how its values are distributed.
AVG, MIN, and MAX come with every
database. The functions on this page describe the rest of a distribution.
The names below are DuckDB's, which is what runs on this site; each section
says what PostgreSQL and MySQL call the same thing.
select count(*) as products,
round(avg(price), 2) as avg_price,
median(price) as median_price,
round(stddev_samp(price), 2) as price_stddev,
min(price) as min_price,
max(price) as max_price
from products The functions
| Function | Returns |
|---|---|
median(x) | The middle value; the average of the two middle values when the count is even. |
quantile_cont(x, p) |
The value at fraction p of the way up, interpolated
between rows.
|
quantile_disc(x, p) | The same point, as a value that actually occurs. |
stddev_samp(x) / stddev_pop(x) | Standard deviation, treating rows as a sample / the whole population. |
var_samp(x) / var_pop(x) | Variance: the standard deviation squared. |
mode(x) | The most frequent value; one of them, unspecified, on a tie. |
corr(y, x) | Pearson correlation between two columns, from -1 to 1. |
histogram(x) | A map from each distinct value to its count, in one cell. |
approx_count_distinct(x) |
An estimate of count(distinct x) that uses little
memory.
|
Like every aggregate except count(*), they skip NULLs, and they
work with GROUP BY and FILTER as usual.
Median and percentiles
select c.category_name,
median(p.price) as median_price,
round(avg(p.price), 2) as avg_price
from products as p
join categories as c on c.category_id = p.category_id
group by c.category_name
order by c.category_name Tools has a median of 9.50 and an average of 13.17: one $22 tool lifts the average, not the median. Pass a list of fractions to get several percentiles in one call. The result is a list:
select quantile_cont(price, [0.25, 0.5, 0.75]) as quartiles
from products
PostgreSQL writes percentiles as ordered-set aggregates, and DuckDB accepts
the same syntax:
percentile_cont(0.5) within group (order by price), and
percentile_disc for the discrete version. MySQL has no
percentile aggregate. SQL Server's PERCENTILE_CONT is a window
function and needs an OVER clause.
Spread
select round(stddev_samp(price), 2) as sample_stddev,
round(stddev_pop(price), 2) as population_stddev,
round(var_samp(price), 2) as sample_variance
from products
Plain stddev() and variance() mean the sample
versions in DuckDB and PostgreSQL, but MySQL's STDDEV() is the
population version. A one-row group returns NULL from the sample versions
and 0 from the population ones.
Most common value
select mode(status) as most_common_status
from orders mode hides its count and settles ties silently. When either
matters, count per value and rank the counts, as in
the most common value lesson.
PostgreSQL spells it mode() within group (order by status).
Correlation
select round(corr(price, cost), 3) as price_cost_corr
from products
0.992 is close to 1: price and cost rise together almost in a straight line.
PostgreSQL has corr too, along with the regr_*
family (regr_slope, regr_intercept), which DuckDB
also runs.
Outliers
with scored as (
select product_name,
price,
round((price - avg(price) over ()) / stddev_samp(price) over (), 2) as z_score
from products
)
select product_name, price, z_score
from scored
where abs(z_score) > 1.5
order by z_score desc
A z-score is how many standard deviations a value sits from the average. The
aggregates run as window functions here, with an empty over (),
so every row can be compared with the whole table without a join.
Histograms
select floor(price / 10) * 10 as price_band,
count(*) as products
from products
group by price_band
order by price_band
Equal-width bands and a GROUP BY work in every database.
DuckDB's histogram(floor(price / 10) * 10) returns the same
counts as a single map.
The histogram lesson covers named bands
and keeping the empty ones.
Approximate counts
select approx_count_distinct(customer_id) as approx_customers,
count(distinct customer_id) as exact_customers
from orders On Garden Shop's 24 orders the estimate is 17 customers and the exact count is 16. The approximation pays off on millions of rows, where an exact distinct count has to remember every value it has seen. Use the exact count whenever the number goes into a report.
Related
The middle value and the slow end, which an average blurs.
Median, percentiles, spread, mode, and histogram recipes.
Recompute from totals instead of averaging grouped averages.
Measure how far values spread around their average.
Tied rows share rank 1; break the tie on purpose.
MODE() for a quick look, counts and ROW_NUMBER for ties.