Messy Student Records SQL Practice Dataset
Student, enrollment, and grade records merged from three schools, with the mixed formats, padding, and duplicates left to clean.
- Tables
- 6
- Rows
- 64,384
- Size
- Medium
- Level
- Intermediate
- Domain
- Education
- Formats
- CSV · Parquet · Schema SQL
- In browser
- 2,368 rows in the browser
Download this dataset, free
No account, no email, no attribution required:
dedicated to the public domain under CC0 1.0, so commercial use and
redistribution are fine.
Open it in the browser playground, or download the CSV or Parquet files below and run messy_student_records.schema.sql in DuckDB to create the tables and load them.
This is a large dataset. The downloads here are the complete 64,384 rows. Opening it in the playground loads a 2,368-row sample instead, so the browser tab stays responsive. The schema is identical, so a query you write against the sample runs unchanged against the full download.
A school district merged the student records of three high schools into one set of tables. Each school exported from a different system, and nobody cleaned anything on the way in. The result is the kind of data a first analytics job hands you: about 64,000 rows where almost every column is text, and the queries are easy only after the data is fixed. Fixing it is the practice.
Each school's conventions survive in the data. Northgate
writes ISO dates and Fall 2024. Lakeside's legacy system writes
MM/DD/YYYY, 2024-FA, scores like 84%,
and pads every course code to ten characters, so an exact join from its
enrollments to the course catalog matches none of its 9,828 rows (see
why trailing spaces make a join match nothing). Ridgeview kept its records in a spreadsheet: DD/MM/YYYY
dates, FA24, hand-typed codes like math 101, and
scores of n/a, INC and ABS. The
schools table says which system each school used, and that is
the key to reading the rest. A Ridgeview date such as
03/09/2024 is the 3rd of September, but it also parses as a US
date without an error, as March 9th: 424 Ridgeview enrollment dates land on
the wrong day if you read them all one way. The same trap has its own page
at dates stored as text.
The merge added defects of its own. One batch of 300
Lakeside enrollments was loaded twice, identical except for
import_row. About sixty students who changed schools at the
semester break were registered again under a new student_id,
with their names in different case and their birth date in the new school's
format. Matching registrations on name alone pairs 787 of them, almost all
strangers who share a name; matching on name and a parsed birth date leaves
59, and one of those is still two different students at the same school. Ten
students were deleted from the roster after the export, so their enrollments
and grades point at a student_id that does not exist, and an
inner join drops them without a word.
Cleaning the course codes (trim, upper case, and one regular expression to
put the hyphen back) matches every enrollment to the catalog except 166,
which are for a retired course, HLTH-100, that no cleaning can
match. Enrollment status is written twelve ways for three meanings. Scores
hold 421 placeholders that no cast can read, and 46 typos above 100, like
870 for 87. The tools for all of it are on this site:
CAST and TRY_CAST,
string functions,
regular expressions, and
removing duplicates.
Download the CSV files to practice the whole job on your own machine, or open the dataset in the playground, which loads a sample of 90 students with every kind of defect kept in. All of it is synthetic. No real school, student, or family is represented, and every phone number uses the 555-01xx range reserved for fiction.
Schema
Join conditions
- students.school_id = schools.school_id
- contacts.student_id = students.student_id
- enrollments.school_id = schools.school_id
- enrollments.student_id = students.student_id
- grades.school_id = schools.school_id
- grades.student_id = students.student_id
- enrollments.course_code = courses.course_code (after cleaning)
- grades.course_code = courses.course_code (after cleaning)
Schema · Messy Student Records View dataset schema 6 tables
The three schools whose records were merged, and the system each one exported from. Most format differences in the other tables follow source_system.
The district course catalog, and the one clean table here. course_code is the canonical form, such as MATH-101, that the codes in enrollments and grades have to be cleaned to match.
One row per student registration, not per person: a student who transferred at the semester break has a second row at the new school. birth_date and grade_level are text in each school's own format.
Parents and guardians. relationship and phone are free text, with one phone number written five different ways.
One row per student per course per term, as exported, so every column but the keys is text. import_row is the only unique column: one batch was loaded twice.
Final grades from each school's gradebook, one per enrollment. score is text: a number, a percentage, or a placeholder such as n/a, INC or ABS.
Best for
- Cleaning before querying: trimming padded codes, casting text scores with TRY_CAST, and parsing dates written three ways.
- Deduplicating at two levels: a batch of rows loaded twice, and students registered twice under different ids.
- Finding what a join would quietly drop: enrollments for deleted students, and a course code no cleaning can match.
Starter questions
- How many ways is an enrollment's status written?
- Which course codes match nothing in the catalog?
- Which students were registered at two schools?
Practice questions
Work these in the playground against the browser sample. Each link opens the dataset with the question in the editor, ready for your query. Solutions are collapsed, so open one only after you have an answer of your own.
Beginner
-
1. How many different ways is an enrollment's status written, and how many rows use each one?
Profiling a column with GROUP BY before trusting it: three meanings, a dozen spellings.
Show solution
select status, count(*) as rows from enrollments group by status order by rows desc;
-
2. Which course codes in enrollments match no course in the catalog exactly? Show each code's length.
An anti-join that exposes padding: length() shows the spaces a result grid hides.
Show solution
select e.course_code, length(e.course_code) as length, count(*) as rows from enrollments e left join courses c on c.course_code = e.course_code where c.course_id is null group by e.course_code order by rows desc; -
3. Which students have a first or last name with stray spaces, or written all in capitals or all in lower case?
Comparing a value with its cleaned form, so the dirty rows are the ones that change.
Show solution
select student_id, school_id, '[' || first_name || ']' as first_name, '[' || last_name || ']' as last_name from students where first_name <> trim(first_name) or last_name <> trim(last_name) or first_name in (upper(first_name), lower(first_name)) or last_name in (upper(last_name), lower(last_name)) order by student_id; -
4. Which scores in grades are not numbers at all, even after removing a percent sign?
TRY_CAST returns NULL instead of failing, which turns 'will this cast?' into a WHERE clause.
Show solution
select school_id, score, count(*) as rows from grades where score is not null and try_cast(replace(trim(score), '%', '') as double) is null group by school_id, score order by rows desc;
-
5. How many students does each school have in each grade, with grade_level turned into a number?
Pulling the digits out of '09', 'Grade 9' and '9th' with regexp_extract, then casting.
Show solution
select s.school_name, cast(regexp_extract(st.grade_level, '[0-9]+') as integer) as grade, count(*) as students from students st join schools s on s.school_id = st.school_id group by s.school_name, grade order by s.school_name, grade; -
6. How many contacts are parents, guardians, or grandparents, once every spelling of each is counted together?
Folding spellings into categories with CASE and LIKE, where the order of the WHEN clauses decides what Parent/Guardian means.
Show solution
select case when lower(relationship) like '%grand%' then 'grandparent' when lower(relationship) like 'parent%' then 'parent' when lower(relationship) like '%guardian%' then 'guardian' end as relationship, count(distinct relationship) as spellings, count(*) as contacts from contacts group by 1 order by contacts desc;
Intermediate
-
1. What are the ten most-enrolled courses once the course codes are cleaned to the catalog's form?
Joining on a normalized key: trim, upper-case, and one regex that puts the hyphen back.
Show solution
select c.course_code, c.course_name, count(*) as enrollments from enrollments e join courses c on c.course_code = regexp_replace(upper(trim(e.course_code)), '^([A-Z]+)[ -]?([0-9]+)$', '\1-\2') group by c.course_code, c.course_name order by enrollments desc, c.course_code limit 10;
-
2. Which student_ids in enrollments have no row in students, and how many enrollment rows does each one have?
NOT EXISTS as an integrity check: the orphans an inner join would silently drop.
Show solution
select e.student_id, count(*) as enrollment_rows from enrollments e where not exists ( select 1 from students s where s.student_id = e.student_id ) group by e.student_id order by e.student_id;
-
3. Which enrollment rows were loaded twice? Show each duplicated row once, with the two import_row values.
Grouping by every column except the one that is unique per load.
Show solution
select school_id, student_id, course_code, term, enrolled_on, status, count(*) as copies, min(import_row) as first_import_row, max(import_row) as last_import_row from enrollments group by school_id, student_id, course_code, term, enrolled_on, status having count(*) > 1 order by student_id, course_code; -
4. What is each school's average final score, counting only scores that are real numbers between 0 and 100?
Cleaning before aggregating, and counting what was excluded rather than hiding it.
Show solution
select s.school_name, count(g.clean_score) as scored, count(*) - count(g.clean_score) as unusable, round(avg(g.clean_score), 1) as avg_score from ( select school_id, try_cast(replace(trim(score), '%', '') as double) as raw_score, case when raw_score between 0 and 100 then raw_score end as clean_score from grades where letter_grade is distinct from 'W' ) g join schools s on s.school_id = g.school_id group by s.school_name order by s.school_name; -
5. Turn every phone number into ten digits, whatever format it was typed in.
regexp_replace with the 'g' flag to strip everything but digits, then a second pass for the +1.
Show solution
select phone, regexp_replace( regexp_replace(phone, '[^0-9]', '', 'g'), '^1([0-9]{10})$', '\1' ) as digits from contacts where regexp_matches(phone, '[0-9]') order by length(phone) desc, phone; -
6. Which email addresses belong to more than one student registration once upper and lower case are treated as the same?
Comparing on lower(email): Lakeside wrote some addresses in capitals, so an exact match never pairs them.
Show solution
select lower(email) as email, count(*) as registrations, count(distinct email) as spellings, string_agg(student_id::varchar, ', ' order by student_id) as student_ids from students where email like '%@%' group by lower(email) having count(*) > 1 order by email; -
7. Which students have a birth date that cannot be read in their own school's format? Say whether each one is missing, not a date at all, or a day that does not exist.
TRY_STRPTIME per source to find the failures, then a CASE that explains them, because NULL is only one of the three.
Show solution
select st.student_id, s.school_name, st.birth_date, case when st.birth_date is null then 'missing' when not regexp_matches(st.birth_date, '[0-9]') then 'not a date' else 'impossible date' end as problem from students st join schools s on s.school_id = st.school_id where case st.school_id when 1 then try_strptime(st.birth_date, '%Y-%m-%d') when 2 then try_strptime(st.birth_date, '%m/%d/%Y') when 3 then try_strptime(st.birth_date, '%d/%m/%Y') end is null order by st.student_id;
Advanced
-
1. Parse enrolled_on into a real date using each school's own format. How many dates per school would land on the wrong day if you read them all as US dates?
The format belongs to the source, not the value: 03/09/2024 parses either way and only one is right.
Show solution
with parsed as ( select school_id, case school_id when 1 then try_strptime(enrolled_on, '%Y-%m-%d') when 2 then try_strptime(enrolled_on, '%m/%d/%Y') when 3 then try_strptime(enrolled_on, '%d/%m/%Y') end::date as enrolled_date, try_strptime(enrolled_on, '%m/%d/%Y')::date as read_as_us from enrollments ) select school_id, count(*) filter (where enrolled_date is null) as unparsed, count(*) filter ( where read_as_us is not null and read_as_us <> enrolled_date ) as wrong_if_read_as_us, min(enrolled_date) as first_date, max(enrolled_date) as last_date from parsed group by school_id order by school_id; -
2. Which students were registered twice because they changed schools? Match registrations on cleaned name and birth date.
A normalized matching key: a name alone pairs strangers, and the birth date has to be parsed per school before it can be compared.
Show solution
with clean as ( select student_id, school_id, lower(trim(first_name)) as first_name, lower(trim(last_name)) as last_name, case school_id when 1 then try_strptime(birth_date, '%Y-%m-%d') when 2 then try_strptime(birth_date, '%m/%d/%Y') when 3 then try_strptime(birth_date, '%d/%m/%Y') end::date as born from students ) select a.student_id as first_id, a.school_id as first_school, b.student_id as second_id, b.school_id as second_school, a.first_name, a.last_name, a.born from clean a join clean b on b.first_name = a.first_name and b.last_name = a.last_name and b.born = a.born and b.school_id <> a.school_id and b.student_id > a.student_id order by a.student_id; -
3. Build a clean enrollments table: one row per student, course and term, with standard codes, terms and statuses, the double-loaded rows removed, and missing students left out. Count the result by term and status.
Chaining the fixes in CTEs, deduplicating with QUALIFY, and checking the result with a count.
Show solution
with cleaned as ( select import_row, student_id, regexp_replace(upper(trim(course_code)), '^([A-Z]+)[ -]?([0-9]+)$', '\1-\2') as course_code, case when upper(term) in ('FALL 2024', '2024-FA', 'FA24') then 'Fall 2024' when upper(term) in ('SPRING 2025', '2025-SP', 'SP25') then 'Spring 2025' end as term, case lower(trim(status)) when 'w' then 'withdrawn' else lower(trim(status)) end as status from enrollments ), deduped as ( select * from cleaned qualify row_number() over ( partition by student_id, course_code, term order by import_row ) = 1 ) select term, status, count(*) as enrollments from deduped where student_id in (select student_id from students) group by term, status order by term, enrollments desc; -
4. For each department, what is the average final score and the share of scores below 70, using only scores that clean to a number from 0 to 100?
An end-to-end report: two cleaning steps feed one join, and the filter decides what counts.
Show solution
with scores as ( select regexp_replace(upper(trim(course_code)), '^([A-Z]+)[ -]?([0-9]+)$', '\1-\2') as course_code, try_cast(replace(trim(score), '%', '') as double) as score from grades ) select c.department, count(*) as graded, round(avg(s.score), 1) as avg_score, round(100.0 * count(*) filter (where s.score < 70) / count(*), 1) as pct_below_70 from scores s join courses c on c.course_code = s.course_code where s.score between 0 and 100 group by c.department order by avg_score desc; -
5. Account for every enrollment's course code, school by school: how many match the catalog exactly, how many match only after cleaning, and how many match nothing even then? Name the codes that are left.
A reconciliation, where the three counts must add up to the total, and what is left over is the finding: a course the catalog no longer has.
Show solution
with codes as ( select e.school_id, e.course_code in (select course_code from courses) as exact, regexp_replace(upper(trim(e.course_code)), '^([A-Z]+)[ -]?([0-9]+)$', '\1-\2') as clean_code from enrollments e ), checked as ( select *, clean_code in (select course_code from courses) as cleaned from codes ) select s.school_name, count(*) as enrollments, count(*) filter (where exact) as exact_match, count(*) filter (where cleaned and not exact) as after_cleaning, count(*) filter (where not cleaned) as no_match, string_agg(distinct clean_code, ', ') filter (where not cleaned) as unmatched_codes from checked join schools s on s.school_id = checked.school_id group by s.school_name order by s.school_name;
Practice with this dataset
Guided lessons for this dataset are still being written. In the meantime, open it in the playground: the tables are already loaded, and the starter query is waiting in the editor.
Open Messy Student Records in the playground →More practice datasets
Departments, employees, projects, and assignments, with a reporting line for self-join practice.
Customers, products, orders, line items, payments, and returns at realistic online-retail volume.
Sessions, pageviews, events, users, and campaigns for funnel and attribution practice.
Channels, campaigns, ad groups, and two years of daily performance where every metric is a ratio.
Accounts, merchants, and two years of posted transactions with no stored balance to read off.