Resource guide / Data validation

Turn data-quality expectations into explicit tests

Validation checks whether information is complete, plausible, consistent, and fit for a defined use. It does not prove that every value is true.

Working principle: A passing rule means the record met a stated condition. It is not a substitute for source verification or domain judgment.

Scope

Validation and verification answer different questions

Validation asks whether data behaves as expected

It checks schema, required fields, ranges, relationships, uniqueness, formats, and business rules.

Verification asks whether a claim is supported

It connects the value or assertion to credible evidence, current sources, and an appropriate review decision.

Validation workflow

Build checks around the decision the data will support

01

Define the use and tolerance

Identify who will use the result, what could go wrong, and which failures should block release.

02

Write observable rules

Express expectations as testable conditions with clear pass, warning, and fail states.

03

Test across levels

Check fields, records, relationships, files, time series, and cross-source consistency.

04

Triage exceptions

Separate actual defects from legitimate edge cases, then assign an owner and resolution state.

05

Retest and record

Run the same checks after correction and preserve the rule version, result, and reviewer decision.

Rule families

Use a compact set of explainable checks

Completeness

Required fields, expected coverage, missing values, and gaps across time or entities.

Conformance

Types, formats, allowed values, units, and schema compatibility.

Consistency

Cross-field logic, related-table integrity, stable definitions, and agreement across sources.

Plausibility

Ranges, distributions, changes, and domain constraints that surface suspicious values.

Uniqueness

Duplicate identifiers, repeated events, and records that may represent the same entity.

Freshness

Observation dates, update cadence, expired records, and time-sensitive source requirements.

A practical next step

Bring a real question. Leave with a clearer evidence path.

Show us the dataset, claim, report, or market question your team needs to trust. We will map the relevant workflow and the review points it requires.