Most engineering blogs treat data quality like a problem you fix with a dedicated squad, a mountain of tooling, and a governance framework that outweighs your production database. I’m going to suggest something less comfortable: you can get 80% of the value by thinking differently, without anyone changing their job title. This isn’t cathedral-building. It’s about not letting the pipes burst while you’re busy choosing the right shade of stained glass.
Start With What You Already Have, Not What You Wish You Had
There’s a peculiar habit in our industry of designing quality systems for a data estate that doesn’t exist yet. We write policies for perfect schemas, complete lineage, and automated validation—while the actual tables are half-documented, the ingestion scripts run on a cron job someone set up two years ago, and the only person who understood the partitioning logic left for a startup. This isn’t cynicism. It’s just Tuesday.
The practical starting point is discovery, not design. Walk through the actual pipelines. Open the dashboards people actually use. Find the five reports that, if they broke silently, would cause a VP to send a terse email. Those are your quality targets. Everything else can wait.
A simple exercise: for each of those critical outputs, ask “What would make this report untrustworthy?” List the specific failure modes—late data, duplicate rows, a join key that went NULL without warning. Write them down. Congratulations, you now have a data quality spec that fits on a single page. It’s not glamorous, but it’s real.
The False Promise of One-Size-Fits-All Quality Metrics
Freshness, completeness, accuracy, consistency—these words show up in every data quality framework ever sold. They’re not wrong, but they’re abstraction traps. Saying “we measure completeness” means nothing until you define it for a specific table: does completeness mean every expected partition exists? Every expected customer ID? Every column populated that the downstream model assumes is non-null? Without that precision, you’re not measuring quality; you’re measuring your ability to generate dashboards about quality.
I’ve seen teams proudly display a dashboard showing 99.8% data freshness across all sources, while a single stale table—one that feeds the CFO’s monthly close report—went unnoticed for a week. The aggregate metric was fine. The business was not.
Instead, treat quality checks as surgical tests attached to specific assets. A test that says “row count for table X must not drop by more than 10% day-over-day” is worth ten abstract completeness scores. A test that says “column currency_code must not contain NULLs when amount > 0″ is an actual business rule, not a platitude.
Ownership Without a Data Quality Team
The standard advice is to assign data stewards. In practice, that often means someone gets a new title and no additional time, and the quality work happens exactly never. The alternative is ruthless, explicit ownership: the person who writes the pipeline is responsible for the checks. The person who builds the dashboard is responsible for documenting where the numbers come from. No handoffs, no “quality gate” that someone else operates.
This only works if the checks are trivial to add. If writing a data quality test requires a pull request to a separate repository that only the platform team understands, it won’t happen. The test needs to live next to the transformation code, in the same repo, using the same language. A SQL expression in a YAML file, checked into the dbt project or equivalent, is the right level of friction. Anything more, and you’re designing for the team you wish you had.
One team I worked with had a simple rule: every new data model in the warehouse required at least one not-null test and one uniqueness test. That’s it. Compliance was high because the bar was low. Over a year, they caught dozens of silent regressions that a more ambitious framework—still under design—would have missed entirely.
Monitoring: Alert on What Breaks, Not on What’s Interesting
Alert fatigue is the silent killer of data quality initiatives. If you set up Slack notifications for every minor freshness deviation, people will mute the channel. If you page on-call for a 1% row count drop in a table that feeds an experimental dashboard, you’ll lose credibility.
Effective monitoring starts with a severity taxonomy that everyone agrees on, and it must be embarrassingly simple. I use three levels:
- Blocking: data is missing or wrong in a way that stops a business-critical process. Page someone.
- Warning: something looks off, but downstream processes still run. File a ticket, or surface it in a daily summary.
- Informational: interesting, but no one needs to act now. Log it to a dashboard and move on.
The key is that blocking must be defined narrowly enough that it actually means “stop what you’re doing.” If you have more than five blocking alerts in a month, the definition is too broad. Refine it.
One useful pattern: tie alerts to the consumers of the data, not the producers. If the finance team’s monthly close report depends on three tables, the alert fires when any of those tables fails its freshness check within 24 hours of the close deadline. The rest of the month, a delay in those same tables might be a warning at most. Context matters more than absolute thresholds.
Documentation That Someone Will Actually Read
I have a bias against data catalogs that require a separate login. If the documentation for a table isn’t within two clicks of the table itself, it won’t be read. The best documentation I’ve seen is embedded directly in the code that defines the schema: a comment block at the top of a SQL file that explains what the table is for, who uses it, and what the known sharp edges are.
For example, a comment like “This table aggregates daily sales by region; NULL regions indicate online orders that haven’t been geocoded yet—exclude them from regional reporting” is worth more than a beautifully formatted wiki page that no one updates. The documentation lives because it’s in the same pull request as the code change. The proximity is the point.
If you must have a catalog, make it automated. Scrape the comments. Render them as static pages. Never ask an engineer to document the same thing in two places. They won’t, and they’ll resent you for asking.
Build Quality In, Don’t Inspect It In Later
There’s a manufacturing analogy that gets overused in software, but it fits here: inspecting quality at the end of the line is expensive. If your data quality checks only run after the data lands in the warehouse, you’re already too late. The bad data has been joined, aggregated, and served to dashboards. Fixing it means backfills, apologies, and a loss of trust that takes weeks to rebuild.
The shift is to move checks as far upstream as possible. Validate schema and basic constraints at ingestion, before the data touches anything else. If a source system sends a file with a missing column, reject it immediately and alert the provider. If an API starts returning a new value in an enum field, log a warning and quarantine the records. These are not “data quality team” tasks; they’re engineering tasks that any competent pipeline developer can implement.
One team I know added a five-line Python script to their ingestion layer that checked for NULLs in a handful of critical columns. It took twenty minutes to write and has caught more incidents than their entire monitoring stack. The lesson: simple, early checks beat elaborate, late ones every time.
The Pragmatic FAQ
What’s the minimum viable set of data quality checks?
Start with three: freshness (did the data arrive on time?), volume (did we get about the right number of records?), and schema (are the columns we expect actually there?). These three catch a surprising fraction of real-world failures. Add business-rule checks—like “discount amount must not exceed total price”—only after the basics are stable and monitored.
How do we get engineers to care about data quality without a mandate?
Make the pain visible. When a dashboard breaks, don’t just fix the data—trace it back to the pipeline change that caused it, and show the engineer the downstream impact. Most engineers don’t want to ship broken things; they just don’t see the connection between their code change and the analyst’s panicked Slack message. Close that feedback loop, and ownership follows naturally.
When should we actually consider a dedicated data quality team?
Not before you’ve exhausted the embedded approach. If you have more than a dozen critical data assets, a complex web of interdependencies, and regulatory requirements that demand formal sign-offs, a small team focused on quality infrastructure might make sense. But even then, their job should be to build tools and frameworks that enable the pipeline owners, not to take over responsibility. The moment quality becomes someone else’s job, it stops being everyone’s job—and that’s usually the beginning of the end.
How do we handle data quality in a fast-changing environment where schemas shift weekly?
Embrace schema-on-read where it makes sense, but enforce contracts at the handoff points. If a source system can change its output format without warning, you need a contract: a formal or informal agreement that certain fields will remain stable, with a process for communicating changes. Failing that, write defensive ingestion that can tolerate new fields without breaking, and alert on unexpected changes rather than blocking them. The goal is to stay informed without grinding development to a halt.