Most engineering teams file data documentation under literacy. The schemas are confusing, so someone should write clearer descriptions. The dashboards have drifted, so a technical writer should update the runbooks. The quiet assumption is that bad documentation is a writing failure—not enough time, not enough skill, not enough willingness from the people who produce the words.
That assumption falls apart the moment you look at documentation that was perfectly clear on the day it shipped. Six months later, it’s worse than useless. The grammar is still fine. The sentences still hang together. But the systems it describes have moved on without it. The problem was never the writing. The problem is that documentation has a maintenance surface, and almost nobody treats it that way.
Treat data documentation as a writing problem, and you’ll optimise for the wrong things. You’ll hire people who can craft lovely prose. You’ll invest in style guides and review workflows. You’ll end up with a pile of static artefacts that decay at a rate proportional to the velocity of the teams producing the data. The writing was never the bottleneck. Coupling was.
The Half-Life of a Written Description
Data systems change constantly. A column gets renamed during a migration. A business metric gets redefined after a quarterly review. An upstream pipeline adds a transformation step that quietly shifts the distribution of a field. Each change is small enough that nobody files a ticket to update the docs. Together, they make the existing descriptions misleading.
The half-life of a written description isn’t set by how well it was written. It’s set by how tightly it’s coupled to a moving target. The more manual the coupling, the shorter the half-life. A beautifully worded description of a customer_lifetime_value field is worthless if the definition changed last sprint and the description didn’t.
Teams that understand this stop trying to write better documentation and start trying to reduce the maintenance burden of keeping it accurate. They treat documentation as a side effect of the system’s design, not as a separate deliverable. That shift in mindset changes everything.
Why “Write Better Docs” Is a Trap
When a data team notices their documentation is out of date, the default reflex is a documentation sprint. They block a week, assign owners, and produce a flurry of updates. For a brief moment, the documentation matches reality. Then the sprint ends, the teams go back to building, and the decay resumes.
This cycle isn’t expensive because writing is hard. It’s expensive because it treats documentation as a batch process. Batches work when the underlying asset is stable. Data systems are not stable. The only way to keep documentation aligned with a moving system is to make the alignment continuous. That’s not a writing challenge. It’s an integration challenge.
A more productive question than “Who should write the docs?” is “Where should the docs live so they update themselves?” If the answer involves a human copying information from one place to another, you’ve already lost.
Documentation as a Coupling Problem
Coupling is an architectural idea. Two components are coupled if a change in one forces a change in the other. In software engineering, we spend enormous effort reducing coupling between modules because tight coupling makes systems brittle and expensive to maintain. The same logic applies to documentation.
Think about a data dictionary entry that describes a field in a production database. The entry has the field name, its type, a human-readable description, and maybe some example values. The field name and type live in the database schema. The description and examples live only in the documentation. The two artefacts are tightly coupled: if the schema changes, the documentation must change. But the coupling is manual. Nothing enforces it.
The fix isn’t to write the description more carefully. The fix is to break the manual coupling. Pull the field name and type directly from the schema at build time. Generate the documentation page from a template that grabs live metadata. Now the structural parts of the documentation stay accurate without anyone lifting a pen. The human-written description still needs maintenance, but its scope is narrower, and its half-life is longer because it’s attached to a concept, not a specific implementation detail.
Where Automation Actually Helps (and Where It Doesn’t)
There’s a temptation to throw tooling at the problem. We’ll auto-generate everything from the code. That impulse is half-right. Automating the extraction of structural metadata—column names, types, relationships, lineage—is straightforward and high-impact. Those facts already live in the system. Duplicating them by hand is pure waste.
But automation can’t write the context. A column named status_code with values like 0, 1, and 2 needs a human to explain what those codes mean and why they exist. That explanation isn’t derivable from the schema. It lives in the heads of the people who designed the system. The maintenance problem for that kind of knowledge is different: it’s about making sure the explanation survives staff turnover and organisational forgetting, not about keeping it in sync with a schema migration.
The practical line is this: anything derivable from a machine-readable source should never be written by hand. Anything that requires judgement should be written as close as possible to the thing it describes, so the person changing the code is forced to confront the documentation. A comment in the schema definition file is worth ten wiki pages.
Proximity Is a Maintenance Strategy
One of the most underrated patterns in documentation maintenance is physical proximity. If the description of a field lives in a separate system—a wiki, a Confluence space, a Google Doc—the person changing the field’s definition in code won’t see it. They might not even know it exists. The documentation drifts silently.
Put the description in the same repository as the schema definition. Use a format the build pipeline can extract and publish. Now the person making the change sees the description right next to the code they’re modifying. They have a chance to update it. They might still ignore it, but the friction is lower, and the visibility is higher. Over time, that small change in proximity shifts the culture. Documentation stops being a separate chore and becomes part of the same commit.
This isn’t a writing improvement. It’s a workflow design improvement. It acknowledges that engineers are lazy in the right way: they’ll do the thing that requires the least context-switching. Make documentation maintenance the path of least resistance, and it’ll happen more often.
The Social Contract of Data Documentation
There’s a social dimension to the maintenance problem that technical solutions alone can’t fix. Documentation decays because nobody feels responsible for it after the initial write-up. The person who created the dataset has moved to another team. The consumers of the data assume the provider maintains the docs. The provider assumes the consumers will ask if something is unclear. Both assumptions are optimistic.
Clear ownership is a maintenance strategy. Every piece of documentation should have an owner who’s on the hook for its accuracy, and that ownership should be visible. The owner doesn’t have to do all the updating themselves, but they’re the person who gets pinged when something looks off. Without ownership, documentation is an orphan. Orphans don’t get maintained.
Ownership also forces a capacity conversation. If a team owns fifteen datasets and each dataset needs ongoing documentation maintenance, that maintenance should show up in their sprint planning. If it doesn’t, the organisation is implicitly deciding that documentation decay is acceptable. That might be a reasonable trade-off in some contexts, but it should be an explicit choice, not an accident.
Designing for Low-Maintenance Documentation
If you accept that documentation is a maintenance problem, you start designing systems differently. You ask a different set of questions during design reviews:
- What parts of this documentation can be generated from the system itself?
- Where will the human-written parts live, and how close are they to the code?
- Who owns this documentation, and how will they know when it needs updating?
- What’s the expected rate of change for the underlying system, and how does that affect the maintenance burden?
These questions have nothing to do with writing quality. They have everything to do with system architecture and team process. A team that asks them consistently will end up with documentation that stays useful longer, even if the prose is mediocre. A team that ignores them will produce beautifully written documentation that rots within a quarter.
Signs Your Documentation Is a Maintenance Problem
You can spot the issue without reading a single sentence. Look for these patterns:
- Documentation updates happen in bursts, usually after someone complains.
- The same field is described differently in three places.
- New team members trust the documentation less than they trust asking a colleague.
- The documentation contains version numbers or dates that are obviously stale.
- No one can name the owner of a given document.
If two or more of these ring true, your problem isn’t that your writers need training. Your problem is that your documentation has no maintenance model.
What a Maintenance-Minded Approach Looks Like
A maintenance-minded approach to data documentation starts with a simple principle: the documentation should be as close to the truth as possible, for as long as possible, with as little human intervention as possible. That principle leads to concrete practices:
- Generate structural metadata. Column names, types, nullability, and relationships should be pulled from the database catalog, the schema registry, or the transformation tool’s metadata layer. Never type them by hand.
- Embed descriptions in code. Use comment fields in schema definition files,
COMMENTstatements in SQL, or docstrings in data pipeline code. Publish these to a searchable catalogue automatically. - Version documentation alongside code. When a schema change is committed, the matching description change should be part of the same pull request. Reviewers should treat documentation drift as a review blocker.
- Assign clear ownership. Every data asset should have a documented owner. Ownership should be visible in the data catalogue. Owners should be notified when their assets change.
- Measure staleness. If you can’t measure staleness automatically, you can at least timestamp every documentation page and surface the oldest ones. A page that hasn’t been touched in eighteen months is probably wrong.
None of these practices require better writing. They require better engineering discipline. The irony is that teams who adopt them often find their writing improves as a side effect, because writers are no longer wasting energy keeping structural facts accurate and can focus on the explanations that actually need human thought.
The Cost of Treating It as a Writing Problem
Organisations that frame documentation as a writing problem spend money on the wrong things. They hire technical writers and set them loose on a sprawling, fast-changing data estate with no integration points. The writers do their best, but they’re playing a losing game. They can’t keep up with the rate of change because they aren’t plugged into the change process. They produce high-quality snapshots that are obsolete by the time they’re published.
The cost isn’t just wasted salary. The real cost is lost trust. When data consumers learn the documentation is unreliable, they stop consulting it. They build their own tribal knowledge. They ping the data engineers directly. They make decisions based on assumptions because finding the truth is too slow. The documentation becomes a graveyard of good intentions, and the organisation pays the tax in slower decisions and duplicated investigative work.
Reframing the problem as a maintenance issue shifts the investment toward integration, automation, and process design. It treats documentation as a living part of the system, not as a decorative layer slapped on after the fact. That shift is uncomfortable because it demands more from engineers and less from dedicated writers. But it’s the only approach that scales with the velocity of modern data teams.
FAQ
Why does documentation decay even when it’s well-written?
Well-written documentation decays because the systems it describes change independently of the text. A clear description of a field is still wrong if the field’s definition, source, or meaning has changed. The quality of the writing doesn’t protect against drift; only a maintenance process does.
Should we stop using wikis for data documentation?
Wikis aren’t inherently bad, but they create distance between the documentation and the thing it describes. If your schema definitions live in a repository and your descriptions live in a wiki, you have a manual coupling problem. Consider moving descriptions into the repository and using the wiki only for high-level, slow-changing context like data governance policies or onboarding guides.
How do you convince engineers to maintain documentation?
Don’t try to convince them through appeals to craftsmanship. Reduce the friction instead. Embed documentation in the files they already work with. Make stale documentation visible and annoying. Include documentation updates in the definition of done for schema changes. Engineers maintain things that are part of their workflow. Make documentation part of the workflow.
What is the role of a technical writer in a maintenance-minded approach?
The role shifts from producing static documents to designing information architecture. A technical writer can define templates, establish style conventions for embedded descriptions, and audit documentation for consistency and clarity. They become stewards of the documentation system rather than authors of individual pages. Their value lies in making the system produce clearer output, not in writing all the output themselves.