Before you grab a marker and sketch another event-sourced microservices diagram on the whiteboard, pause. Look at the two basic states your data actually occupies. Pretending the distinction doesn’t matter isn’t agility—it’s a sign nobody read the field manual. This piece walks through the difference between data at rest and data in motion. No rocket ship diagrams. No grand transformation promises. Just the two states that will, sooner or later, break your system if you treat them as the same thing.
Data at Rest: The Static You Mistake for Safe
Data at rest is information parked on a non-volatile medium. Hard drives. SSDs. Tape. Optical discs if you’re feeling nostalgic. It’s not moving across a network, not being processed by a CPU, not sitting in RAM waiting for a garbage collector to sweep it away. It’s inert. The common assumption is that inert equals secure, which is a dangerous oversimplification.
The practical worry with data at rest is unauthorized access by someone who already has physical or logical proximity to the storage medium. A stolen laptop. A decommissioned server that didn’t get wiped properly. A misconfigured S3 bucket with public read permissions. The threats are overwhelmingly about confidentiality violations through direct file access.
Encryption is the standard mitigation, but the implementation details are where every project gets lazy. Disk-level encryption (like LUKS on Linux or BitLocker on Windows) protects against someone pulling the drive out of the chassis. It does nothing against a running system with a logged-in user. File-level encryption gets more granular but brings key management headaches that most teams underestimate until they’re locked out of their own production data at 03:00. Application-level encryption, where the app handles the keys and encrypts fields before writing them to the database, offers the tightest control—and the highest operational burden.
From an engineering standpoint, the boring truth is that data at rest is a storage format problem. You’re worried about the bits on the platter. The attack surface is whoever can read those bits outside your application’s normal access controls. If you’re not thinking about key rotation, secure key storage (not in a config file in the repo), and a verifiable destruction process for disposed media, you’re not doing data at rest security. You’re doing theater.
Data in Motion: The Transit You Assume Is Trusted
Data in motion is information actively traveling across a network boundary. This includes client-to-server communication, inter-service API calls inside your Kubernetes cluster, database replication streams, and that unencrypted log data you’re shipping to your SIEM because “it’s on an internal VLAN.” The moment data leaves the memory space of one process and enters a network socket, it’s in motion.
The threat model shifts entirely. Confidentiality is still a concern—sniffing unencrypted traffic on a compromised switch or a rogue access point. But you also get integrity attacks: a man-in-the-middle altering API responses, injecting malicious payloads, or replaying valid transactions. And availability: a denial-of-service attack that floods the transport layer or exploits a protocol handshake to exhaust connection pools.
The standard answer is Transport Layer Security (TLS). And the standard failure is treating TLS as a binary setting—”we enabled it, so we’re done.” TLS certificates expire. Certificate chains break when intermediate CAs rotate. Mutual TLS (mTLS) for service-to-service communication requires a PKI infrastructure that someone has to maintain. Cipher suite selection matters, because enabling a deprecated cipher to support a legacy client can downgrade the entire connection to something a motivated attacker can break in real time.
Then there’s the architectural blind spot: data in motion is not only about the network pipe. It’s about the serialization format on the wire. An API that sends sensitive fields in clear text inside a JSON body over HTTPS is still exposing that data to any logging middleware, load balancer, or reverse proxy that inspects the payload. Data in motion security means understanding every hop, every proxy, every termination point where the encrypted tunnel ends and clear text begins again.
Protocol-Specific Pitfalls
Different protocols introduce their own failure modes. HTTP/2 multiplexing can break certain WAF inspection models. WebSockets maintain long-lived connections that bypass typical session timeout controls. Database wire protocols (MySQL, PostgreSQL, MongoDB) often have their own TLS implementations with configuration syntax completely different from web servers. It’s not uncommon to find an application with HTTPS enforced for client traffic while the database connection string still uses an unencrypted port because “the DB is on the same subnet.” That’s not a separate problem. That’s data in motion left unprotected.
The Boundary Is a Lie
Here’s where the architectural trends I’m suspicious of cause real damage. The industry has spent a decade promoting event-driven architectures, message brokers, and streaming platforms. Kafka topics. RabbitMQ queues. Kinesis streams. These systems sit exactly on the boundary between rest and motion, and too many engineers don’t think about what state the data is actually in.
Consider a message sitting in a Kafka topic with a retention period of seven days. Is it at rest? It’s on disk. The broker persists it to a filesystem. But it’s also in the broker’s memory, being served to consumers, potentially replicated across multiple data centers. If you encrypt data at rest on the broker’s storage volumes but leave the topic unencrypted at the application layer, any consumer with access to the topic can read the clear text. If you encrypt at the application layer but the broker’s disk is unencrypted, a decommissioned broker node that still has data on it becomes a disclosure risk.
The correct, uncomfortable answer is that messaging systems require both protections simultaneously. Encrypt the data before it enters the broker. Encrypt the broker’s storage volumes. Enforce TLS on all connections to and from the broker. Authenticate producers and consumers with strong, regularly rotated credentials. Most architecture diagrams I’ve reviewed in the past two years skip at least two of these four requirements, usually with a note that says “to be addressed later.”
Regulatory Compliance: Where Both States Collide
If you’re operating under GDPR, HIPAA, PCI DSS, or any other regulatory framework that actually has enforcement teeth, the distinction between rest and motion stops being an academic exercise and starts being an audit finding. Most regulations specify separate control requirements for each state.
PCI DSS Requirement 3 covers protecting stored cardholder data—data at rest. Requirement 4 covers encrypting transmission of cardholder data across open, public networks—data in motion. The controls are tracked separately, tested separately, and failed separately. A QSA who finds your database encrypted at rest but your replication traffic unencrypted will write up a finding specifically against Requirement 4, not a general “you need more security” hand-wave.
GDPR Article 32 requires “appropriate technical and organisational measures” for both stored and transmitted personal data. The supervisory authorities in several EU member states have issued fines specifically for unencrypted data transmission—not just data at rest breaches. The assumption that “internal network traffic is fine” does not hold up when the regulator asks for a network diagram and sees no encryption between application servers and database servers processing special category data.
Engineering Practices That Actually Work
Stop treating encryption as a feature toggle. Treat it as a lifecycle. Every piece of data in your system should have a documented encryption policy that covers:
State classification: Is the data at rest, in motion, or in a queued/intermediate state? Each gets a different protection answer.
Key management: Who generates the keys? Where are they stored? How are they rotated? Who has access to the key material, and is that access logged?
Cryptographic inventory: You cannot manage what you cannot list. Maintain a current inventory of every data store, every message queue, every API endpoint, and every replication stream, with the encryption status of each. Update it when things change. If you don’t have this document, you don’t know if you’re compliant.
Testing and validation: Encryption configurations drift. Certificates expire at the worst possible times. Write automated tests that attempt unencrypted connections to services that should require TLS. Verify that data at rest is actually encrypted on disk, not just configured to be encrypted. A misconfigured mount point can silently store data outside the encrypted volume for months.
Incident response specificity: Your incident response plan should distinguish between a data-at-rest breach (stolen backup tape, compromised cloud storage bucket) and a data-in-motion breach (TLS man-in-the-middle, compromised proxy server). The containment steps are different. The notification triggers may be different. Lumping them together is a plan to do the wrong thing when you’re already behind.
When “Best Practices” Become Distractions
I’ve sat through enough architecture review meetings to recognize the pattern. Someone proposes a service mesh to “solve” data-in-motion security across the cluster. It’s a reasonable tool—Istio, Linkerd, Consul Connect all handle mTLS and certificate rotation at the sidecar level. But the proposal often skips over the fact that a service mesh only protects traffic that goes through the sidecar. Your database connection that bypasses the mesh because it’s using a legacy driver? Still in clear text. Your cron job that connects directly to the database without the sidecar injected? Unprotected.
The tool is not the solution. The solution is knowing, for every connection in your system, whether the data is encrypted in transit and whether the encryption is actually enforced. If you can’t answer that for a given connection, you have a gap. No orchestration platform fills that gap automatically.
FAQ
Is data in RAM considered data at rest or data in motion?
Neither cleanly. Data in volatile memory (RAM, CPU caches) is typically classified separately in threat models. It’s not persisted, so it doesn’t fit the “at rest” definition tied to non-volatile storage. It’s not crossing a network boundary, so it’s not “in motion.” The relevant threats are cold boot attacks, memory scraping malware, and core dumps that write RAM contents to disk. Protection mechanisms include memory encryption (like Intel TME), limiting sensitive data lifetime in memory, and disabling core dumps in production.
Do internal networks need encryption between services?
Yes, unless you maintain a zero-trust architecture where every service authenticates every connection and the network itself provides no implicit trust. The “hard outer shell, soft inner center” perimeter model has failed repeatedly. An attacker who gains a foothold on one internal host can sniff unencrypted internal traffic, pivot through service-to-service calls, and exfiltrate data without ever touching an edge device. Encrypt internal traffic. It’s not paranoia; it’s acknowledging that perimeter breaches happen.
How does data classification affect the rest/in-motion decision?
Data classification (public, internal, confidential, restricted) drives the minimum encryption requirements. Public data can travel unencrypted. Internal data should use TLS for motion and disk encryption at rest. Confidential data needs application-level encryption at rest and mTLS in motion, with authenticated encryption. Restricted data (regulated personal data, financial account numbers, health records) adds field-level encryption, hardware security modules for key storage, and potentially separate network segments. If you don’t classify your data first, you’ll either over-engineer protection for public assets or—more commonly—under-protect confidential ones.
Can a VPN solve data-in-motion problems?
Partially. A VPN encrypts traffic between two network endpoints, typically a client device and a corporate network. It protects data in motion across the public internet segment of the path. It does not protect data once it exits the VPN tunnel onto the internal network, and it does not encrypt data at rest. It’s a useful layer, not a complete solution. Relying solely on a VPN while running unencrypted internal services is the “hard shell” mistake mentioned above.