A field is deleted from a Protobuf message. The number is not reserved. Months later, a new field takes that number. The schema compiles. The registry’s compatibility check returns a verdict. Old bytes still sit in Kafka topics, S3 archives, and downstream caches. When those bytes are decoded under the new definition, the wire format has no way to tell you the number means something different now.
This is not a hypothetical about a specific incident. It is the documented failure shape of field-number reuse, and it is worth understanding precisely because the tooling that looks like it should catch it does not.
What the wire format actually carries
A Protobuf message on the wire is a series of key-value pairs. The key is the field number plus a wire type. The name and declared type of the field are not on the wire at all.
The Protobuf encoding guide states it directly: “The binary version of a message just uses the field’s number as the key – the name and declared type for each field can only be determined on the decoding end by referencing the message type’s definition (i.e. the .proto file).”
That design is what makes Protobuf lean and forward-compatible. It is also what makes reuse undetectable at the byte level. If field 14 was once an int32 and is now a string, the decoder does not know. It reads the tag, sees field 14, and applies whatever the current .proto says field 14 is.
The language guide is explicit about the consequence: “Reusing a field number makes decoding wire-format messages ambiguous. The Protobuf wire format is lean and doesn’t provide a way to detect fields encoded using one definition and decoded using another.”
There is no checksum, no type tag, no version marker. The number is the identity.
The rule the docs give, and the procedure they prescribe
The proto3 language guide is unambiguous: “Field numbers should never be reused. Never take a field number out of the reserved list for reuse with a new field definition.”
It also explains why the number cannot simply be changed: “This number cannot be changed once your message type is in use because it identifies the field in the message wire format.”
When a field is deleted, the documented procedure is to reserve the number. From the same guide: “However, you must reserve the deleted field number. If you do not reserve the field number, it is possible for a developer to reuse that number in the future.”
The mechanism is a compiler check. “The protoc compiler will generate error messages if any future developers try to use these reserved field numbers.”
Two syntax details matter in practice. Reserved ranges are inclusive: reserved 9 to 11; covers 9, 10, and 11. And field names and field numbers cannot be mixed in the same reserved statement — you need separate lines.
message OrderEvent {
reserved 14;
reserved "legacy_customer_id";
// ...
}The best-practices page reinforces the same point with a blunt rationale: “Never re-use a tag number. It messes up deserialization. Even if you think no one is using the field, don’t re-use a tag number. If the change was live ever, there could be serialized versions of your proto in a log somewhere. Or there could be old code in another server that will break.”
And on the deletion side: “When you delete a field that’s no longer used, reserve its tag number so that no one accidentally re-uses it in the future. Just reserved 2, 3; is enough.”
What the compatibility check actually compares
Confluent Schema Registry enforces compatibility by comparing new schema versions against previous versions using configurable compatibility types. The default is BACKWARD. The documented compatibility types are:
- Backward — consumers using the new schema can read data written with the old schema (add optional fields, remove fields).
- Forward — consumers using the old schema can read data written with the new schema (remove optional fields, add fields).
- Full — both backward and forward compatible (add/remove optional fields only).
- Transitive — compatibility checked against all previous versions, not just the last one.
This is a schema-diff against a policy. It answers the question: does this new schema version satisfy the configured compatibility rule relative to prior versions? It is not a scan of historical bytes for a number that was previously assigned to a different field. The registry documentation describes compatibility policy, not reuse detection.
The allowed changes also depend on how fields were originally defined. The Schema Registry docs note: “The allowed changes are dependent on how fields are originally defined. For example, as described under Backward compatibility, the ability to delete a field and keep the schema compatible requires that the field was either specified as optional or provided a default value in the original version.”
So a deletion can pass the check. A subsequent addition of a new field with the same number can also pass the check, because the check is comparing schema versions against a policy, not auditing the number’s history. The compatibility verdict and the wire-format safety question are different questions.
Confluent’s own guidance for Protobuf is to use BACKWARD_TRANSITIVE, “as adding new message types is not forward compatible.” That is a stronger policy, but it still does not turn the check into a reuse detector. It compares more versions; it does not know that a number was once something else.
The failure shape, illustrated
Consider a message where field 14 was an int32 representing a legacy customer ID. The field is removed. No reserved 14; line is added. The PR merges.
Months later, a new field is added: string region = 14;. The schema compiles. The compatibility check runs against the prior version and returns a verdict based on the configured policy. Nothing in that check knows field 14 used to be an int32.
Old records still carrying the original field 14 bytes now decode under the new definition. Depending on wire-type compatibility, the result is either a parse/merge error or a silently wrong value. The language guide lists the possible outcomes of encoding with one definition and decoding with another: “Developer time lost to debugging. A parse/merge error (best case scenario). Leaked PII/SPII. Data corruption.”
The parse error is the good outcome. The silent wrong value is the one that reaches a dashboard or a downstream join.
This is a hypothetical illustration of the documented failure shape, not a report of a specific observed incident.
The fix is a line in the deletion change
The cheapest control point is the deletion itself. If reserved <number>; is not part of the same commit that removes the field, the number is available for reuse the moment the change merges. No later tooling reliably reconstructs the intent.
Treat the reserved line as a required artifact of a field deletion, reviewed the same way as the field removal. A practical review heuristic:
- Any diff that removes a field definition should also add a
reservedentry for that number. - Any diff that adds a field should be checked against the reserved list before merge.
This is a process argument, not a claim about any specific tool’s behavior. The compiler enforces the reserved list once it exists. The gap is between deletion and reservation.
The cost of the fix is minutes at deletion time. The cost of skipping it is debugging time, parse/merge errors, leaked PII/SPII, or data corruption discovered later — potentially after the old bytes have been replayed, backfilled, or joined into a downstream table.
Why this keeps happening
The language guide names the common causes of field number reuse: “renumbering fields (sometimes done to achieve a more aesthetically pleasing number order for fields). Renumbering effectively deletes and re-adds all the fields involved in the renumbering, resulting in incompatible wire-format changes. deleting a field and not reserving the number to prevent future reuse.”
The first cause is cosmetic. The second is a missing line in a PR. Neither looks dangerous at the time.
The maintenance tax lands on the on-call engineer who has to trace a corrupted value back through a schema history that was never recorded in the .proto file. The reserved clause is the record. It is the only place the intent survives.
FAQ
Does the Confluent Schema Registry compatibility check detect a reused field number?
The retrieved Schema Registry documentation describes compatibility as a comparison of new schema versions against previous versions using configurable compatibility types. It does not describe a mechanism for detecting that a field number was previously assigned to a different field. The check evaluates allowed schema changes per format and policy; reuse detection is not part of that description.
What happens if I reuse a number that was never reserved?
The schema compiles. The wire format has no way to detect that the number means something different now. Old bytes decode under the new definition, which can produce a parse/merge error or a silently wrong value, depending on wire-type compatibility. The language guide lists developer time lost to debugging, parse/merge errors, leaked PII/SPII, and data corruption as possible outcomes.
Can I just change the field number instead of reserving it?
The language guide states that the number cannot be changed once the message type is in use because it identifies the field in the wire format. “Changing” a field number is equivalent to deleting that field and creating a new field with the same type but a new number. The documented procedure for deletion is to reserve the old number.
Do reserved names affect runtime JSON parsing?
The language guide states that reserved names affect only the protoc compiler behavior and not runtime behavior, with one exception: TextProto implementations may discard unknown fields with reserved names at parse time (only the C++ and Go implementations do so today). Runtime JSON parsing is not affected by reserved names.
What is the recommended compatibility mode for Protobuf?
The Confluent Schema Registry documentation notes that the best practice for Protobuf is to use BACKWARD_TRANSITIVE, because adding new message types is not forward compatible. The default compatibility type is BACKWARD.
Sources
- Protobuf Language Guide (proto3), “Assigning Field Numbers,” “Consequences of Reusing Field Numbers,” “Deleting Fields,” “Reserved Field Numbers,” “Reserved Field Names”: https://protobuf.dev/programming-guides/proto3/#reserved
- Protobuf Encoding Guide, “Message Structure”: https://protobuf.dev/programming-guides/encoding/#structure
- Protobuf Best Practices, “Don’t Re-use a Tag Number,” “Do Reserve Tag Numbers for Deleted Fields”: https://protobuf.dev/best-practices/dos-donts/
- Confluent Schema Registry, “Schema Evolution and Compatibility”: https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html