Event Versioning

How to evolve event schemas in event-driven systems — backward/forward compatibility, upcasting, and schema registry.

The event versioning challenge

In an event-driven architecture, events are the contract between producers and consumers. Unlike REST APIs, where a consumer makes a request and receives an immediate response, events are published and can be consumed by multiple services — even services that didn’t exist when the event was defined.

This makes event versioning more complex than API versioning: you can’t simply “deprecate” an event, because there may be consumers reprocessing historical events from the log.

Schema evolution

Compatible changes (safe changes)

These changes don’t break existing consumers:

• Adding optional fields: Consumers unaware of the field simply ignore it
• Adding new event types: Consumers that don’t recognize them discard them
• Widening value ranges: For example, accepting more values in an enum (if consumers handle unknown values gracefully)

Incompatible changes (breaking changes)

These changes require a migration strategy:

• Removing fields: Consumers that depend on the field will fail
• Renaming fields: Equivalent to removing one field and adding another
• Changing data types: A consumer expecting a string that receives a number will fail
• Changing semantics: The field still exists but now means something different

Backward and forward compatibility

Backward compatibility (new readers, old writers)

A schema is backward compatible if consumers using the new schema can read events produced with the old schema. This is achieved when:

• New fields have default values
• No fields that new consumers expect are removed

Forward compatibility (old readers, new writers)

A schema is forward compatible if consumers using the old schema can read events produced with the new schema. This is achieved when:

• Consumers ignore unknown fields
• The types of existing fields aren’t changed

Full compatibility

A schema is fully compatible when it’s both backward and forward compatible. This is the safest level and the recommended approach for most cases.

Event upcasting

Upcasting is a technique for transforming events from an old version into a new version at read time. Instead of migrating all stored events, a transformation is applied when they’re read:

Stored v1 event → v1→v2 upcaster → Consumer receives v2 format

How it works

1. Events are stored in their original format (immutable)
2. On read, a chain of upcasters is applied based on the event’s version
3. The consumer always receives the most recent format

Advantages of upcasting

• No massive data migration required
• Original events are preserved (audit trail stays intact)
• New transformations can be added incrementally

Considerations

• The upcaster chain can grow over time (v1→v2→v3→v4)
• Each upcaster must be deterministic and free of side effects
• It’s worth testing the full chain of transformations

Schema Registry

A Schema Registry is a centralized service that stores and manages the schemas for all events in the system. Tools like Confluent Schema Registry or AWS Glue Schema Registry offer:

Key features

• Centralized storage: All schemas in one place
• Compatibility validation: Rejects new schemas that break the configured compatibility rules
• Controlled evolution: Allows defining compatibility rules per topic/event type
• Serialization/deserialization: Producers and consumers use the registry to serialize and deserialize events

Configurable compatibility levels

Typical flow

1. The producer registers a new schema with the registry
2. The registry validates compatibility against the previous version
3. If compatible, it assigns an ID to the schema
4. The producer serializes the event with the schema ID
5. The consumer uses the schema ID to deserialize

Best practices

• Always include a version field in your events (schemaVersion, eventVersion)
• Use formats that support evolution — Avro, Protobuf, or JSON Schema with clear rules
• Configure FULL compatibility as the default in your schema registry
• Design tolerant consumers — ones that ignore unknown fields and handle missing fields with defaults
• Test compatibility before deploying schema changes — ideally in the CI/CD pipeline
• Document each schema version with its changelog and introduction date