When a product changes, the way it represents its data may change too: a field is renamed, an entity is split, or a new business rule is introduced. The decision is not just technical. You need to choose whether to update existing records, adapt the data each time it is queried, or maintain both formats during a transition.
The useful question is when to migrate data and when to transform it on read. The answer depends on who consumes the information, what level of consistency the product requires, and how much the solution will cost to maintain. Neither approach is universally better: the goal is to make the trade-offs explicit and define how you will verify that the approach works.
What changes with each approach
A persistent migration changes the stored data so it conforms to the new model. It can run as a controlled operation or in batches, depending on volume, infrastructure, and availability requirements. Afterward, queries use the new representation, although they may need to recognize old records during the transition.
Read-time transformation preserves the original data and adapts it when returning results. An access layer, service, or query can convert the old format into the structure expected by the current consumer. This provides compatibility without an immediate large-scale modification, but it adds work to reads and logic that someone will need to maintain.
There is also a middle ground: temporarily maintain both formats and migrate gradually. For example, the application can read the new format and, if it is absent, convert the old one; new writes already store the updated format. This strategy requires clear rules to prevent each consumer from implementing its own variation.
Criteria for choosing
Before deciding, describe the change as an observable need: what data changes, who uses it, and what behavior must be preserved. Then assess these criteria:
- Consistency: If every consumer must see the same value and follow the same rules, a shared persistent representation can reduce discrepancies. If the conversion is specific to an interface or consumer, a read layer may be more appropriate.
- Read frequency and cost: Repeated transformation on a heavily used path can increase latency and resource consumption. If reads are infrequent or the dataset is small, that cost may be acceptable; measure it in the actual context.
- Volume and operations: A full migration may require time, error handling, and the capacity to process records without interrupting service. For large volumes, consider batches and progress tracking rather than assuming a single operation will be safe.
- Number of consumers: With multiple services, applications, reports, or integrations, changing storage can have knock-on effects. Identify who depends on the schema and whether everyone can be updated in a coordinated way.
- Reversibility: Read-time transformation preserves the source data, which can make it easier to return to the previous behavior. A destructive migration requires a backup or tested recovery mechanism, as well as a plan for subsequent changes.
- Business rules: If the conversion depends on decisions that may change, retaining the original data allows you to reevaluate it. If the new form represents a stable fact that everyone needs, persisting it may simplify the system.
Do not compare only the cost of running the migration with the cost of writing an adapter. Include recurring costs: more complex queries, observability, testing, support for older versions, and eventual removal of compatibility logic.
Signs that migration is a better fit
A persistent migration becomes more attractive when the new model will be the primary representation for a long time, many consumers need the same transformation, or adaptation work measurably affects latency. It may also be preferable when read logic has spread across multiple places and there is no longer a single reliable definition.
By contrast, read-time transformation may be reasonable if the change is temporary, the old format must be preserved, there are only a few controlled consumers, or the rules for the new model are still being validated. It can also help when the cost or risk of writing across production data outweighs the immediate benefit, provided performance remains acceptable and compatibility is limited.
There are signs that a temporary solution is turning into technical debt: conditions for every historical version, queries that mix formats without a shared rule, different errors across consumers, and tests that do not cover every variation. If these appear, simply adding another case is not enough. Decide which format will be canonical and set a date or verifiable condition for retiring the old formats.
How to make a gradual transition
A gradual migration reduces risk by separating the application deployment from processing every record. A practical plan can follow these steps:
- Inventory dependencies. Locate producers, consumers, scheduled jobs, reports, and integration contracts that use the affected model. Clarify who is responsible for each one.
- Define compatibility. Document which formats are accepted, which is the source of truth, and what should happen when a field is missing or an unexpected value appears. Avoid silent inferences that hide invalid data.
- Deploy compatible readers. Ensure the application can handle old and new data before it starts writing the new format. This prevents an older version from failing when it encounters updated records.
- Change writes. Make new operations produce the target format. If both representations are written during a period, define which takes precedence and how discrepancies will be detected.
- Process existing data. Run the migration in a controlled way, recording progress and errors. Use batches suited to the system and design the operation so it can resume without duplicating effects.
- Retire compatibility. Remove support for old reads only when metrics and checks show that no consumers or records still depend on it.
The exact sequence may vary. In systems with demanding availability requirements, read and write changes are usually deployed compatibly before the data is modified. The decision must account for the behavior of older versions that may remain active during a deployment.
Validate results and prepare to roll back
Before migrating everything, test the conversion with representative data and compare the results. Check record counts, required fields, boundary values, relationships between entities, and business rules. Matching counts alone do not prove that the transformation is correct: also compare values and behavior for relevant cases.
If possible, first run a comparison without changing the result users receive. Record differences between the old and new representations, classify discrepancies, and establish acceptance thresholds before expanding the process. Monitor errors, latency, and load during the operation, and make sure it is clear who can decide to pause it.
Rollback planning must account for writes made after the change. Restoring an earlier backup could remove valid updates; therefore, determine whether you can return to the old reader without reverting the data, or whether you need a strategy to reconcile changes. Test that path before relying on it.
Decision checklist
- Have you identified the canonical format and source of truth?
- How many consumers depend on the schema, and who controls each one?
- Have you measured the effect of read-time transformation on latency and resources?
- Is temporary logic centralized, tested, and subject to retirement conditions?
- Can the migration be paused and resumed, and can it detect failed records?
- Have you defined comparisons, acceptance thresholds, and owners responsible for validating results?
- Is there a safe rollback that accounts for new writes?
Choose read-time transformation when compatibility is limited, reversible, and operationally manageable. Choose migration when the new model needs to be shared, persistent, and simpler to consume, and you have a safe plan for updating the data. If uncertainty remains, use a temporary transition with metrics, an owner, and an exit condition: postponing a migration can be a valid decision; postponing it without a date or controls turns compatibility into a permanent feature.