In a Three-Month Parallel Run, Some Flows Are Tested Only Three Times

If you have worked on a large legacy modernization project, some of this will sound familiar.

Many financial institutions still run systems that are twenty or thirty years old. Some are written in C or COBOL, and they still work every day.

The problem is not that they suddenly stopped working.

The problem is that the people who understand them are retiring, vendor support is disappearing, and adding new requirements gets harder every year.

So companies start large modernization projects: rebuild the old system on a modern stack while preserving its existing behavior.

That is the kind of project I work on.

I am part of a team at a Korean financial institution replacing more than twenty years of C programs with Java and modern frameworks running in a containerized environment.

The details vary from company to company, but a few characteristics are common in projects like this.

First, the goal is often behavioral equivalence.

The goal is not primarily to add new user-facing value. We are rebuilding something that already works.

Success comes down largely to one question:

Can we prove that the new system behaves like the old one?

That makes modernization very different from greenfield development.

Second, the go-live date is often decided before every legacy behavior has been fully analyzed.

At least in the projects I have worked on, the deadline tends to be difficult to move.

You can spend months discovering that the system is more complicated than expected while the date set before that analysis stays exactly where it was.

Third, the documentation is uneven.

It is not that we have no documentation.

We have interface specifications: file layouts, field definitions, message formats, external protocol documents. Those have to exist because other institutions depend on them.

What is often missing is a reliable specification of the business logic.

Why is this transaction skipped under this condition?

Why is this fee calculated differently when this code appears?

Why does the program take a different branch on a particular date?

Very often, the most reliable answer is still inside twenty-year-old source code.

That leaves us with a strange asymmetry:

We have enough specification to create the input, but not enough specification to confidently define the expected output.

The constraints around testing

Before getting into the testing problem, I need to explain a few constraints.

Without them, the obvious response is: “Why not just test earlier?”

1. Production and non-production environments are tightly separated

That sounds obvious, but the degree of separation matters.

In financial systems, data and software moving between environments are tightly controlled.

2. We cannot simply copy real customer data into test environments

In Korean financial institutions, the use of real customer information in development and test environments is tightly restricted.

Similar constraints exist in many regulated organizations, although the exact rules differ by jurisdiction.

In practice, production data is not something we can casually copy into a test database and use whenever we want.

3. Production deployment requires testing and approval

Financial institutions typically have strict change-management controls.

In our case, deployment into the production environment requires testing, approval, and review under internal control procedures.

In practice, that gives us a straightforward rule:

unfinished or insufficiently tested software should not be deployed into production.

4. The schedule is difficult to move

If the go-live date is fixed and the scope is also fixed—because the new system has to reproduce the existing one—schedule pressure eventually has to show up somewhere.

Very often, it shows up in how much testing can be done.

Why we rely on a parallel run

Our process looks roughly like this:

analysis and design → development and testing → integration and performance testing → parallel run (three months) → go-live

The parallel run is where we check whether the legacy and new systems produce equivalent results under real conditions.

The new system runs in an isolated environment within our production infrastructure.

The same real inputs that reach the legacy system are also fed into the new system, and we reconcile the results.

For batch processing, that may mean processing the same input file and comparing generated files and database records.

For online services, it may mean mirroring requests into the new system and comparing response messages and resulting database changes.

The implementation differs, but the idea is the same.

Why the parallel run is so valuable

One of the hardest parts of software testing is not creating an input.

It is answering this question:

What should the correct output be for this input?

In a modernization project, that can be unusually difficult.

We have the interface specification needed to construct an input.

But we often do not have a complete business specification telling us what the output should be.

To define the expected result manually, someone has to read the legacy implementation.

If that interpretation is wrong, the test can be wrong too.

And one of the most dangerous outcomes in a modernization project is a test that passes against an incorrect expected result, leaving everyone convinced that the behavior has been verified.

A parallel run avoids much of that problem.

Instead of redefining the expected result ourselves, we can use the actual behavior of the currently running legacy system as the comparison baseline.

That does not mean the legacy system is always correct.

It means that if the goal of the modernization is behavioral equivalence, current production behavior is the most practical reference we have.

We do not have to manually recreate thousands of expected values, and we avoid a large class of interpretation mistakes.

That is why parallel runs are so useful in this kind of migration.

So why does the parallel run happen at the end?

If it is so useful, why not use it throughout development?

Because the constraints above form a chain.

We cannot freely use real customer data in the test environment.
Validation against real production inputs therefore needs to happen in a controlled environment close to production.
Software has to pass testing and approval before it can be deployed there.
So the parallel run tends to begin only after the program is largely complete and stabilized.

None of those rules is unreasonable.

Protecting customer data makes sense.

Keeping untested software out of production makes sense.

Yet together they push the most valuable feedback to the latest stage of the project.

No single decision is obviously wrong. The problem emerges from how the controls combine.

Three problems follow

First, we do not control coverage

A parallel run uses whatever data actually arrives.

Frequently occurring cases are exercised repeatedly.

But if a particular condition never occurs during those three months, the corresponding branch may never run.

It did not pass validation.

It simply never had a chance to be validated.

That is especially uncomfortable in a modernization project because omissions tend to hide in exactly those places.

Main paths are read and tested repeatedly. A small exceptional branch buried in twenty-year-old code is much easier to miss.

If an interface defines ten possible code values but production traffic uses only three of them during the parallel-run period, the parallel run alone cannot tell us whether the other seven were implemented correctly.

Reality decides which cases the parallel run covers. We do not.

Second, execution frequency determines how much evidence we get

Consider what “three months of parallel validation” actually means in terms of executions.

Frequency	Approximate executions over three months
Daily process	~90
Monthly process	3
Monthly process that joins late	1–2
Quarterly process	1

In batch systems, this might include month-end processing or fee settlement.

In online systems, it might be an API that is called rarely or a transaction message that appears only under specific conditions.

At the end, all of them may be recorded as having completed the parallel run.

But one or two successful executions do not provide the same level of evidence as ninety.

Low frequency does not mean low importance either.

Some rare processes handle large amounts of money, regulatory reporting, settlement, or unusually dense business logic.

If validation depends on naturally occurring production traffic, we simply get fewer chances to compare those flows against the legacy system.

Third, defects show up when there is the least time left to fix them

The parallel run happens close to go-live.

Any mismatch found there appears when the schedule is already tight.

In one of our recent migrations, most discrepancies found during the parallel run were not major architectural failures.

They were missing pieces of business logic.

A branch existed in the legacy source but had never made it into the new implementation.

Missing logic is hard to detect.

If code exists but behaves incorrectly, a test can catch it.

But if the code is missing entirely, you first have to know that something was supposed to be there.

And that is exactly what an incomplete business specification may fail to tell you.

An interface document tells you which fields exist.

It does not necessarily tell you everything the system is supposed to do with them.

So these omissions can survive until the parallel run, which means they are found when there is the least time left to deal with them.

The deployment unit and the validation unit are not the same

In our environment, deployment and approval are organized around a program as a whole.

But testing already happens at a much smaller level.

One program in our system may contain ten, twenty, or even thirty independently executable units.

Integration testing and performance testing are already performed on those smaller units.

An online system may have a similar structure: one service containing multiple transaction types or message flows.

So the deployment unit is one program, but the validation state inside that program is not uniform.

And reality already reflects that.

Ideally, every unit would complete integration and performance testing before the parallel run begins.

Schedules do not always allow it.

Some units finish testing earlier, while others finish later and enter the parallel-run period with less time remaining.

So even within a single program, different units already end up with different amounts of parallel validation.

That is the part that made me reconsider the process.

The staggered validation is already happening. It just happens because some work finishes late.

What if we planned for the opposite?

What if the units that are ready went first?

Today, a unit may get less time in the parallel run because its testing finished late.

What if a unit that finishes early could start early instead?

As soon as one unit completes development and the required testing, put that unit into the parallel run rather than waiting for every other unit in the program.

The end state is not radically different.

Different units inside one program still enter the parallel run at different times.

The difference is whether that happens because something was late or because we planned the order that way.

A unit that finishes early might get four, five, or six months of validation instead of three.

A monthly process might run five or six times rather than three.

More importantly, defects would be spread across the schedule instead of concentrated near go-live.

We would have a better chance of finding them while there is still time to fix them.

The obvious objection is “unfinished software”

From a manager’s perspective, saying:

“Let’s deploy a program before the whole thing is finished.”

sounds risky.

That reaction is reasonable.

But deployment and execution are not the same thing.

Consider these two states:


State	Can it execute?	What happens?
Unit whose testing is incomplete	Yes	It can process real inputs without full validation
Unit that is not yet ready and is explicitly disabled	No	Its logic never enters the execution path

The second case is only safer if we can actually guarantee that the unit cannot execute.

That guarantee is the whole point.

If an unfinished unit is completely excluded from the execution path, it does nothing.

A unit that is enabled but has not completed testing can still process input.

So the useful comparison is not “finished software” versus “unfinished software.”

It is enabled but not fully validated versus not yet ready and guaranteed not to execute.

That is a more concrete risk comparison.

This only works under two conditions

1. Disabled really has to mean disabled

For batch processing, that might mean the job is not registered with the scheduler.

For an online system, it might mean the route is not exposed.

But this cannot depend on someone remembering not to call it.

The enabled or disabled state of each unit needs to be explicit, controlled, and auditable.

2. The parallel environment must be isolated from the legacy system

This is already a requirement for parallel validation, but it matters even more if deployment starts earlier.

Examples include:

blocking communication with external institutions at the network level
restricting access to legacy databases, ideally with read-only permissions where appropriate
separating generated output paths from production outputs
blocking or redirecting notifications, email, and messaging
avoiding consumption of shared sequences or other stateful resources used by the legacy system

That last one is easy to underestimate.

A shared sequence might not cause an immediate outage, but the effect may surface much later during settlement, reconciliation, or audit.

The requirement is straightforward:

If the new side fails, the legacy side must remain unaffected.

What starting earlier actually buys us

Because the parallel run follows real traffic, a longer validation period naturally gives frequently occurring flows more comparison opportunities.

If a process handles tens of thousands of transactions per day, incorrect behavior can affect a large number of transactions.

A longer parallel run gives us many more chances to compare that behavior against the legacy implementation.

That does not mean frequency equals importance.

A process that runs once a year may still be far more critical if it handles a large financial amount, regulatory reporting, or settlement.

Those low-frequency, high-impact cases need deliberate testing of their own.

But a longer parallel run gives us more than just additional executions.

We also pass through more calendar conditions:

beginning and end of month
quarter-end
holidays
business and non-business days

Rare values get more chances to appear.

And we get more chances to find a mismatch, fix it, deploy the correction, and observe it again under real conditions.

For me, that extra repair-and-recheck cycle is the biggest benefit.

Finding a defect three months before go-live is simply better than finding the same defect three days before go-live.

There is a cost, of course.

Starting the parallel run earlier also means preparing integration testing and environments earlier.

The people doing that work are often the same people who are supposed to be building the next unit.

More calendar time does not automatically create more engineering capacity.

Parallel validation still cannot cover everything

No matter how long the parallel run lasts, some cases may never occur.

A code value may be defined in the interface but appear only once every few years.

An exceptional branch may require a very specific combination of conditions.

Extending a three-month parallel run to five months does not help if the necessary input never appears.

That is where integration testing has a different job.


Validation method	Best suited for
Parallel run	Behavior that actually occurs in production under real operating conditions
Integration testing	Rare branches, boundary values, exceptional cases, and inputs production may not naturally generate

This distinction also changes how I think about test data.

There is limited value in spending most of our effort manually recreating a normal flow that the parallel run will exercise thousands of times with real inputs.

The cases worth constructing deliberately are the ones production will not give us easily:

every defined code value
date and amount boundaries
unusual combinations of conditions
places where the semantics of the legacy language and Java may differ

Those are also the places where missing business logic tends to hide.

This is why cutting integration testing uniformly when the schedule slips is dangerous.

Reducing cases that are already heavily exercised by the parallel run is one thing.

Removing the only tests for behavior that production traffic may never exercise is something else entirely.

Is this really a regulatory constraint?

When I first thought about this problem, I assumed the answer was simple:

financial regulation prevents us from doing it differently.

After looking more closely, I became less certain.

Some constraints are clearly regulatory.

In our environment, the use of real customer data outside production is tightly restricted, and production changes require formal testing and approval.

Those controls are not optional.

But another question is worth asking:

Does the entire program have to be treated as one indivisible unit for testing and approval?

At least in the rules I reviewed, I did not find a requirement saying that an entire program must be fully developed before any part of it can enter parallel validation.

That does not mean unit-by-unit deployment is automatically allowed.

The answer depends on the specific regulations, internal policies, audit requirements, and control framework of each institution.

This is not something an engineering team can decide on its own.

But there is an important distinction:

“Regulation does not allow it” and “our current internal process does not allow it” are not the same statement.

I used to treat them as if they were.

They need to be checked separately.

There are operational costs too.

Smaller deployment units may mean more approval cycles.

Reviewers may have to participate more often.

We may need a way to approve related units as a group or define repeatable approval criteria.

It may also look as though we are deploying the same program many times.

In practice, repeated deployment already happens whenever a discrepancy is found during the parallel run and the program has to be corrected and redeployed.

A staged approach would increase that frequency, but it would not introduce an entirely new kind of operation.

At that point, the hard part is not just the code. It is approval design, auditability, and ownership.

What this does not solve

Parallel validation can only show us what real data happens to exercise.

Cases that never occur still need deliberate testing.

And deliberate testing is expensive because people have to define the expected behavior.

This approach does not remove that cost.

We also have not implemented this process yet.

The project currently in progress is already following the schedule and controls that were defined for it.

This is something I am considering for the next one.

So I am not saying:

“Here is the process that solves the problem.”

It is closer to:

“This is something I want to try next.”

Closing

Parallel validation is one of the most useful tools we have in legacy modernization.

We do not have to recreate every expected output manually, and we can compare the new implementation against real behavior under real operating conditions.

But we usually wait until the entire program is ready and then concentrate that validation into the final three months.

During those same three months, one process may be validated ninety times while another is validated only three times.

Changing the official three-month validation period may be difficult.

But perhaps we can change when each unit enters it.

If units that have completed testing can start earlier, frequently occurring behavior gets observed for longer, more calendar conditions are covered, and we have more time to find a mismatch, fix it, and verify the fix again.

The rare cases that production does not generate should still be created deliberately in integration testing.

The odd part is that staggered entry already happens today. It just happens because some units are late.

Next time, I want to try the opposite:

let the units that are ready go first.