You can never tell what lies below the surface of old legacy systems.
During the lifetime of a system, a great many changes have been made to the code and data.
Every time a change was introduced the technician was forced to choose between “doing the right thing”
and “just getting the job done”. You know as well as I, that shortcuts happen all the time.
I am not even arguing that shortcuts are bad, they get the job done and keep the business running.
What I am trying to emphasize is that they have occurred and how they might come back to haunt you.

Users see the GUI of a system and believe that to be the system. Integration architects might see the system as a series of available services. We all have our different views of the system and all of them are valuable when migrating to a new system.

Most migrations use the database of the legacy system as the main source of migration data.
It is obvious that we will be biased by the data model and use it as the source of knowledge about what has happened in the past in the system. Often, however, you will find that data just makes no sense or give a wrong picture of reality.

Most deviations from reality have to do with naming conventions. Perhaps a field in the GUI has had its name changed, but the name of the underlying data field was not changed. This can lead to confusion when users and data analyst try to build a common understanding. Sometimes, however, the deviations are a bit more complicated and are harder to find.

I have been hit by deviations between data and reality lots of times doing migration projects. Two episodes stand out in my mind, however, the first a curiosity that underlines my point about using the database as the truth. The second was just mind-blowing and hit us like a train out of nowhere.

My curiosity episode occurred in a project migration life insurance policies. We did a profile analysis of the customers and could not match the numbers provided by the business owners. A bit of digging around uncovered the quirk in the system.
It turns out that the system did not have the functionality necessary to register that people had recovered from being sick. The insurance policy secured the policyholder a monthly amount of money in case they became sick. Some people fortunately recovered from their illness and they were then not entitled to further payments. Because the system did not have the functionality necessary for reverting people to a status of “well” the business adopted a practice of writing down the yearly entitled amount to a single dollar. The payment system divided the entitled amount by 12 to calculate a monthly fee and then rounded the amount. The effect was a payment of zero dollars. The customer was technically still registered as sick and receiving money, but for all practical purposes, they were treated as healthy customers.
To steal a quote from Westworld “If you can´t tell the difference, does it matter..?”.
It did not matter for the business, it was a quick and clean workaround to a problem that would have cost the company a fortune to fix.
It did matter to us when we wanted to migrate the correct state of the customer into the new system.
I am not blaming the people who made the fix, the workaround probably the correct solution. The effort required to implement this workaround was probably a fraction of the cost a real fix to the old system would have cost. Also, there was never any risk of migrating the wrong data, we did notice the odd data and acted on our discovery.

I remember the second episode like a bad dream. We were in the final days of a month-long consolidation project. The customer had outsourced his business to us and we were transferring his data to our platform. Again it was payment data that caused us trouble. The system had an integrated payment system and we had migrated the data necessary to continue these payments. The customer had already signed off on the validity of the migration, we were good to go. During the last of our project meetings with our customer, we wanted to obtain a list of last months payment to secure that every payment was accounted for in the transition period. The customer’s reply was “Don’t bother, we don’t use the data in the system to tell us what to pay to our customers, we have a list in excel that tells us what to pay out.” It turns out the customer had signed off on our migration because they knew the numbers were irrelevant. They did not realize that they did not have the ability to manually override payment amounts in the new setup, they just took that for granted. “Assumption is the mother of all fuckups..”

Human processes are invisible in the database of a system. You might never know you are working with incorrect data unless you broaden your view of the system to also include “how the system is being used”

Since that day I have asked a lot of seemingly stupid questions from my customers. I am going back to the basics every time I meet a new representative of the customer. It is easy to see customers as a unity that knows everything about everything in their organization. The truth is more like a tapestry where individuals each have the pieces of information relevant to what they are doing. You cannot ask just a single representative of the company “You do actually use this system to process payments..?” He or she might not know of the workarounds other parts of the organization have implemented.

Validate your input – expect the unexpected

Seek out the truth in whatever form it exists. Start with the obvious like looking at the screen in the old legacy system and compare with the data in the database. Expand the truth by looking at accounting reports, data from the BI department and other sources that provide a broad picture of the data in your system. Compare all these sources to find the flaws and missing pieces of the puzzle. Often the reason for these differences in data is flaws and anomalies, that you have to account for when moving data to the new system. When you have a good grasp on the data you can expand your search to include the actual use of the system.

Remember that the customer is interested in moving a business process. The fact that the business process is implemented in a system is incidental.