If these projects can be rewarding, where does the "torture" comment come from? It comes from the frequent horror stories of data migration projects going horribly wrong. I hear these stories when we're coming into a project to rescue it or from a prospective client that has gone through a horrible data migration. Each time I hear the story it is similar: the project gets behind, there are a large number of specification changes, the tools being used can't handle the specification changes, the data is incredibly dirty, and when the data is loaded into the new system, almost no records can actually get processed.
Once the data migration starts to go off the rails, the entire project team is usually pulled in to help get it back on track. Everyone from the project sponsor to the interns spends every waking moment doing whatever they can to get the data ready to go. This painful process can go on for weeks or months, until the project gets scrapped, significantly delayed, or reduced in scope. Once that failure happens, undoubtedly some people will lose their jobs.
That is the pain people associate with data migration projects, and that pain is real. However, it doesn't have to be that way. A proper plan, executed by a team that is well trained and equipped, can make the most complex data migration projects run smoothly. There will be some tense weeks with long hours, but overall these projects can be successful and the pain points limited. Even on large projects for multi-billion-dollar corporations, it is possible to achieve smooth go-lives where portions of the data migration team can take vacations during the cutover period. I know this, because I am part of these efforts.
In upcoming posts, I'll elaborate on where projects tend to go wrong and outline some guidelines that can help keep a project on track.