Client Guide to KELL Data Migration Process
KELL has migrated over 1,000 organizations to Salesforce, and over the years we've developed and perfected our migration process (although we'd be quick to say that perfect is never done!). Migrating your CRM data is a big undertaking and we realize the process can be overwhelming, so we wrote up this guide to help you see the forest through the trees.
To start with, you’ll need to send your data to KELL. This needs to be a full backup, not just report exports. Ideally, it's a relational database and not flat files. Click here to learn the difference. After receiving your backup, you'll upload it to Egnyte to securely deliver it to KELL. If you're nervous about data security, click here to read about how Egnyte ensures security and privacy.
Next, the KELL data team generates a “data dictionary” from your data. This is a document that lists out every field in your current database, with some high level information about how populated those fields are.
Once the data dictionary has been generated, the KELL team member mapping your data goes through that document line by line. Each field must be either mapped to a new field in Salesforce, or disregarded.
This is a time-intensive process, as the data map is shaping the architecture of your new system, and determining how you’ll be able to use and report on your data in the future. You’ll be meeting with KELL at least once a week during this process, so that we can ask you clarifying questions and make sure we’re on the right track. In between calls, you’ll need to do research to answer KELL’s questions in preparation for your next call.
Some fields are “easy” and map directly to a single field in Salesforce. Others require complex logic, mapping to different fields or even objects in Salesforce depending on what the content is on an individual record. Your data mapper is doing their best to make choices that will support what you need from your system in the future, while also keeping your budget in mind. The more “complex” mapping choices that are made, the more time must be spent on the mapping, migration and transformation of your data. They will talk with you throughout the process about options and trade-offs, and help you understand the implication of different choices on your capabilities, budget, and timeline.
While the data map is the foundation of the new system architecture, it is not all-encompassing. Your new system will also likely have new fields for tracking data that you did not have in your previous system.
Cleaning and De-duplication
We want to ensure that your data is entered in Salesforce in better condition than we found it. "Garbage in, garbage out" is what we're trying to avoid here.
First, we perform some standard cleaning on records to try to ensure Salutation, First Name, Middle Name, Last Name, and Suffix are in the correct fields. This process helps us make better matches when we perform de-duplication. Here are some examples of cleaning:
- Salutation = Mr. & Mrs., First Name = Bob & Mary, Last Name = Smith will be split into two Contacts - one for Mr. Bob Smith and another for Mrs. Mary Smith.
- First Name = Robert "Bobby", Last Name = Smith would be migrated as First Name = Robert, Nickname = Bobby, and Last Name = Smith.
- First Name = Rebecca A., Last Name = Johnson would be migrated as First Name = Rebecca, Middle Name = A, Last Name = Johnson
Next, we perform de-duplication, matching on standard rules, which also take into account some common nicknames and variations on English First Names. Here are some examples of de-duplication:
- Bob M. Johnson at 123 Main Street, Boston, MA and Robert M. Johnson at 123 Main St., Boston, MA are separate records in your old database - we'll merge them when we migrate to Salesforce
- Elizabeth Smith with email [email protected] and a mailing address, and Liz Smith with email [email protected] with no mailing address are separate records in your old database - we'll merge them when we migrate to Salesforce
For most projects, we use KELL’s standard cleaning and deduplication rules, which will be provided to you to review. We’ve found these rules to be ideal for the majority of our clients. These standard rules utilize programmatic scripts we've developed and therefore do not typically involve a manual review of every record.
Occasionally clients choose to have us develop custom cleaning and deduplication rules. If you’ve chosen this route, you’ll also be working with the KELL data team to define how your data rules need to differ from our standard. You must remember that data rules are code, there is not a human reviewing each individual record. A good de-duplication rule is targeted enough to only capture the correct records, but identifies a common enough scenario that it’s worthwhile to write code to capture it. For instance, if you have three instances of a certain scenario, it would be much more time efficient for you to manually adjust those records than it would be to write code to update them in the migration process. Custom cleansing and de-duplication rules add to a project’s scope.
Finalizing the Data Map
Once the data has been mapped, your KELL data expert will have several calls with you to walk through the decisions made and explain the outcomes that you can expect. You will then be asked to review and approve the data map. Approving the data map means that you have looked at each mapping choice and agree that it accurately represents how your data needs to be organized in your new Salesforce system.
Data Validation Scenarios
It’s critical that we start this journey knowing the destination. To this end, after the data map has been completed, we will work with you to create “Data Validation Scenarios”. These scenarios are examples of data records in your current system, that your KELL data expert has identified and mapped out how they should appear in your new system. We will enter your identified scenarios into a spreadsheet to revisit once the data has been loaded.
We will also ask you to look at your data dictionary and identify specific records that you want to make sure get reviewed during validation. Some examples include:
- A board member or major individual donor with multiple gifts
- An account with multi-year gifts or pledges
- An event attendee
- A recurring donor
- A volunteer or program participant
- A large campaign with total pledged versus received
After the map has been approved and the validation scenarios are created, the next step is the initial load. We take a full backup/export of your current database(s), and load the data into Salesforce as defined in your data map, applying our standard or your custom cleaning and deduplication rules. The load itself takes anywhere from a 2-6 weeks. A more specific time frame will be identified prior to the load for your project.
Data Validation - Round 1
Once the initial load is complete, you will be asked to validate the data in Salesforce. This is where your data validation scenarios come in to play. Use these scenarios to review the records you’ve selected, and compare how the data was in the old system, to how it is now in Salesforce. Check to make sure that the data is accurate, and in the right fields as defined by the data map. You will have a spreadsheet to keep track of any issues you identify whether that’s a mistake we made or just something that you’d like to see changed. Data migrations are a complex process, and we budget for some mistakes, misunderstandings, and changes in our timelines and scope. Your KELL project manager will work with you to understand what changes can be accommodated within your project budget and timeline, and which ones may require additional scope or trade-offs in other areas.
When you feel confident that all issues have been reported and addressed, you will sign off on the initial load which gives approval for KELL to move forward with the main load with the approved logic.
For the initial load, there was no need for you to stop using your current database; we were just taking a copy and practicing getting your data in Salesforce for testing purposes. When you are ready to do your main data load, we’re actually marking the moment that you move from the old system to the new. Unless you opt to purchase a differential load, all data entry into your old system will need to stop once we pull the data from your old system. You will be locked out of both old and new systems for 2-4 weeks while the data is migrated. During this time, you will need to keep track of any data entry updates in spreadsheets to be entered once you are live in Salesforce.
Data Validation - Round 2
When the main data load is complete, you will again be asked to validate your data, using your data validation scenarios. Most of the time, this validation phase goes more smoothly than the initial load validation. Our hope is that you’ve already identified the ways that you wanted to change your data logic, and we’ve already corrected any mistakes or misunderstandings in the initial load. When you are confident that the data is correct, you will be asked to sign off on the main load.
Optional - Differential Load
If minimizing downtime is a priority for your organization, you may want to consider purchasing a differential load. With a differential load you can continue to enter data in your old database while we migrate your data into the new system. Once you have approved the main load, we will identify any records in your old database that were created or updated since the data backup for the main load, and move only those records over at that time. You will have a small amount of downtime, but significantly less than you would have without it, since this load is significantly smaller and takes less time to complete than the full loads.
After you've signed off on your main or differential load, you're ready to go live in Salesforce! At this point, your system could be configured sufficiently to support moving all new data entry into Salesforce. Your legacy data is in Salesforce and ready to be added to, edited, and reported on.