Running data migration

Data migration is a two-phase process in which you migrate both history data and transaction data, separately, in any order. In releases prior to 8.5, it was necessary to bring down your entire deployment when migrating your history data and transaction data. However in Release 8.5, the property, yfs.api.history.disable, was introduced, which allows you to migrate your history data when the application is running on the transaction data. The yfs.api.history.disable property can be used when performing an upgrade in nonsharded mode or sharded all-colony mode. In Release 9.1 the property, yfs.api.history.disable.colony.<colony_id>, was introduced, which allows you to migrate your history data for individual colonies when the application is running on the transaction data.

You can upgrade transaction data before history data in the following modes:

Additionally, you can use the yfs.api.history.disable property to upgrade history data before transaction data in sharded all-colony mode.

Between the time that you run your transaction data migration and your history data migration the following restrictions exist:

You should evaluate the impact of not being able to purge data or use the restoreOrder API, based on how your system is configured and customized.

The migration conducted for an "upgrade from version" directory is the most recent release directory. All migrations between Release 8.0 (or higher) to Release 9.4 are done from the 9.3 folder. In this case, you must see the <INSTALL_DIR>/Migration/9.3 folder.

For every command that is executed, log files and done files are generated in different folders depending on from which release you are migrating. For example, if you are migrating from Release 8.0, the files are generated in 8.0, 8.2, 8.5, 9.0, 9.1, 9.2.1, and 9.3 folders. If you are migrating from Release 8.2, no files are generated in the 8.0 folder.

Migrating transaction data before history data in nonsharded and sharded all-colony modes

If you are upgrading to Release 9.4 in nonsharded mode or sharded all-colony mode and want to use the yfs.api.history.disable property to migrate transaction data before history data, follow this process:

Allocating memory in buildmigration.properties and ycdmigration.properties

During the upgrade process, you can pass JVM-specific JAVA and ANT arguments in <INSTALL_DIR>/Migration/9.3/buildmigration.properties and <INSTALL_DIR>/COM/Migration/9.3/COM_Add-in/ycdmigration.properties to avoid out-of-memory errors.

If similar out-of-memory errors are seen in other directories when doing upgrade, similar changes are required in the buildmigration.properties or ycdmigration.properties files in the kit-directories of the hops. For example, you can pass JVM-specific JAVA and ANT arguments in <INSTALL_DIR>/Migration/9.1/buildmigration.properties and <INSTALL_DIR>/COM/Migration/9.1/COM_Add-in/ycdmigration.properties to avoid out-of-memory errors for the corresponding directory.

Data migration custom document type

Factory setup changes made to document type specific entities between Release 8.0 and Release 9.4 must also be applied for custom document types. To apply the changes, you must provide the mapping between the custom document types and the system-provided document types. To provide this mapping, save the customdoctype.properties.sample file located in the <INSTALL_DIR>/Migration/9.3/transaction directory as customdoctype.properties. To modify the customdoctype.properties file, follow the instructions provided in the sample file.

The following tables are updated during custom document type migration:

Data migration log files

The log files that are created by the ANT calls during migration may contain references to locations of other log files affected by the data migration. You must verify all the log files for any migration errors that occur when performing data migration.

Note: The log file is created in the directory from which the ANT command is executed.

For example, in a multihop upgrade when you run the following command, the upgrade-history-tables.log file is created in the <INSTALL_DIR>/Migration/9.3 folder:

${ANT_HOME}/bin/ant -Druntime=<INSTALL_DIR> 
-Druntime.old=<YFS_HOME_OLD> -f buildmigration.xml -logfile 
upgrade-history-tables.log -Dtarget=upgrade-history-tables 
migrate

This log file contains references to the log files created in the 9.2.1 folder. In such a scenario, the 9.2.1 migration folder must contain a log file named upgrade-history-tables-9.2.1-9.3.log, following the convention of the ANT target and hop version.