In a deployment model where remote systems communicate with a corporate
server network, outages may occur.
Sterling Selling and
Fulfillment Foundation provides
a data synchronization feature that allows you to synchronize configuration
data and master data from one environment to another. Data synchronization
involves these major tasks:
- Export data from the source database
- Import data into the target database
This section provides data obtained from the following data synchronization
operations:
- FULL Export
- DELTA Export
- Import data from the FULL Export
- Import data from the DELTA Export
This data is provided as an example that you can use as a general
guideline when planning your own data synchronization export and import
operations. Actual results may vary. The section also provides recommendations
about performing an export and an import.
Setup data for FULL synchronization
The
following setup data was used for the FULL synchronization.
- 3000 stores
- 1.4 million items in the catalog
- 70,000 users
- 160,000 pricing rules
- 40,000 pricelist lines for one store
Setup data for DELTA synchronization
The
following setup data was used for the DELTA synchronization.
- 150 stores (5% of the 3000 stores)
- 70,000 items (5% of the 1.4 million catalog)
- 3500 users (5% of the 70,000 users)
- 1600 pricing rules (10% of the 160,000 users)
- 4000 pricelist lines for one store (10% of the 40.000 pricelist
lines)
FULL export: time estimates and hard drive space
Table 1 provides
the time estimates and hard drive space for a FULL export, based on
the data in
Setup data for FULL synchronization.
Table 1. FULL exportProfile |
Time |
Hard Disk Size (uncompressed/compressed) |
Configuration |
30 min. |
90MB/10MB |
Catalog |
1 hour 30 min. |
16GB/500MB |
User |
10 min. |
125MB/10MB |
Pricing rule |
10 min. |
520MB/25MB |
Pricelist line (one store) |
10 min. |
25MB/1MB |
Import data from the FULL export: time estimates
Table 2 provides the time
estimates for the import of data from the FULL export described in
Table 1.
Table 2. Import data from the FULL exportProfile |
Time |
Configuration |
40 min. |
Catalog |
1 hour |
User |
10 min. |
Pricing rule |
30 min. |
Pricelist line (one store) |
10 min. |
DELTA export: time estimates and hard drive space
Table 3 provides the time
estimates and hard drive space for the DELTA export, based on the
data in
Setup data for DELTA synchronization.
Table 3. DELTA exportProfile |
Time |
Hard Disk Size (uncompressed/compressed) |
Configuration |
5 min. |
1MB/30KB |
Catalog |
10 min. |
65MB/2MB |
User |
5 min. |
1MB/30KB |
Pricing rule |
5 min. |
2MB/75/KB |
Pricelist line (one store) |
5 min. |
2 MB/100KB |
Import from the DELTA export: time estimates
Table 4 provides the time
estimates for the import of data from the DELTA export described in
Table 3.
Table 4. Import data from the DELTA exportProfile |
Time |
Configuration |
5 min. |
Catalog |
10 min. |
User |
5 min. |
Pricing rule |
5 min. |
Pricelist line (one store) |
5 min. |
Recommendations for full export
This section
provides recommendation for running a FULL export.
- Depending on the size of the data to export, the FULL export may
require at least a 4GB JVM.
- For a very large data set, consider increasing the number of messages
and decreasing the locking entities per file on the agent criteria.
However, do not set the locking entities per file to 1. This configuration
ensures that less data is stored in each XML file and alleviates any
database locks that might occur.
- Depending on your data set, the FULL export might require a large
amount of disk space on the machine where the agent is running. After
the export is complete, the XML files are compressed to about 5-to-10
percent of their original size.
- During the first part of the FULL export, the database can reach
a CPU utilization of 50 percent or higher, and depending on your data,
up to a 30 percent disk IO utilization.
- Full exports should be run on quiet systems.
Recommendations for full import
This section
provides recommendations for running a FULL import.
- Depending on your data set, imports might require a large amount
of disk space. The import first uncompresses all exported files, reclaiming
the original disk space.
- To improve database performance, disable entity change tracking
on the target system.
- Depending on the size of your data and the size of your generated
XML files, you might need to increase the space of your database transaction/undo
log because commits are executed at the end of every XML file.