Migration Guide for solidDB HotStandby Users
This guide is targeted for users migrating from an earlier
HotStandby (CarrierGrade) solidDB product. This guide consists of
two parts:
- Migrating with HotStandby (migration from 3.1, 3.7, 4.0, 4.1, 4.2 and 4.5 is supported)
- New features of each version, starting with version 4.1
MIGRATING WITH HOTSTANDBY
HotStandby and Database Compatibility
HotStandby (HSB) compatibility of two versions means that the HSB connection
may be effectively established between two servers running
two different versions of the HSB software. For example versions
4.0 and 4.1 are not HSB-compatible while versions from 4.1 thru 6.0
are mutually HSB-compatible.
- Version 6.0 is HSB-compatible with versions 4.1, 4.2 and 4.5
- Version 6.0 is HSB-incompatible with versions 3.0, 3.1, 3.7, and 4.0.
If the versions are not HSB-compatible, a special command line option
(-x migratehsbg2) must be used for the hot (that is, rolling)
migration to be successful.
Database compatibility means that a database created with one
version can be opened with another one, without a need to convert the
database. Typically, major versions, like 4.2, 4.5 and 6.0 are mutually
database-incompatible. In such a case, in order to migrate to a new
version with an existing database, the conversion
command line option (-x convert/autoconvert) has to be used unless
the hsb migration option is already used.
Version 6.0is database-incompatible with any earlier version.
When moving from a version to a new one, you can choose
between a "cold" and "hot" migration.
Cold migration is the traditional way to migrate.
In cold migration, you shut down the whole system (both servers) and restart with
new software and configuration data.
Hot migration uses the mechanism of a controlled
switchover to maintain the database service during the system
upgrade.
MIGRATING FROM HSB-COMPATIBLE VERSIONS
Note: this information only applies to you if you
are migrating to solidDB 6.0
from Solid Database Engine 4.1, 4.2 or 4.5
You can choose between a "cold" and "hot" migration.
A. Cold Migration
In cold migration, you shut down the whole system (both servers) and
restart with new software and configuration data. If the new version is
database-incompatible with the previous one, use the command line
option '-x autoconvert' when starting a new version of the server with
an
old database.
B. Hot Migration
With HSB-compatible versions, hot migration involves
the following steps:
- Disconnect and shutdown the Secondary.
- Install the new software at the Secondary's site.
- Restart the Secondary with the new software
(use -x autoconvert if the two versions are database-incompatible).
- Perform a switchover.
- Disconnect and shutdown the old Primary (now Secondary).
- Install the new software at that site.
- Restart the old Primary as Secondary with the new software.
(use -x autoconvert if the two versions are database-incompatible)
- Reconnect the servers.
MIGRATING FROM HSB-INCOMPATIBLE VERSIONS
Starting from version 4.1, Solid Database Engine uses an updated
version of its HotStandby ("HSB") capability. The updated version uses a
replication protocol that is incompatible with the
older versions.
Note: this information only applies to you if you
are migrating to solidDB 6.0
from one of the following:
- FlowEngine 3.1
- FlowEngine 3.7
- FlowEngine or BoostEngine 4.0
and you are using the HotStandby feature (Solid CarrierGrade
Option).
You can choose between a "cold" and "hot" migration.
A. Cold Migration
The cold migration scenario includes:
- Disconnect the servers and shut them down.
- Install the new version of the software.
- Update the solid.ini files following the guidelines below.
- Start the Primary with the command line parameter
-x autoconvert
which instructs the server to convert existing database
to the new format.
- Set the Primary to the PRIMARY ALONE state.
- Perform 'hsb copy' or 'hsb netcopy' from Primary to Secondary
- Connect the servers.
As a result both servers will be using the same database
and running the new software.
B. Hot Migration
In hot migration, you upgrade one server at a time, and use switchover
to maintain a continuous database service.
In the description below, we assume that you have two servers: S1
(initial Primary) and S2 (initial Secondary). Note that the servers'
states will change during the hot migration.
- Disconnect the Primary from the Secondary by running, at the Primary (S1):
ADMIN COMMAND 'hsb set broken';
ADMIN COMMAND 'hsb status connect ping';
- Shut down server S2 (the original Secondary).
- Update the software on S2.
Make sure that you also update the configuration parameters in
the solid.ini file for server S2, following the guidelines below.
- Start the new version of server S2 with the command line parameter
-x migratehsbg2
This will cause the server to convert the database to a new format
and will enable the server to communicate with the Primary using the
old replication protocol. When the server starts, its state will
be SECONDARY ALONE.
- Connect from server S1 (the Primary).
ADMIN COMMAND 'hsb connect';
Note that you cannot connect from the Secondary
if it is running a newer version of the server.
- Normal catchup should be performed successfully. If it fails,
you must proceed as follows:
- shut down server S2 (the Secondary) and
- do a 'hsb copy' from S1 (the Primary)
- recover the copy with the old version of the server (S2)
- shut down S2 (the Secondary)
- go back to step 4.
(Note: in step B, you must use 'hsb copy' rather than 'hsb netcopy'
because 'hsb netcopy' does not work between different server versions.)
- Once the servers are connected, perform 'hsb switch secondary'
at the Primary (S1).
- Shut down server S1 (the old Primary).
- Update the software at server S1 (the old Primary).
Make sure that you also update the configuration parameters in
the solid.ini file for server S2, following the guidelines below.
- Restart server S1 using the command line parameter
-x backupserver
The server is started in the "offline" mode, that is, without a database.
- Netcopy from S2 (the new Primary) to S1 (the new Secondary).
- Connect the servers (from Primary or Secondary).
As a result of this scenario, the servers are running
the new version of the software in a reversed configuration
(the old Primary is now a Secondary and vice versa).
If you want, you can revert the roles to the original ones
by doing one more switchover.
NEW FEATURES IN HOTSTANDBY INTRODUCED BETWEEN 4.2 AND 4.5
Introduction of the CarrierGrade (CG) Connectivity
The purpose of the CarrierGrade (CG) Connectivity is to provide
a degree of Transparent Failover (TF) for the applications.
This is achieved by maintaining an application-level connection
handle over a failover or switchover. For more on TF, see the
Chapter "Using HotStandby with Applications" in the High Availability
User Guide.
NEW FEATURES IN HOTSTANDBY INTRODUCED BETWEEN 4.1 AND 4.2
Introduction of 1-safe Replication
In addition to 2-safe, synchronous replication from Primary to
Secondary, the asynchronous, 1-safe replication is now available and
it brings significant performance and latency improvements. With 1-
safe replication, the commit statement is acknowledged immediately
once the commit processing is completed at the Primary. The committed
transaction is transmitted to the secondary asynchronously, after the
control has been returned to the application. The delay involved in
transmitting the transaction may range from few milliseconds to a few
hundred milliseconds. The downside of 1-safe is that, in the case of
a failure, a few transactions may be lost in a failover.
The 1-safe replication may be set with a new parameter:
[HotStandby]
SafenessLevel=2safe ;values: 1safe, 2safe, auto. Default is 2safe
The delay of transferring the transactions to the Secondary may
be controlled with this parameter:
[HotStandby]
1SafeMaxDelay=5000 ;(default) Max delay (in ms) of 1-safe transaction
;transfer to Secondary
The safeness level may be set dynamically, per session or transaction,
with a new command:
SET [TRANSACTION] SAFENESS {1SAFE| 2SAFE| DEFAULT}
The value DEFAULT resets the safeness level to the session default that
is equal to the server-level value set with the configuration parameter.
It is also possible to control the safeness level with the
programmatic durability controls (like SET DURABILITY RELAXED) when
the SafenessLevel parameter is set to a special value "auto" (i.e.
"automatic").
Operation without Log Files
The creation of the log files may be disabled with the previously
known parameter:
[Logging]
LogEnabled=no
also in the case of the HSB operation. To sustain the HSB operation,
an in-memory log is used. This log is also used to cache transactions
during the PRIMARY ALONE operation (to be used later in catchup).
The size of the in-memory log is set with the new parameter:
[HotStandby]
MaxMemLogSize=8m ;default
When the server runs out of the in-memory log space, when in
the PRIMARY ALONE state, it issues an error message to
solmsg.out and solerr.out and stays in the PRIMARY ALONE state.
When HSB is used without a log file, in the case of a total system failure
(both servers fail), the database is recovered using the latest checkpoint
state.
Other New Configuration Parameters
NetcopyRpcTimeout=30000 | ;Timeout (in ms) of netcopy data transfer
| MaxLogSize=0 | ;max size of the disk-based HSB log
;By default: unlimited |
NEW FEATURES IN HOTSTANDBY INTRODUCED BETWEEN 4.0 AND 4.1
This section summarizes the new features in HSB and
is intended for existing customers who are familiar
with the existing features in Solid Availability (HotStandby)
Option version 4.0 (or 3.7) and would like to learn about the
new features in version 4.1. In addition to new features, the
existing users will see many-fold performance improvements
in most situations.
This section also summarizes any changes that are not
backwards compatible.
New Features
- HSB now supports BLOB data. Previous versions did not replicate blob data.
- HSB now supports in-memory tables. (Note that this applies
only to "persistent" in-memory tables; data in Temporary Tables
and Transient Tables is not replicated to the secondary.)
- Allow mixing DML and DDL statements in same transaction.
- Support (or improved support) for variations on
hotstandby scenarios:
- Spare Node scenario for Standalone (N + 1 Spare)
- Spare Node scenario for HotStandby (2N + 1 Spare)
- Multiple HSB scenario (2N + M Spares)
- Server rolling upgrade scenario ("hot migration")
- Three alternative safeness levels
2-safe received
2-safe visible and
2-safe durable
These are specified with a new solid.ini configuration
parameter (2SafeAckPolicy).
- Addition of the "Adaptive Durability" feature.
In version 4.0, the server allowed users to choose between
"strict" and "relaxed" durability. In version 4.1, we
also allow users to choose adaptive durability, which
means that the server will use relaxed durability when
properly connected to a Secondary server, and strict
durability in other situations (e.g. PRIMARY ALONE state).
To achieve this, the configuration parameter DurabilityLevel
should be set to "2".
Note that in version 4.0, you could set the DurabilityLevel
configuration parameter (in the solid.ini file) to
"2", however, this was treated as "strict durability"
in version 4.0.
- A new failover method using the 'hsb set primary alone'
command. In this method, the role switch is performed
unconditionally, without checking the state of the other
server. By using this method, the failover time may be
reduced to 10-20% of time used when the command
'hsb switch primary' is utilized, and be as short
as few tens of milliseconds.
New Commands
- You can now explicitly instruct a Primary or Secondary
server to disconnect from the other server in the HSB
pair. The command to do this is:
ADMIN COMMAND 'hsb disconnect';
Note that this command can be executed on either the
Primary or the Secondary server; its usage is not
limited to the Primary server.
This command normally causes both servers to go into
an "Alone" mode; i.e. the Primary server switches
from PRIMARY ACTIVE to PRIMARY ALONE, while the Secondary
server switches from SECONDARY ACTIVE to SECONDARY ALONE.
- You can instruct a Secondary server to connect to
the Primary server. (Previously, you could issue the
"connect" command to the Primary, but not to the
Secondary.)
- ADMIN COMMAND 'hsb set secondary alone'
Sets the server state to SECONDARY ALONE. Available in
states STANDALONE, PRIMARY ALONE and PRIMARY UNCERTAIN.
- ADMIN COMMAND 'hsb set standalone'
Sets the server state to STANDALONE. Available in
states PRIMARY ALONE and SECONDARY ALONE.
(For the Primary, this command replaces the command
ADMIN COMMAND 'hsb set nohsblog'.
For the Secondary, there was no corresponding command
in earlier versions.)
- ADMIN COMMAND 'hsb state'
Returns new state names as character strings, e.g. 'PRIMARY ACTIVE'.
- ADMIN COMMAND 'hsb status catchup'
Returns the status of the catchup. Possible values are:
'ACTIVE' and 'NOT ACTIVE'.
- The command
ADMIN COMMAND 'hsb parameter '
is deprecated starting from this version. To change the values
of the HSB configuration parameters dynamically, the general
parameter manipulation command should be used:
ADMIN COMMAND 'parameter hotstandby.='
- ADMIN COMMAND 'parameter hotstandby[.]
This general-purpose command returns now the values of
all the HSB parameters
- ADMIN COMMAND 'describe parameter hotstandby[.]
This general-purpose command return now the descriptions of
all the HSB parameters.
- ADMIN COMMAND 'hsb logpos'
This command can be used by a Watchdog program (or other HA manager)
to determine which of two servers should be switched to Primary and
which should be switched to Secondary. (This is useful if both
servers are in SECONDARY ALONE state, for example.) This approach
detects which of the servers is "ahead" (i.e. which has accepted
more transactions) and thus should be made the primary before
establishing the HSB connection. Note that the server that was
the Primary before the servers lost contact with each other is
not necessarily the server that should become the Primary now.
(When relaxed durability is enabled, if the Primary fails, the Secondary
may have transactions that have not been flushed in the Primary.
Thus the Secondary may actually have more transactions and should
become the new Primary.)
To use this command, follow the instructions below:
- Both servers should be up and running and in SECONDARY ALONE state.
- Connect to both servers.
- Use ADMIN COMMAND 'hsb logpos' in each server.
Successful admin commands will return error code 0 and a
string that indicates the position of the last locally-executed
transaction written to the log. (The string should be
regarded as an opaque value.)
- Compare the string values. For example, in C, use the
strcmp() function. The server that returned the string that
was "greater" should be chosen to be the new Primary.
You can then set it to the Primary state by executing
ADMIN COMMAND 'hsb set primary alone';
If the values are equal, select the server
arbitrarily or by some preference rule.
NOTE: This procedure does not guarantee that the server
with the higher string value is a superset of the other server.
It is still possible that the two servers will each have accepted
transactions that the other did not -- e.g. both servers may have
been running in PRIMARY ALONE state. To detect the possibility
that neither server is a superset of the other, the servers
compare information when executing the "connect" command. If
neither server is a superset, then the Connect command will
fail and give an appropriate error message.
Modified Commands
- The command
ADMIN COMMAND 'hsb set primary alone'
is now also accepted in the SECONDARY ACTIVE state, resulting
in the immediate role switch. This state transition is supposed
to be used in a case when Primary has failed without a doubt. If,
however, Primary was active at the time the command was issued
at Secondary, it will move to the PRIMARY UNCERTAIN state.
It can be then brought to the SECONDARY ALONE state and
reconnected with the new Primary.
Other Useful Commands
- The command
ADMIN COMMAND 'shutdown force';
is equivalent to the sequence:
ADMIN COMMAND 'close';
ADMIN COMMAND 'throwout all';
ADMIN COMMAND 'shutdown';
- The command
ADMIN COMMAND 'errorexit ';
causes an immediate process exit with the given process exit code.
It can be used to simulate server process errors in failover
Scenarios.
New solid.ini Parameters
[HotStandby] (the default values are shown)
HSBEnabled=yes
Enables HSB-related ADMIN COMMAND statements and implicitly defines
the default initial state of the server (SECONDARY ALONE).
The parameter value has to be set to 'yes', for HSB operation.
2SafeAckPolicy=1
Specifies the timing of acknowledgment in the 2-safe protocol
Possible value:
1 = 2-safe received
2 = 2-safe visible
3 = 2-safe durable
Default: 1
AutoPrimaryAlone=no
Replaces the old parameter PrimaryAlone. If this parameter is set to 'yes',
the Primary moves automatically to the PRIMARY ALONE state when the
connection with Secondary is broken. If this parameter is set to 'no',
and if there are open transactions when the connection is broken, then
the Primary switches to the state PRIMARY UNCERTAIN.
CatchupSpeedRate=70
The value of the parameter tells, in percent, how much of the available
CPU resources are used to perform the catchup. The lower is the number
the slower is the catchup and the higher is the performance of Primary
during the catchup.
PingInterval = 1000
The value is equal to the interval (in milliseconds) between two
consecutive pings sent by a server to the other one. Default is
1000 millisec. (1 sec).
Deprecated solid.ini Parameters
[HotStandby]
PrimaryAlone -- replaced with AutoPrimaryAlone.
MaxLogSize -- unnecessary, as the implementation of the HSB log has changed.
Timeout -- not used any more.
CHANGES IN SETUP AND CONFIGURATION
Required Changes
The following changes should be made to the solid.ini
configuration file:
- To "turn on" HSB functionality, you must set the new HotStandby
parameter HSBEnabled to "yes". In other words, your
solid.ini should have the following lines:
[HotStandby]
HSBEnabled=yes
In previous versions of the product, you turned on HSB by
specifying a connect string in the "[HotStandby]" section
of the solid.ini file. In version 4.1, specifying a
connect string is neither sufficient nor required.
- Transaction logging must be enabled for HSB to work.
In earlier versions, it was possible (although uncommon)
to use HSB without enabling transaction logging.
Logging is turned on by default, so unless you have
explicitly disabled it, you won't have to change this
parameter. If you have explicitly disabled logging, remove
the parameter or set it:
[Logging]
LogEnabled=yes
Recommended Changes
The following changes are not required. HSB will still work
if you either do not set the values of these parameters or if
you leave the values as they were in earlier versions. However,
we recommend that you update your solid.ini file to set these
parameters to appropriate values for version 4.1 and later.
- If you are using the parameter named PrimaryAlone, please
rename it to "AutoPrimaryAlone". The purpose and valid
values are unchanged.
- You may want to specify a value for 2SafeAckPolicy.
This parameter specifies the timing of acknowledgment in
the 2-safe protocol. Possible values are:
1 = 2-safe received
2 = 2-safe visible
3 = 2-safe durable
Default: 1
A value of 1 gives highest performance.
A value of 3 is the safest.
For more information, see the appendix titled
"Configuration Parameters" in the High Availability Option Guide.
- The default value for
[HotStandby]
ConnectTimeout
has changed from 3000 milliseconds to "no timeout".
If you have not explicitly set this parameter, and if
you wish to keep the same behavior as before, then
please set this parameter to a value of 3000.
- The following parameters are no longer used and we
recommend that you remove them from your solid.ini
file:
MaxLogSize
Timeout
For more information about any of these solid.ini configuration
parameters, see the descriptions of these parameters
in one of the following:
High Availability Option Guide
Solid Database Engine Administrator Guide
CHANGES IN USAGE
HotStandby before v. 4.1 used to use a set of "roles" (Primary,
Primary Alone, Primary Broken, etc.). The server is now
described in terms of "states". In most cases, the new states
are similar to the old roles; however, there are some changes.
OLD ROLE NAME
|
NEW STATE NAME
|
PRIMARY | PRIMARY ACTIVE |
PRIMARY ALONE | PRIMARY ALONE |
PRIMARY BROKEN | PRIMARY UNCERTAIN |
PRIMARY CATCHUP | (now part of PRIMARY ALONE) |
PRIMARY NOHSBLOG | STANDALONE |
SECONDARY | SECONDARY ACTIVE |
SECONDARY BROKEN | SECONDARY ALONE * |
SECONDARY CATCHUP | (now part of SECONDARY ALONE) |
* SECONDARY ALONE, unlike SECONDARY BROKEN, accepts
read-only queries.
In most cases, you can handle the change by merely
changing the ADMIN command that you use and the
role (state) name that you expect to see. E.g.
if you have a watchdog that executes
ADMIN COMMAND 'hsb role';
and then executes particular instructions if the
returned role is PRIMARY, then you can update your
code by merely changing
ADMIN COMMAND 'hsb role';
to
ADMIN COMMAND 'hsb state';
and changing
"PRIMARY"
to
"PRIMARY ACTIVE".
For all states except SECONDARY ALONE, the server's
behavior has either not changed significantly, or else you
simply don't see the old role. If the server behaves
the same, then of course you don't need to change
your code (e.g. in your watchdog), other than to take
into account the new state name. If an old role no
longer exists, then any related code in your watchdog
simply doesn't get executed; you don't have to worry
about actually doing the wrong thing. You may remove
the no-longer-needed code when it is convenient for you.
The only state that is likely to require significant work
is the new state SECONDARY ALONE. If the Primary is
down and the Secondary is in SECONDARY ALONE state,
then you may want to switch the Secondary to
the PRIMARY ALONE state.
(Your watchdog probably already has code to switch a
server from SECONDARY BROKEN to PRIMARY ALONE; this
code can easily be adapted.) If the Primary is in
PRIMARY ALONE state and the Secondary is in SECONDARY
ALONE state, then you may want to connect the Primary
server to the Secondary server using the command
ADMIN COMMAND 'hsb connect';
Your watchdog almost certainly already has code to
execute this when the Primary is in PRIMARY ALONE role
and the Secondary is in SECONDARY BROKEN role; you can
easily adapt the code to execute the same commands when
the Primary is in PRIMARY ALONE state and the Secondary
is in SECONDARY ALONE state.
WARNINGS:
- As before, it is unsafe to have more than one server
at a time operating in PRIMARY ALONE or STANDALONE state.
If both servers are accepting new data but they don't
see the same new data, then the databases will become
out of sync, and there is no way to merge or reconcile them.
Changes to solid.ini Parameters
The following HotStandby parameters in the solid.ini
configuration file have changed. Note that all of these
parameters are found in the "[HotStandby]" section of
the solid.ini file.
AutoPrimaryAlone
This is a new parameter, but we mention it here because it
replaces the old parameter PrimaryAlone. If this parameter
is set to 'yes', the Primary moves automatically to the
PRIMARY ALONE state when a connection with Secondary is broken
or Secondary is disconnected. Otherwise, if the Primary
loses its connection to the Secondary, and if the Primary
still has open transactions, then the Primary switches to
the state PRIMARY UNCERTAIN.
The default value is "no".
If you are using the old parameter PrimaryAlone, then we recommend
that you remove that parameter and use this one instead.
PrimaryAlone
This parameter has been deprecated.
We recommend that you use the new parameter AutoPrimaryAlone
instead.
If you set both AutoPrimaryAlone and PrimaryAlone, and
you set them to different values, then AutoPrimaryAlone
takes precedence.
ConnectTimeout
This indicates the timeout that server should use for
connect requests between Primary and Secondary. The value
is given in milliseconds, for example, 30000 means 30
seconds. Zero means no timeout (or, more precisely, it means
that the server will wait for the system timeout if there
is one).
The default value is 0 (no timeout, or system timeout).
This parameter existed in the previous version of the software,
but the default was different; the default used to be 30000
(30 seconds), and now the default is no timeout.
Note: There is also a parameter named "ConnectTimeOut" in
the "[Srv]" section of the solid.ini file. That parameter
is not related and has not been changed.
Connect
"Connect" is not mandatory any more to start the server
as an HSB server. If the value is not given in the INI file, the connect
string should be assigned dynamically before the 'hsb connect' command
is issued.
PingTimeout
The meaning has changed so that it really means timeout on each ping sent
by the server. After the time specified (in milliseconds) has passed, the
server concludes that a connection is broken and changes the state
accordingly. Default is 4000 (4 seconds). With the default settings, the server
reacts to a connection break in 4-5 seconds.
MaxLogSize
This parameter is now ignored. To avoid confusion, we recommend
that you remove it from your solid.ini configuration file.
Timeout
This parameter is now ignored. To avoid confusion, we recommend
that you remove it from your solid.ini configuration file.
Changes to Commands
Deprecated Commands
ADMIN COMMAND 'hsb set primary nohsblog'
(aliased with ADMIN COMMAND 'hsb set standalone')
ADMIN COMMAND 'hsb role'
Returns old state ("role") names; the new corresponding command is
ADMIN COMMAND 'hsb state';
ADMIN COMMAND 'hsb parameter '
(replaced with the general command
ADMIN COMMAND 'parameter hotstandby.=')
Illegal Commands
The following commands return a syntax error.
ADMIN COMMAND 'hsb status logsize'
ADMIN COMMAND 'hsb status maxlogsize'
ADMIN COMMAND 'hsb status transcount'
Note that we did not eliminate all "hsb status" commands;
some 'hsb status' commands are still valid, for example:
ADMIN COMMAND 'hsb status connect';
Other Command Changes
In the following commands, the optional clause shown in
brackets [ ] has become non-operational (it is silently ignored):
ADMIN COMMAND 'hsb connect [force]'
ADMIN COMMAND 'hsb switch primary [noconnect]'
ADMIN COMMAND 'hsb status connect [ping]'
The command
ADMIN COMMAND 'hsb copy'
can only be run when the Primary server is in PRIMARY ALONE
role (or state). It can no longer be run when the Primary is in
PRIMARY NOHSBLOG role (or STANDALONE state).
Changes to Events
The event SYS_EVENT_HSBLOGSIZE is no longer supported.
Copyright (c) 2008 Solid Information Technology, Ltd.
All Rights Reserved.
|