IM InfoSphere Identity Insight, Version 8.0

Pipeline configuration file parameters

The pipeline configuration file (usually pipeline.ini) contains the initial values for variables and configuration information that pipelines need to process incoming data. Settings in the pipeline configuration file override the global system settings (such as in system parameters) for all the pipelines that use the same pipeline configuration file.

This file should be named the same as the corresponding pipeline node.

You can manually add or change parameters or values in the pipeline configuration file. However, by doing so, it is possible to corrupt this file, create an invalid configuration, or cause pipeline processing disruption. Before you make any changes to the pipeline configuration file, make a copy of the file for safekeeping.

[pipeline] section parameters

Contains configuration data for the pipelines. Do not rename this section header; it must remain [pipeline] or the pipelines using this configuration file will shut down with an error.

CMEAdminTransport
Specifies the HTTP Uniform Resource Identifier (URI) of the transport to the Configuration Console, which is the component that contains the application monitor. You can use the application monitor to monitor the pipeline status and statistics and to route results from pipelines.
The default value is empty. This setting is usually commented out.
InputTransport
Specifies the URI of the transport where the pipeline receives incoming data. If the specified transport does not exist, the system either does not start the pipeline or shuts down the pipeline.
You can specify multiple transports by using a blank space between URIs.
The default value is empty. This setting is usually commented out, because input transports are usually specified on the command line when starting pipelines.
Concurrency
Specifies the number of concurrent parallel pipeline processing threads spawned when a single pipeline process is started. This parameter can be any positive integer greater than or equal to 0. The higher the number, the more pipeline processing threads start with each pipeline start command, and the more incoming data records can be processed at the same time. (One record is processed by each parallel processing thread.)
This setting takes precedence over the DEFAULT_CONCURRENCY system parameter setting specified in the System Parameters tab in the Configuration Console. This setting is not in the default pipeline.ini file, it must be added if you wish to have the pipeline.ini concurrency setting override the DEFAULT_CONCURRENCY system parameter.
The default value is 1, which means that only 1 pipeline processing thread is spawned with each pipeline start command. However, you can override this default value by specifying the concurrency number in the transport parameter of the pipeline start command.
ErrorLimit
Specifies the number of errors that the system can encounter during processing in a 24 hour period before it shuts down the pipeline that is encountering the errors. This error limit includes database errors, pipeline error, queue errors, or UMF parsing errors. If this setting value is 0, the pipelines never shut down, regardless of the number of errors encountered.
UMF exception errors are not included in this error limit setting. UMF exception error limits are set using the LogOnAllUMFExceptions and StopOnAllUMFExceptions parameters
This setting interacts with the ErrorRestInterval parameter.
The default value is set to 10.
ErrorResetInterval
Specifies the number of minutes the system counts errors that apply to the ErrorLimit parameter before resetting the error count for the pipeline.
If the specified number of minutes elapses without the system exceeding the ErrorLimit count, the system resets the number of errors counted for that pipeline.
If the system exceeds the number of errors specified in the Error Limit parameter before the number of minutes specified in the ErrorResetInteral parameter, the system shuts down the affected pipeline.
The default value is set to 1440.
LogOnAllUMFExceptions
Indicates whether UMF exceptions are logged to the pipeline log file, *.msg where * is the name of the pipeline where the exception occurred. Valid values are Y or N:
  • If this parameter is set to Y, all incoming data that generates UMF exceptions are placed in a *.msg log file, and the exception is logged in the UMF_EXCEPT table.
  • If this parameter is set to N, the system does not log UMF exceptions to a *.msg log file. However, the exceptions are still logged to the UMF_EXCEPT table.
This setting interacts with the StopOnAllUMFExceptions setting.
The default value is set to Y.
StopOnAllUMFExceptions
Indicates whether or not the system stops processing incoming data and shuts down the pipeline when it encounters a UMF exception. Valid values are Y or N:
  • If this parameter is set to Y, the system automatically stops processing all incoming data when the first UMF error is encountered and shuts down the pipeline. This setting is typically only used during initial implementations to gather additional information about incoming UMF exceptions.
  • If this parameter is set to N, but the LogOnAllUMFExceptions parameter is set to Y, the system logs the UMF exception to the UMF log file and continues processing the incoming data. The data involved in the UMF exception is not processed, which means that you must review the UMF exception log to find the problem with the UMF, correct the data, and then reload the entire UMF record into a pipeline for processing.
  • If this parameter is set to N, and the LogOnAllUMFExceptions parameter is set to N, the pipeline completes a partial data load, loading only the data that is not included in the UMF exception. The data included in the UMF exception is not processed, which means that you must review the UMF exception log to find the problem with the UMF, correct the data, and then reload the corrected data into a pipeline for processing.
The default value is set to N, and the default value for the LogOnAllUMFExceptions parameter is set to Y. This means that by default, the system logs the UMF exception to the UMF log file and does not process the incoming data record with the UMF error.

[SQL] section parameters

Defines the configuration for the database connection between the pipelines and the entity database.

Connection
Specifies the URI (universal resource indicator) for pipelines to connect to the entity database. Each database type uses a specific different syntax, but the base syntax is indicated as follows:
type://user:password@database
To specify a DB2® database connection
db2://user:password@database
where
  • db2:// indicates the database type
  • user:password@database specifies the login (user name and password) to access the specified database
  • and /?schema=schemaname is the name of a custom DB2 schema. (This setting is optional and typically used only when you want to specify a non-standard or custom DB2 database schema.)
Note: The DB2 custom schema feature is not compatible with the reports generator in the Visualizer and the Configuration Console. If you specify a custom DB2 schema, the Visualizer and Configuration Console reports will not work.
To specify an Oracle database connection
oci://user:password@SID
where user password specifies the login (user name and password) to access the database
SID matches the SID parameter set for this Oracle database.
To specify a Microsoft SQL server database connection
mssql://user.password@DSN
where user passwordspecifies the login (user name and password) to access the database
and DSN matches the DSN parameter set for this Microsoft SQL server database.
The default value is empty.
LogTable
Specifies the table to use when the system logs UMF messages. Use this parameter if you have multiple pipelines sending data to the same entity database; each pipeline needs to write log information to separate tables.
If you specify a table other than the default value of UMF_LOG, you must create the new table in the database, and that table must contain the same fields as the UMF_LOG table.
DeadLockRetries
Specifies the number of retries the system attempts during the processing of an incoming UMF message, after the pipeline times out or exceeds the deadlock conditions. If this number is exceeded, the pipeline shuts down.
The default value is set to 3, but it is usually commented out using the number sign.
DebugLevel
Controls the level of detail of the messages sent to the SQL debug log, *.SqlDebug.log, where * is the name of the pipeline node set to debug mode. Valid values include:
  • If this parameter is set to 0, no log is created. Use only for debugging.
  • If this parameter is set to 1, the system logs performance statistics.
  • If this parameter is set to 2, the system logs all SQL messages.
  • If this parameter is set to 3, the system logs all performance statistics and logs all SQL messages.
The default value is set to 0, which means that by default, no messages are sent to the SQL debug log.

[OAC] section parameters

Defines the configuration parameters for address correction that is integrated into the pipeline processing.
Note: Each feature of the software is provided by independent software vendors and is licensed separately by these vendors.
AddrConnection
Specifies the URI for the address correction software. The value must use a specific syntax:
product://host:port
product
  • If you use Pitney Bowes Universal Addressing Module, set this to g1unc.
host
Specifies the name of the host machine that runs the address correction software, or the IP address of the host server for the address correction software.
port
Specifies the port numbers to use for the address correction software. You can use the default port number(s) for the address correction software your system uses, or if your system is configured to use other port numbers, you specify those port numbers here.
This list contains the default port numbers by address correction software:
  • For Pitney Bowes Universal Addressing Module, the default port number is 8080.
The default value for these parameters is empty.
OverrideState
Use this parameter only if you want the system to replace the incoming United States state value with the associated two-digit state code. Valid values are Y or N:
  • If this parameter is set to Y, the incoming United States state value is replaced with its associated two-digit state code.
  • If this parameter is set to N, the incoming United States state value is not replaced and left as is.
The default value is set to Y.

[MM] section parameters

Defines configuration parameters for entity resolution.

DOBConfThreshold
Specifies the threshold for the date of birth (DOB) confirmation or denial. The DOB scoring is a point scale from zero to 100, based on the date of birth resolution algorithms. This parameter sets the point level where differences in dates of birth become denials.
The higher the threshold number, the less the difference can be between the dates of birth to score high during the confirmation and denial stage of entity resolution.
The default value is set to 90.
CircaDOBAttribute
Specifies the ATTR_TYPE_ID value in the ATTR_TYPE table that indicates a circa date of birth.
The default value is set to 4. However, because circa dates of birth are used infrequently, this default setting is usually commented out using the number sign.
CircaRangeThreshold
Specifies the number of units that a date of birth (DOB) can differ from a circa DOB and still be considered by the system as matching values. This threshold is used with the CircaRangeType parameter.
The default value is set to 1. The default CircaRangeType parameter is set to Y. Together, these parameters indicate that the default number of units that a date of birth can differ from a circa date of birth is 1 year.
However, because circa dates of birth are used infrequently, this default setting is usually commented out using the number sign.
CircaRangeType
Indicates the type of threshold unit for the circa date of birth (DOB). This parameter is used with the CircaRangeThreshold threshold.
Valid values are M or Y:
  • If this parameter is set to M, the circa DOB threshold is in months.
  • If this parameter is set to Y, the circa DOB threshold is in years.
The default value is set to Y, which means that the system uses years as the circa date of birth threshold. The default CircaRangeThreshold value is set to 1. Together, these parameters indicate that the default number of units that a date of birth can differ from a circa date of birth is 1 year.
However, because circa dates of birth are used infrequently, this default setting is usually commented out using the number sign.
DateRangeThreshold
Specifies the number of units for the From and Through date thresholds. This parameter is used with the DateRangeType parameter.
  • If this parameter is set to -1, disregard all From and Through date processing.
  • If this parameter is set to 0, use From and Through dates as given.
  • If this parameter is set to a number between 1 and x, the number represents the maximum gap size for non-overlapping date ranges.
The default value is set to 0. So by default, the system processes From and Through dates as specified in the incoming UMF message. This setting is usually commented out using the number sign.
DateRangeType
Specifies the unit for the date range threshold. This parameter is used with DateRangeThreshold.
Valid values are D, M, or Y:
  • If this parameter is set to D, the date range threshold is in days.
  • If this parameter is set to M, the date range threshold is in months.
  • If this parameter is set to Y, the date range threshold is in years.
The default entry is set to M. So by default, the system processes incoming From and Through date ranges in months. This setting is usually commented out using the number sign.
LogDenials
Specifies whether to log denial information from entity resolution. This setting is either commented out or must be manually entered.
Valid values are Y or N:
  • If this parameter is set to Y, the system logs denials.
  • If this parameter is set to N, the system does not log denials. If this setting is present in the configuration file, the default value is N.


Feedback

Last updated: 2011