There are a set of configuration options passed into the chunker when it starts, that control various parameters of the operation of the streamed batch program.
- The ChunkMainParameters.chunkSize parameter controls the number of records in each chuck. Because this value typically has to be tuned for productive use, so that the transaction time for each chunk remains low, it is typically exposed as an EnvVar with a sensible default value.
- The ChunkMainParameters.dontRunStream parameter controls whether or not a stream is run in the chunker process while waiting for the other streams to complete. Because this value typically has to be tuned for productive use, it may be the case that the machine hosting the chunker is required for other processing while the streams run elsewhere, it is typically exposed as an EnvVar with a default value to run the stream (false).
- The ChunkMainParameters.startChunkKey parameter specifies the key value for the first chunk to be picked up by the streams. Where extra processing has been implemented this value is typically offset by one to allow for the ChunkResult used for the extra rocessing, see Extra Processing for further details.
- The ChunkMainParameters.unProcessedChunkReadWait parameter controls the wait time when re-scanning to detected unprocessed chunks once all the chunks have been handed out to streams. Because this value typically has to be tuned for productive use, so that the value is sensible relative to the transaction time for each chunk, it is typically exposed as an EnvVar with a sensible default value.