It is also important to understand the different sorts of dependencies than can exist between potential processing units. There are serial dependencies where dependent processing needs to be carried out in set order (before or after) the related processing. This serial dependency may form a single logical transaction of work - where both sets of processing should either be completed or not. There are "fan out" dependencies where during the processing of one element a set of other elements requiring processing may be identified. Such fan out's of processing either need to be handled in the context of the processing that identified them, which may cause issues with the overlaps between streams and/or with the volume of work in processing a single record (with the fan out) becoming excessive; or the additional processing required needs to noted to be processed at a later stage (this could be implemented using Multiple Batch Programs, see Multiple Batch Programs.
There are many approaches to this sub-division of the processing, but some of the more common are listed below:
- Indirect units
- An indirect unit is where the unit of processing is not made up of the artifacts being processed directly, but rather using a different (related) grouping. For example when generating payments, which processes Instruction Line Items (ILIs) the processing unit is a participant - this is necessary because of the business requirement to issue all payments (ILIs) due to a single participant in a single payment instrument, grouping by ILI would lead to overlaps as the roll-up of a participant payment pulled in other ILIs.
- Composite keys
- Composite keys can be used where there is no "natural" key in the data which uniquely identifies the unit of processing, for example where the combination of a participant and case needs to be used. However, because the database design of the application makes it unlikely that any composition wouldn't have a unique ID in it's own right (Case Participant Role in the example above), this is not directly supported by the batch streaming infrastructure, but some further information is available in Using Composite Keys to Identify Processing Units.
- Sub-division of processing space
- Sub-division of processing space is useful where rather than dividing the entire processing space into a single logical set of processing units, it is necessary to first divide the processing space into large sub-units and then produce a set of processing units within each of these. For example where there are a number of types of financial transactions to be processed, say payments, bills and account transfers, it may not be possible to divide all of the financial transactions as a group into suitable processing units. By first sub-dividing by transaction type, each sub-division can then be broken down into processing units. This is not directly supported by the batch streaming infrastructure, but can be achieved using Multiple Batch Programs, see Multiple Batch Programs.