Information about features of IBM® Integration Bus that you can configure, and aspects of the product’s use, that you should consider to help your organization with GDPR readiness.
This document is intended to help you in your preparations for GDPR readiness. It provides information about features of IBM Integration Bus that you can configure, and aspects of the product’s use, that you should consider to help your organization with GDPR readiness. This information is not an exhaustive list, due to the many ways that clients can choose and configure features, and the large variety of ways that the product can be used in itself and with third-party applications and systems.
Clients are responsible for ensuring their own compliance with various laws and regulations, including the European Union General Data Protection Regulation. Clients are solely responsible for obtaining advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulations that may affect the clients’ business and any actions the clients may need to take to comply with such laws and regulations.
The products, services, and other capabilities described herein are not suitable for all client situations and may have restricted availability. IBM does not provide legal, accounting, or auditing advice or represent or warrant that its services or products will ensure that clients are in compliance with any law or regulation.
General Data Protection Regulation (GDPR) has been adopted by the European Union ("EU") and applies from May 25, 2018.
Why is GDPR important?
GDPR establishes a stronger data protection regulatory framework for processing of personal data of individuals, impacts IBM and IBM's client contracts, policies and procedures when handling personal data. GDPR brings:
Read more about GDPR:
The following sections provide considerations for configuring IBM Integration Bus to help your organization with GDPR readiness.
IBM Integration Bus (IIB) is a general-purpose integration engine which enables users to route and transform data as it is passed between third-party applications. IIB supports a large range of protocols and data formats for the purpose of connecting to bespoke applications, and provides pre-built components that are capable of communicating with popular packaged applications. As such, IIB touches many forms of data, some of which could potentially be subject to GDPR. Most frequently, data passes through the IIB architecture in real time, with IIB making synchronous connections to online endpoints. However, IIB also interacts with persistent forms of data such as messaging systems (both traditional on-premises message systems such as IBM MQ and Kafka, and cloud messaging providers such as IBM Event Streams), databases (relational databases and NoSQL databases), data held on local or remote file systems, email systems, and other CRM and ERP systems.
There are several third-party products with which IIB might exchange data. Some of these are IBM-owned, but many others are provided by other technology
suppliers. The IBM Integration Bus system requirements
website provides links to Software Product Compatibility Reports (SPCR) that list the
associated software; for example,
the SPCR page for IIB v10 on Windows. For organizations considering a third-party product to support their GDPR
readiness, consult that product's documentation.
IIB users control the way in which IIB interacts with data passing through it, by the definition of message flows. A message flow is commonly constructed by a user acting in the role of "IIB developer", working with the IIB Toolkit. A message flow is composed from a set of discrete building blocks (known as message flow nodes) that are wired together in an ordered fashion by the IIB developer. Message flow nodes are configured graphically, and in some cases also allow the IIB developer to drop into "code" (Java™, ESQL, .NET).
As a general-purpose integration engine, there is no one definitive answer to this question because use cases vary from user to user. However, it is entirely possible, that customers of IIB use it to interact with data that relates to the following categories:
IIB clients can submit online comments/feedback/requests to contact IBM about IIB subjects in a variety of ways, primarily:
Typically, only the client name and email address are used, to enable personal replies for the
subject of the contact, and the use of personal data conforms to the IBM Online Privacy Statement.
IIB can be used to collect personal data. When assessing your use of IIB and your GDPR readiness, you should consider the types of personal data which in your circumstances are passing through IIB. You may wish to consider aspects such as:
How does data arrive into your IIB message flows (synchronously / asynchronously? Across which protocols? Is the data encrypted? Is the data signed?)?
How is data sent out from your IIB message flows (synchronously / asynchronously? Across which protocols? Is the data encrypted? Is the data signed?)?
How is data stored as it passes through your IIB message flows? (Do you use any of the product features highlighted in the Data Lifecycle section of this document? If so, are you aware of how those features could potentially expose aspects of the data passing through the product?)
How are credentials collected and stored where needed by IIB to access third-party applications ?
IIB typically needs to communicate with third-party applications and systems, such as DB2® and SAP, and many such systems require IIB to authenticate. Where needed, authentication data (userids, passwords, and API keys) is collected and stored by IIB for its use in such communication.
Wherever possible, you should avoid using personal credentials for IIB authentication, to limit the chances of your personal credentials being used by message flows which were not your original intention. In any case, consider the protection of the storage used for authentication data. (See Data Storage below.)
What test data do you want to collect during development?
IIB provides a Flow Exerciser feature, which is used during unit testing to
record on disk the logical tree structure that represents data structures that are normally held in
memory of the product as the data is flowing through IIB.
Read more:
When developing new message flows, the product provides discovery capabilities that allow IIB to connect to third-party systems so that it can retrieve metadata about interfaces to those systems, which then enable IIB to construct data in the correct formats ready for exchange with those systems. For example, IIB provides discovery capabilities for SAP (to discover BAPI and IDoc structures), REST APIs, and WSDL-based web services. It is possible that during this build-time discovery phase, sensitive data could be discovered and persisted as part of these operations.
When configuring particular message flow nodes, a developer using IBM Integration Bus might want to pass a property which they consider to be personal or sensitive information. The IIB product has catered for the most obvious examples of this kind of data by providing some nodes with properties to express a security identity. The security identity provides a method of abstracting a reference to a userid / password combination which is then defined to a runtime IIB system using the mqsisetdbparms command. This approach avoids a developer having to expose this kind of sensitive information in version control systems and in BAR files that might be considered unecessarily exposing security risks to a larger population of employees within the client's context. For other cases, IBM Integration Bus provides the ability to override message flow node properties on a message by message basis at runtime, using the LocalEnvironment tree structure. IIB also provides the option of using User Defined Properties which can be passed in to message flows and avoid personal data being carried on the build-time flow artifact. Similarly, user-defined configurable services might be of interest for the same reason.
When data travels through IIB message flows as part of normal operation, IIB does not mandatorily persist that data directly to stateful media. However, users of IIB are given entitlement to the IBM MQ messaging product, which can be used to define persistent queues that are designed to hold messages for exchange in an asynchronous manner between applications. So, although IIB does not directly persist data to stateful stores, by association IIB users may want to consider securing data at rest that is written to input or output queues.
The following items highlight areas where IIB does require Data Storage mechanisms, which users may wish to consider when assessing their GDPR readiness:
Aggregation and Collector nodes: IIB provides Aggregation and Collector nodes that are designed to deal with scenarios where multiple third-party systems might send data in to IIB to be combined into a new format. Aggregation typically involves a "fanout" flow and a "fanin" flow. These flows exchange metadata so that IIB knows how many messages have been fanned out to external systems, so that when response messages come back into IIB it knows when all replies have been received, or when to give up waiting for replies. In the interim before all replies have been received, IIB needs to store the messages that have returned. This storage of messages is so that in the unlikely event that IIB is stopped (either deliberately or due to hardware failure of some form), when IIB is restarted the messages can be recovered from the storage location. IIB uses an IBM MQ queue manager to store this data in queues.
Record & Replay: If you need an audit record of data passing through the flows deployed to an IIB integration node, then you can use IIB's Record & Replay feature to record messages to a database, and then view them and replay them if needed. You might want to record messages for audit and security purposes, or if you want to keep a history of the data for development and test purposes, or to help with problem determination. You can configure bespoke settings in a message flow to specify which data should be recorded (including which messages, which fields from within them, and using XPath expressions you can select messages which meet particular criteria). The data is stored in a database, which can be an Oracle, Microsoft SQL Server, or DB2 database. After you have recorded data, you can view it using the IIB web user interface, including seeing a list of recorded messages or the details of a specific message. You can also view recorded data by using the IBM Integration API or the IIB Representational State Transfer (REST) API. You can replay a message to an MQ queue by using the web interface or IBM Integration API. You can also replay the message to another message flow for testing or problem determination.
Read more:
Business Transaction Monitoring: You can use business transaction monitoring to monitor a message across multiple message flows, so you can track and report the lifecycle of a payload message through an end-to-end enterprise transaction. The IIB web user interface is used to create a business transaction definition to identify the message flows that form the business transaction, and define how they interact with each other. You can view the monitoring events that are defined for the message flows that make up the business transaction. Some of these events are significant to the business transaction. The business transaction definition defines how these events apply to the business transaction. Existing monitoring events (the same monitoring events that drive the Record & Replay feature) can be selected as start, end, and failure events in the business transaction to show how the transaction is progressing. You can be selective - not all monitoring events need to be flagged as business transaction events.
Data storage in databases: Many IIB users might choose to have the product persistently store data inside Relational Databases, or NoSQL databases. IIB provides JDBC and ODBC connectivity to the Relational Databases, and a Loopback Request node which assists with NoSQL databases.
Read more:
User and service trace: IIB provides user and service trace features, which record the internal IIB code paths through which the data flows. As part of these features, IIB users can decide to take "snapshots" of what the data looks like passing through IIB. IIB also provides real-time interfaces such as a graphical debugger view in the Integration toolkit, which presents this snapshot data to the IIB developer.
User trace and Service trace may include entries made by parsers and protocols including data from customer messages passing through IIB. This information is only discovered when trace is turned on, and the trace files are completely within the customers control.
To protect access to user trace and service trace, consider the following actions:
Read more:
Authentication data needed for IIB to connect to third-party applications and systems: IIB typically needs to communicate with third-party applications and systems, such as DB2 and SAP, and many such systems require IIB to authenticate. Where needed, authentication data (userids, passwords, and API keys) is collected and stored by IIB for its use in such communication.
An IIB administrator configures IIB with authentication data, which is then stored in the product configuration directory (for example, C:\ProgramData\IBM\MQSI on Windows).
When IIB is installed, the production configuration directory is set up with group-based access control such that IIB can read the configuration files and use the passwords to connect to these systems. IIB administrators are also members of this group so have read access to the files. The files are obfuscated using a private algorithm but they are not encrypted. For this reason, to fully protect access to passwords, you should consider the following actions:
Read more:
IIB-owned data can be accessed through the following defined set of product interfaces, some of which are designed for access through a remote connection, and others for access through a local connection.
These interfaces are designed to allow users to make administrative changes to your IIB installation. Administration can be secured, such that there are three logical, ordered stages involved when a request is made - Authentication, role mapping, and authorization.
If the administrative action was requested from a local connection, then the source of this connection is a running process on the same system. The user running this process had to have been authenticated by the operating system at some point. Therefore, for local connections, the user name of the owner that runs the process from which the connection was made is taken to be a "pre-authenticated" user. IIB does not perform any additional authentication steps in this case. For example, this could be the name of the user running the shell from which an IIB command has been issued.
If the administrative action was requested from a remote connection, then communications with IIB are made through an HTTPS interface (whether it originates from the IIB Web Admin UI, the Integration toolkit, or other applications using the IIB REST API). The username and password which are provided by the user are checked against settings held in IIB's local registry. It is also possible to configure IIB to pass the username and password to an LDAP registry for authentication.
In the Role mapping stage, the username that was provided in the authentication stage is mapped to a system id. This system id is then used when authorizing which administrative activities are allowed to be carried out by the authenticated user.
IIB v10 removed the mandatory product prerequisite of IBM MQ. In IIB releases before v10, authorizations were configured using MQ queues. In IIB v10, the option to define authorizations using MQ is maintained in addition to the provision of a new file-based alternative that was introduced.
IIB defines a list of administrative actions that a user can perform. To provide a user with
authorization for each action, specific read / write / execute permissions are required on
documented MQ SYSTEM queues (or files if you have opted to use the new file-based alternative).
These permissions must be set to allow the user to complete these administrative tasks when
administration security is enabled. For a list of the administrative actions and the permissions
that are needed, see Tasks and authorizations for administration security
Some users of IIB would like to create an audit record of administrative accesses to IIB. Examples of desirable audit logs might include configuration changes that contain information about the change in addition to who requested it.
The following sources of information are available out of the box in the context of this requirement:
Items (1) and (2) do not include the name of the user that requested the change, but (3) does. When considering these kind of solutions, IIB users might want to give consideration to the following points:
Users of IBM Integration Bus (IIB) can control the way in which IIB processes personal data, through the definition and configuration of the message flows that are deployed to the IIB runtime. Message flows begin processing when input data arrives into IIB through an input node, and they complete when data is sent out from an IIB output node or request node. A large range of protocols are supported, some of which include provision for the data to be encrypted when it is passed into and sent out from IIB. Encryption provides a method by which the data is converted from a readable form to an encoded version that can only be decoded by another program if it has access to a decryption key.
You can secure message flows by configuring nodes within the message flows to use SSL and
certificates. After you establish a public key infrastructure configuration for your whole
integration node or for some of its integration servers, you can use the configuration within your
message flows. For information about establishing a public key infrastructure, see Setting up a public key infrastructure. When you configure
message flow nodes that make or accept TCP connections, you can set SSL encryption and certificates
on those connections.
For more information about using SSL and certificates to secure your message flows, refer to the following topics in the IIB v10 product documentation:
Using the PKI security facilities that are provided by transport mechanisms is the first step towards securing data processing with IIB. However, without enabling further IIB security features, the behaviour of the IIB runtime is to process all messages that are delivered to it, using the integration node service identity as a proxy identity for all message instances. In these circumstances, any identity that is present in the incoming message is ignored.
The next step to securing IIB's data processing is to use an IIB security profile. You can use a security profile to configure an integration node for processing end-to-end an identity that is carried in a message through a message flow. By creating a security profile, you can configure security for a message flow to control access based on the identity that is associated with the message, and this provides a security mechanism that is independent of both the transport type and message format.
Only a subset of the connectors available in IIB use security profiles to control and vary the identity that is used when the connector interacts with an external system. For other connectors, a fixed identity can be specified, which is used to authorize access to the external system. For those connectors, the integration node has its own repository of identities, which can be updated with the mqsisetdbparms command. This command is covered elsewhere in this document.
Security profiles are used by the IIB security manager component which enables the integration node to perform the following activities:
Security profiles are created by the integration node administrator and accessed by the security manager at runtime. The following external security providers (also known as Policy Decision Points or PDPs) are supported:
You can invoke message flow security by configuring either a security enabled input node or a SecurityPEP node. You can use the SecurityPEP node to invoke the message flow security manager at any point in the message flow between an input node and an output (or request) node. For an overview of the sequence of events that occur when the message flow security manager is invoked by using either a security enabled input node or a SecurityPEP node, see the following topics in the IIB v10 product documentation:
The following input nodes support message flow security:
The WS-Security specification provides mechanisms for securing web services at the message level including identity, authentication, authorization, integrity, confidentiality, nonrepudiation, and basic message exchange. Both traditional web and message-level security share many of the same mechanisms for handling security, including digital certificates, encryption, and digital signatures. These details can be provided to IIB using Policy sets and bindings. IIB Policy sets and bindings define how to configure your WS-Security (and WS-RM) requirements for several IIB nodes including the SOAPInput, SOAPReply, SOAPRequest, SOAPAsyncRequest, and SOAPAsyncResponse nodes. A policy set is a container for the WS-Security and WS-RM policy types. A policy set binding is associated with a policy set and contains information that is specific to the environment and platform, such as information about keys. You can use policy sets and bindings to define the following items for both request and response SOAP messages:
Either the whole SOAP message body, or specific parts of the SOAP message header and body can be encrypted and signed.
IIB provides commands and user interface actions to delete data which has been provided to the product. This enables users of IIB with facilities to delete data which relates to particular individuals, should this be required.
Areas of IIB behaviour to consider for GDPR Client Data deletion
Areas of IIB behaviour to consider for GDPR Account Data deletion
IIB provides a "Monitoring" feature that IIB users can exploit to publish copies of the data passing through a flow over MQ or MQTT transports.
Read more:
IIB provides a feature that enables users to distribute system log information to an IBM Cloud Log Analysis service, so that the IIB user can combine data from multiple IIB systems and can examine that feedback by using Elastic Search, LogStash, and Kibana (ELK).
Read more:
As part of standard operation, IIB records information in the System Log of the platform where it is installed (for example, Windows Event Viewer) to describe Informational, Warning and Error information. It is possible in some circumstances that the diagnostic information provided in these entries will include extracted portions of data from individual messages that have passed through IIB. For example, if you were passed an input message into IIB and a parsing exception were thrown, IIB would inform the user where the parsing failed, with a byte location and an extract of the data which was found immediately prior to the parsing failure. This is designed to help users interpret and correct the problem (either by correcting the data itself, or the message model used to parse it)
Activity Logs provide immediate, basic information about what is happening in your message flows, and how they are interacting with external resources. The logs are also useful for diagnosing problems that would not be easily resolved by using lower-level trace. For example, they can help you to understand why your message flow is not processing any messages from a remote resource. Activity logging is enabled by default. To view activity log you can use the web user interface, or if wanting to see information over a longer period you can configure IIB to write the same data to a file. Some users may prefer to control access to this information by applying authorization rules for users logging in to the IIB web user interface.
Read more:
Using the facilities summarized in this document, IIB enables an end-user to restrict usage of their personal data.
Under GDPR, users have rights to Access, Modify and Restrict Processing. Refer to other sections of this document to control the following: