Best Practices for Memory Management

Outlines memory management best practices to maintain a healthy runtime environment and avoid out of memory errors.

An Integration Appliance sets aside a pool of memory to run orchestration jobs. Managing this pool of memory is key to maintaining a healthy runtime environment.

There is a direct relationship between the amount of memory used and garbage collection cycles. Garbage collection is a process that returns memory, held by completed jobs and variable data, back to a pool of memory used by all orchestration jobs. This garbage collection process ensures that the Integration Appliance has a constant supply of free memory that new orchestration jobs can use.

Garbage collection (GC) is a continual process that runs in the background with no effect on orchestration job performance. However, when memory reaches a critical threshold, a more in-depth process known as full garbage collection occurs. The full garbage collection process puts all the running jobs into a sleep state and returns unused memory back to the pool. Because all orchestration jobs stop during a full garbage collection, frequent full garbage collections can affect orchestration performance.

When the GC Activity value on the Resource Utilization Graph spikes at regular intervals, there is most likely a high demand on memory that the Integration Appliance is trying to manage by initiating more frequent full garbage collection cycles. However, high memory usage does not necessarily mean an increase in full garbage collection cycles. For example, a large volume of short running orchestrations results in a larger percentage of memory being used. But the garbage collection cycles, constantly running in the background, most likely return memory back to the pool of memory quickly enough that the overall memory usage is never high enough to trigger a full garbage collection. Full garbage collections stop all orchestrations, so frequent full garbage collections can affect orchestration performance.

Scenarios where high memory usage likely results in frequent full garbage collection cycles include a mix of orchestration types. For example, having multiple, non-persistent, long running orchestrations with numerous variables processing large objects. These types of orchestrations would consume and tie up large amounts of memory that could not quickly be returned to the memory pool.
Note: These types of orchestrations are not recommended by Cast Iron®.

The following table provides a list of best practices that will help you manage memory usage and maintain a healthy runtime environment.

Table 1. Best Practices for Memory Management
Best Practice Principles Description
Reduce logging levels. More detailed logging requires more memory to process and store the data and also increases the I/O load on the Integration Appliance disk, which can slow performance. Detailed logging levels are recommended for debugging purposes only and not recommended for production environments, where high volumes of data are processed.

The Integration Appliance generates system logs and orchestration job logs.

Cast Iron recommends setting the System Log level to "Warning" for all components that are tracked in the system log:
  • Hardware
  • Resources
  • Network
  • Security
  • Orchestration
  • Deployment
If the number of warnings generated by the various system components is high, memory usage might become an issue. Either resolve the problem that is causing the warnings to be generated or increase the system logging level to "Error". For more information about setting system log levels, see Specify System Log Settings.

Orchestration logging levels are specified for each orchestration in a project. Cast Iron recommends setting Logging Levels to "Error Values" for all orchestrations under all projects on the Integration Appliance. For more information about setting orchestration log levels, see Editing Orchestration Settings.

Verify persistence is enabled for all orchestrations. By default, persistence is enabled for an orchestration and variable data is written to disk instead of memory. Enabling persistence has the added benefit of point-in-time data recovery should a fault occur during runtime.

If you disable persistence for an orchestration, data is stored in memory. While disabling persistence can improve performance, as the number of running jobs increases the likelihood of running out of memory also increases. Because of the high risk of running out of memory, proceed with extreme caution when you choose to disable persistence.

Note: Turning off persistence reduces the number of orchestration jobs that the Integration Appliance can run simultaneously. The limit is set by the amount of memory available on the Integration Appliance.
Use scheduling capabilities available in the Web Management Console (WMC) and Studio to stagger orchestration jobs. By scheduling memory intensive jobs to start at different times or to run during off-peak hours, you relieve some of the load on memory, reduce the number of full garbage collections, and potentially improve performance.

In Studio, you can use the Schedule Job activity or configure polling intervals for activities in an orchestration to stagger orchestration jobs and maximize resource utilization. You can also create a schedule in the WMC, to control when the Integration Appliance should run an orchestration job. Measure the average execution time of a given orchestration job, so you can optimize job schedules.

Minimize the number of variables used in an orchestration. The more variables you use in an orchestration, the more memory is required to store the data. This in turn could increase the number of garbage collection cycles and affect performance. Review your orchestrations to see if you can reduce the number of variables used.
Reduce the number of simultaneously running jobs. In the WMC, you are able to specify the maximum number of simultaneously running jobs for an orchestration. As the number of concurrently running orchestrations increases, so does the percentage of memory usage increase. If you notice memory is getting overloaded, reduce the number of jobs running simultaneously.

The effects of reducing the number of simultaneously running jobs is especially impactful for non-persistent, long running orchestrations jobs that contain a large number of variables.

Note: These types of orchestrations are not recommended by Cast Iron.
Do not exceed 75% load on memory. In the WMC Resource Utilization Graph, you can track the percentage of memory used. When the percentage of memory used is approximately 75% or less, the Integration Appliance has capacity to process more orchestrations. When the percentage of memory used exceeds approximately 75%, you will likely notice performance issues as the number of full garbage collections increase.
Limit full garbage collection cycles. In the WMC Resource Utilization Graph, you can use the GC Activity value to track a percentage of full garbage collections. When this percentage exceeds 50%, you might notice a reduction in the throughput of your orchestrations jobs. Implement the best practices for memory management to reduce memory usage and thus reduce the likelihood of full garbage collection cycles occurring.
Add additional Integration Appliances to your environment. After implementing the best practices principles, if memory usage continues to be high, it might be time to add another Integration Appliance to your environment.



Feedback | Notices


Timestamp icon Last updated: Wednesday, 15 June 2016


http://pic.dhe.ibm.com/infocenter/wci/v7r0m0/topic/com.ibm.wci.appliance.doc/ApplianceHelp/Best_Practices/best_practices.html