Consumer section

Settings related to the application and resources assigned to the consumer associated with the application.

applicationName

Name of the application associated with the consumer. The application name must be unique within the cluster.

Enclose the application name in double quotes if it contains spaces, for example, name="Commodity Derivatives App".

Where used

Consumer

Required/Optional

Required

Valid values

In the Platform Management Console: Accepts "a-z, A-Z, 0-9 or spaces. Do not use "all" as an application name—it is a reserved word.

On the command line: Accepts all ASCII printable characters except "\ / : * ? ’ | & < or >". Does not accept leading or trailing spaces in application names. Do not use "all" as an application name—it is a reserved word.

consumerId

Name and path of the consumer with which to associate this application. The consumer must already exist in Symphony. The consumer you associate with the application determines how Symphony allocates resources to the application.

The consumer ID includes the path to the consumer within the consumer tree.

The consumer must be a leaf consumer. A leaf consumer always resides at the lowest level of the consumer tree.

A unique consumer ID must be specified in Symphony, but it is not required to be pre-defined in SymphonyDE. SymphonyDE uses the consumer ID to associate applications and service packages. Only one application can be enabled within a consumer ID and service packages must be deployed to the same consumer ID.

Where used

Consumer

Required/Optional

Required

Valid values

Alphanumeric string that does not include any special characters.

The consumer ID begins with a forward slash (/) and each level is separated from the next with a forward slash.

Example

consumerId="/New York Cluster/Capital Markets/Derivatives"

resourceGroupName

Resource group from which resources are requested to run service instance managers and services to perform computations for clients. The resource group should contain compute hosts, not management hosts. The resource group must be an existing resource group in EGO.

Not required for Symphony DE.

Where used

Consumer

Required/Optional

Optional

Default value

" " (empty)—all resource groups which are associated with the consumer will be used.

Related attributes

  • numOfPreloadedServices—Defines the number of service instances prestarted for an application cannot be greater than the number of resources available in the resource group, defined by resourceGroupName

  • resReq—Defines the resources an application requires

resReq

String that describes criteria for selecting resources allocated to a consumer for compute hosts.

Ignored in Symphony DE.

Where used

Consumer

Required/Optional

Optional

Default value

select (!mg)

Related attributes

  • resourceGroupName—Defines the resources available to an application

Synopsis

"select(select_string)""select(select_string) order(order_string)"

Description

A resource requirement string describes the criteria for defining a set of resources for compute hosts.

Note:

If you specify a resource requirement, ensure you indicate "select(!mg)" in addition to your resource requirement string. This is so that the system only selects compute hosts to run work, not management hosts.

The entire resource requirement string cannot contain more than 512 characters.

Options

select(select_string)

Specifies the criteria for selecting the resources. The selection string filters out the resources that do not meet the criteria, resulting in a list of one or more eligible resources.

The parentheses must be typed as shown.

order(order_string)

Specifies the sort order for selecting the best resource from the list of eligible resources. The most appropriate resource is listed first, the least is listed last.

The parentheses must be typed as shown.

select

select synopsis

select(expression)select(expression operator expression)select((expression operator expression) operator expression)

select description

The selection string is a logical expression used to select one or more resources to match one or more criteria. Any resource that satisfies the criteria is selected.

resource_name operator value
resource_name

The following resources can be used as selection criteria.

Static resources

Static resources are built-in resources that represent host information that does not change over time, such as the maximum RAM available to user processes or the number of processors in a machine. Most static resources are determined at start-up time, or when hardware configuration changes are detected.

Static resources can be used to select appropriate hosts based on binary architecture, relative CPU speed, and system configuration.

Note:

The resources ncpus, ncores, nprocs, nthreads, maxmem, maxswp, and maxtmp are not static on UNIX hosts that support dynamic hardware reconfiguration.


Index

Measures

Units

Determined by

type

host type

string

configuration

model

host model

string

configuration

hname

host name

string

configuration

cpuf

CPU factor

relative

configuration

server

host can run remote jobs

Boolean

configuration

rexpri

execution priority

nice(2) argument

configuration

ncpus

number of processors

processors

LIM

ndiks

number of local disks

disks

LIM

maxmem

maximum RAM

MB

LIM

maxswp

maximum swap space

MB

LIM

maxtmp

maximum space in /tmp

MB

LIM


CPU factor (cpuf)

The CPU factor is the speed of the host’s CPU relative to other hosts in the cluster. If one processor is twice the speed of another, its CPU factor should be twice as large. CPU factors are defined by the cluster administrator. For multiprocessor hosts, the CPU factor is the speed of a single processor.

Server

The server static resource is Boolean. It has the following values:

  • 1 if the host is configured to run jobs from other hosts.

  • 0 if the host is a client for submitting jobs to other hosts.

operator
The following operators can be used in selection strings. If you are using the selection string in an XML format, you must use the applicable escape characters in the XML Equivalence column. The operators are listed in order of decreasing precedence:

Operator

XML Equivalent

Syntax

Meaning

!

n/a

!a

Logical NOT: 1 if a==0, 0 otherwise

*

n/a

a*b

Multiply a and b

/

n/a

a / b

Divide a by b

+

n/a

a+b

Add a and b

-

n/a

a-b

Subtract b from a

>

&gt;

a > b

1 if a is greater than b, 0 otherwise

<

&lt;

a < b

1 if a is less than b, 0 otherwise

>=

&gt;=

a >= b

1 if a is greater than or equal to b, 0 otherwise

<=

&lt;=

a <= b

1 if a is less than or equal to b, 0 otherwise

==

n/a

a == b

1 if a is equal to b, 0 otherwise

!=

n/a

a != b

1 if a is not equal to b, 0 otherwise

&&

&amp;&amp;

a && b

Logical AND: 1 if both a and b are non-zero, 0 otherwise

||

n/a

a || b

Logical OR: 1 if either a or b is non-zero, 0 otherwise


value

Specifies the value to be used as criteria for selecting a resource. Value can be numerical, such as when referring to available memory or swap space, or it can be textual, such as when referring to a specific type of host.

Simple selection expression using static resources

The following example selects resources with total memory greater than 500MB:
select(maxmem &gt; 500) 

Compound selection expression using static resources

The following example selects resources with total memory greater than 500MB and total swap space greater than or equal to 300MB:
select(maxmem &gt; 500 &amp;&amp; maxswp &gt;=300) 

order

Sorts the selected resources into an order of preference according to the values of the resources.

order synopsis

order(order_string)

order description

The order string acts on the results of a select string, sorting the selected resources to identify the most favorable resources, and eliminate the least desirable resources.

Resources are sorted into ascending order based on the result of an arithmetical expression.

The result orders the resources from smallest to largest.

order_string

Specify an arithmetic expression that expresses the desired outcome.

You can create an expression using operators and numbers.

Use the following operators:

+

add

subtract-

*

multiply

/

divide

Order resources based on available swap space

The following example orders the selected resources based on their available swap space, but sorts by descending order, ordering hosts with the greatest amount of available swap space first:
order(0-swp) 

policy

Specifies the workload scheduling policy used. The scheduling policy determines how and when Symphony allocates resources to run services for the application.

Where used

Consumer

Required/Optional

Optional

Default value

R_Proportion

Valid values

  • R_Proportion


    Summary

    A new session can get CPU slots immediately without waiting for current sessions to finish. Sessions simultaneously share a portion of the total

    CPU slots assigned to the application. CPU slots are proportionally assigned to sessions based on priority.

    Disadvantage

    This scheduling policy can have performance issues if it takes a long time to bind a service to a session since the service instances may toggle among open sessions.

    Initial slot allocation

    Symphony assigns CPU slots first to higher priority sessions to ensure they finish faster.

    CPU slots are assigned proportionally to sessions according to their priority.

    CPU factors are taken into account during allocation. The most powerful idle CPU is allocated first to higher priority sessions.

    The CPU factor of the compute host is defined in the conf\vem_resource.conf file under the Symphony DE installation directory, and is detected by Platform EGO in Symphony cluster when Symphony starts on the host.

    Slot reallocation based on workload

    Yes.

    When a session does not need all slots allocated to it, the slots are released and reallocated to other sessions.

    Slots are only reallocated to sessions with pending tasks.

    Higher priority sessions can take slots away

    Yes.

    As soon as a higher priority session needs more resources, Symphony can allocate slots that are being used by lower priority sessions to the higher priority session after the tasks running on the slots are finished.

    Service instance unbound from session

    Service instances are unbound from the session based on workload, when there are not enough pending tasks for all the service instances currently allocated.


  • R_MinimumServices


    Summary

    Use this scheduling policy when you are using common data so that service instances will be reused for tasks in the same session, eliminating the need to reload data for each task.

    With this scheduling policy, a minimum number of CPU slots is allocated to a session regardless of workload or priority.

    Initial slot allocation

    Symphony attempts to allocate slots to the session to satisfy the minimum configured number of service instances.

    If there are enough slots to meet the minimum number of instances for all open sessions:

    • Minimum instances are allocated to sessions based on session submission time.

    • Leftover slots are allocated to sessions proportionally based on session priority and submission time.

    If there are not enough slots to meet the minimum number for all sessions:

    • Higher priority sessions get all minimum instances satisfied first.

    CPU factors are taken into account. CPU slots are evenly distributed among sessions based on speed so that all sessions get a proportion of the fastest CPUs.

    Slot reallocation based on workload

    Partial.

    A minimum number of CPU slots is always allocated to a session, regardless of workload.

    Additional CPU slots are distributed proportionally to other open sessions with pending tasks according to priority and submission time.

    When there is no workload in the session, CPU slots not serving the minimum required service instances are distributed proportionally to other open sessions with pending tasks according to priority and submission time.

    Sessions with no pending tasks do not receive any slots, except the required minimum service instances.

    Higher priority sessions can take slots away

    Partial.

    A session always retains the required CPU slots to serve the minimum number of service instances, regardless of session priority. CPU slots serving the minimum number of service instances are not released.

    CPU slots not serving the minimum number of service instances are released and distributed to other sessions proportionally based on priority.

    Service instance unbound from session

    The minimum number of service instances are always bound to the session until the session is suspended, killed, or closed.

    Service instances additional to the minimum number are unbound from the session as needed, based on the priority and pending workload of all sessions.


Related attributes

SessionType:

  • priority—Determines how are tasks are assigned resources.

  • minServices—Determines the minimum number of CPU slots required for sessions.

Consumer:
  • taskHighWaterMark—Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested.

  • taskLowWaterMark—Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released.

  • resourceBalanceInterval—Defines when Symphony next checks to determine whether resources need to be requested or released from Platform EGO.

  • sessionSchedulingInterval—Determines when resources are balanced among sessions.

taskHighWaterMark

Applies to the whole application. Ratio is used to determine whether more CPU slots need to be requested to meet the service level agreement of the application. The ratio is the total number of pending and running (unprocessed) tasks in open sessions to CPU slots allocated.

If an application has a taskHighWaterMark of 5, it means that for every 5 unprocessed tasks, 1 CPU slot is requested.

In Symphony DE, the number of CPU slots on a host is configured in the conf/vem_resource.conf file under the Symphony DE installation directory.

In Symphony, the number of CPU slots on a host is configured in the ResourceGroups.xml file and can be configured from PMC.

Where used

Consumer

Required/Optional

Optional

Valid values

integer, 0.00000 to 9999999999.99999

Default value

1.0

Related attributes

  • resourceBalanceInterval—Defines when Symphony next checks to determine whether resources need to be requested or released from Platform EGO

  • taskLowWaterMark—Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released

taskLowWaterMark

Applies to the whole application. Ratio is used to determine whether idle CPU slots are released and made available for other applications to use. The ratio is the total number of pending and running (unprocessed) tasks in open sessions to CPU slots allocated.

This enables the application to be configured to avoid charges for unnecessary or unused resources.

For example, you set taskLowWaterMark to 2.0. You have thousands of unprocessed tasks. 100 CPU slots are allocated to the Session Manager.

  • When the number of unprocessed tasks reaches 400, the ratio of unprocessed tasks to CPU slots is 400/100= 4.0. Session Manager does not release any CPU slots.

  • When the number of unprocessed tasks reaches 200, the ratio of unprocessed tasks to CPU slots is 200/100=2.0. Session Manager does not release any CPU slots.

  • When the number of unprocessed tasks drops below 200, Session Manager releases slots until the ratio becomes 2.0 or greater.

The value of the low-water mark should be less than or equal to the high-water mark.

If taskLowWaterMark is 1.0 or less, any resources the application releases must be idle. If taskLowWaterMark is equal to 0, the application never returns resources voluntarily.

Where used

Consumer

Required/Optional

Optional

Valid values

integer, 0.00000 to 9999999999.99999

Default value

0.0

Related attributes

  • resourceBalanceInterval—Defines when Symphony next checks to determine whether resources need to be requested or released form Platform EGO

  • taskHighWaterMark—Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested

resourceBalanceInterval

Minimum number of seconds between checks to determine whether the session manager should release unused resources or request additional resources for the application.

Where used

Consumer

Valid values

1 or more

Required/Optional

Optional

Default value

5 seconds

Related attributes

  • resReq—Defines the resources an application requires

  • resourceGroupName—Defines the resource groups available to an application

  • sessionSchedulingInterval—Determines when resources are balanced among sessions

  • taskHighWaterMark—Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested

  • taskLowWaterMark—Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released

sessionSchedulingInterval

Minimum number of milliseconds between checks to determine whether compute resources need to be reallocated among sessions.

The system reallocates resources according to application session priority, scheduling policy, and pending tasks.

Where used

Consumer

Required/Optional

Optional

Valid values

1 or more

Default value

500 milliseconds

Related attributes

  • policy—Defines how resources are shared.

  • priority—Determines how are tasks are assigned resources.

  • taskLowWaterMark—Defines the minimum ratio of unprocessed tasks to CPU slots before unused resources are released

  • taskHighWaterMark—Defines the maximum ratio of unprocessed tasks to CPU slots before more resources are requested

preStartApplication

Specifies whether the session manager, service instance manager, and service instances are prestarted for enabled applications when Symphony is started, or when an application is enabled.

Prestarting the session manager provides quicker response to the first client request for this application. Prestarting the service instances manager and service instances provides quicker response to tasks.

Where used

Consumer

Required/Optional

Optional

Valid values

true | false

Default value

false

Related attributes

  • numOfPreloadedServices—Defines the number of service instances prestarted for an application cannot be greater than the number of resources available in the resource group, defined by resourceGroupName

numOfPreloadedServices

Used with pre-start application. Defines the number of service instances that are prestarted if preStartApplication is true.

One service instance manager is started for every service instance. One service instance can run on each CPU slot.

Where used

Consumer

Required/Optional

Optional

Valid values

1 or greater

Default value

1

Related attributes

  • preStartApplication—Services are preloaded only if preStartApplication is true

  • resourceGroupName—Defines the resources available to an application

transientDisconnectionTimeout

Number of seconds the session manager waits for the client to reconnect when the connection between the client and session manager is broken.

When abortSessionIfClientDisconnect is true, sessions which are open on the client connection are aborted if a disconnected client does not reconnect to the session manager.

The transientDisconnectionTimeout counter is reset when a client connects or re-connects to the session manager. Reconnection is done by the Symphony Client API, which is transparent to the client application.

Where used

Consumer

Required/Optional

Optional

Valid values

1 or more

Default value

10 seconds

Related attributes

  • abortSessionIfClientDisconnect—Specifies whether the session is aborted if the session manager detects that the connection between the client and the session manager is broken.

flushDataAsap

Used for recoverable sessions. Specifies whether or not the session manager caches data before writing to disk.

When set to true, data is not cached, it is immediately written to disk. When set to false, data is cached before it is written to disk. Note that the setting in flushDataAsap does not affect any operating system configuration. You need to disable write-behind caching for your operating system. On Windows, see http://support.microsoft.com/kb/247485 for more details. On Linux, contact your system administrator.

Important:

Setting this parameter to true may substantially degrade performance because system cannot continue with its next operation until the data is written to the disk.

Where used

Consumer

Required/Optional

Optional

Valid values

true | false

Default value

false

ioRetryDelay

Specifies the amount of time to wait, in seconds, before retrying an I/O operation after a previous failure.

Where used

Consumer

Required/Optional

Optional

Valid values

Non-negative integer

Default value

1 second

ssmRecoveryTime

For internal system use only.

tttvhjbhjhj