Add a New Service to the Cluster

Goal

You plan, prepare the cluster for, and add four new services ensuring you allocate resources to run workload units.

Description

This scenario takes you through the steps of getting your cluster prepared for registering four new services, modifying your service profile, and registering your service. You have already created a basic tree that follows your business structure and you have either modified the default resource group ComputeHosts or you have decided to leave that resource group as-is (with all your compute hosts).

Suppose we have two different lines of business under a consumer branch called Research: Data Flow and Storage.

Data Flow

  • Data Flow has one service: SuperMortgage

  • Data Flow needs 30 CPUs

Storage

  • Storage has three services: Fast, Slow, and SuperFast

  • Fast needs 10 CPUs, Slow needs 50 CPUs, SuperFast needs 20 CPUs

At a glance

  1. Gather the facts

  2. Plan

  3. Create consumer users

  4. Create workload execution user accounts and ensure they exist on all hosts

  5. Create the consumer structure

  6. Change your resource plan

  7. Register a new service

  8. Run your service to see instances

Gather the facts

Before starting to register your services, organize the information you have. This is the most important part of the process—planning.

Pay special attention to the number of services, how many CPUs are required, and the operating system type of your machines.

Business process

Services

Host OS type

CPU slots required

Data Flow

SuperMortgage

Linux

30

Storage

Fast

Windows

10

Slow

Linux

50

SuperFast

Windows

20

Plan

Adding services requires creating consumers and changing the resource plan to ensure the new consumers have the necessary resources.

The checklists below provide guidance for the types of information that you need when actually creating a new service in the software.

How many services do you have?

  • Number of consumers

    There must be one consumer per service.

    The consumer must be a leaf consumer—it cannot have any descendants.

  • Free CPU slots on management hosts

    There must be one CPU slot free for each service on management hosts.

    • To identify free CPU slots in the cluster, use the resource plan in the Platform Management Console.

  • Free CPU slots on compute hosts

    Are there enough CPU slots available on compute hosts to be allocated to the new services?

    The compute host resource group must have enough CPU slots available. Otherwise, you may need to add more hosts or shift resources around existing consumers.

    • To identify free CPU slots in the cluster, use the resource plan in the Platform Management Console.

Which user accounts are you going to use?

  • Workload unit submission user accounts

    In order to submit workload units to the cluster, there must be user accounts defined in EGO that can connect and submit workload units. These accounts only exist in the EGO user database.

  • OS user accounts

    Each consumer must be associated with an operating system user account (workload execution user account). A consumer’s workload units run under this account.

    The user account must exist on all the hosts in the cluster, including management hosts. On Windows, it is recommended that these user accounts be domain accounts. Also, the password must be the same on all hosts.

Summarize your plan

Have a summary of your plan so that you can refer to it when working and you can use it as a checklist.

Business process

Services

Leaf consumer name

Workload execution user account

CPU slots required on compute hosts to run workload units

CPU slots required on management hosts to run service

Data Flow

SuperMortgage

CSuperM

userm

30

1

Storage

Fast

CFast

userdf

10

1

Slow

CSlow

userds

50

1

SuperFast

CSuperFast

userdsf

20

1

Draw your consumer structure

It helps to draw the consumer and service structure before you work in the software. Note that the leaf consumers are on the right, attached to a service.

Create consumer users

Through the Platform Management Console, create user accounts for your consumers that are able to submit workload units. You give these user accounts the role of consumer users when you create the consumer.

Create workload execution user accounts and ensure they exist on all hosts

When you create a consumer, you are required to specify an OS user account—this is the OS account under which workload units run. Ensure this account exists on all hosts in the cluster.

If you specify a Windows user account, and the user has not already been configured, you have to run egosh ego execpasswd to register the user account password with EGO before the execution user can run an activity without exiting.

Create the consumer structure

Create consumers and the consumer structure that you have planned out.

Note that the consumers that are to be attached to services must specify hosts from both a resource group that contains compute hosts (ComputeHosts by default) and the ManagementHosts resource group.

Use your plan summary to enter the required information for the consumer.

Create consumers

When creating a consumer, you need the following information:

  • User account for consumer user

  • User account for consumer administrator

  • OS user account (and domain if Windows or a mixed cluster)

  • Resource group name(s) for compute hosts

  1. In the Platform Management Console, click Consumers > Consumers & Plans.

    A list of existing top-level consumers in the consumer tree displays.

    For example, CSuperM (the first leaf consumer) is under Data Flow according to the plan above. On the tree, click Data Flow. A message displays, indicating “’DataFlow’ is a lowest level consumer. Currently, no applications are registered to it, so subordinate consumers can be created”.

  2. In the consumer tree, locate and click the consumer for which you would like to add a sub-consumer.
  3. Select Global Actions > Create a Consumer.
    1. Specify the consumer name.

      For example, create a consumer with the name CSuperM.

    2. Review consumer administrators.

      A consumer administrator is pre-selected if this is a lower level consumer. Consumer administrators are only specified at the top-level consumer; they get inherited down through their branch.

      If you have not specified a consumer administrator for this branch, the default is the cluster administrator.

    3. Add a consumer user.

      Consumer users can monitor and control only the workload units they are assigned to.

    4. Specify an OS user account.

      The OS user account is the account under which all workload runs.

      If you specify a Windows user account, include the domain. For example mydomain\userA.

    5. Specify one or more resource groups this consumer should have access to.

      At minimum, select the ManagementHosts resource group to run the service. If you intend to run workload units for this consumer, select at least one other resource group for the service instance to run the activity. Only the resource groups specified by the parent consumer in the tree are available for selection. If you have not modified your resource groups, you can keep the default resource group selections.

    6. (Optional) Specify a reclaim grace period.

      If you intend this consumer to have a service that runs vital workload units that cannot be interrupted, specify the average length of a workload unit as the reclaim period. This only applies if you have set this consumer to borrow resources from other consumers (an advanced feature).

    7. For all consumers (except for those in the ManagementHosts and InternalResourceGroup resource groups), ensure the box Rebalance when time intervals change is checked within individual Consumer Properties dialog boxes.

      This ensures that when your resource plan changes according to set time intervals, that originally configured share ratios, allocations, and lend/borrow policies are reapplied and enforced across all consumer branches in the consumer tree.

    8. Click Create.
  4. Continue creating the consumers you need for your services.

    For example, under Storage create three consumers: CSlow, CFast, and CSuperFast.

Now that you have your infrastructure in place for the services you plan to register, you must make sure you allocate enough resources to the new consumers.

Change your resource plan

Change your resource plan to allocate resources for the new consumers.

Refer to your plan summary that you already created.

Allocate resources for the consumer created for the service

You must be a cluster administrator or consumer administrator to perform this task.

When allocating resources, you need to ensure there are CPU slots are available from a resource group that is not ManagementHosts.

  1. In the Platform Management Console, click Consumers > Consumers & Plans.
  2. In the consumer tree, click the cluster name. (Resource plans are set at the cluster level.)

    A list of consumers that make up the cluster displays.

  3. Click Resource Plan.
  4. From the resource group menu, select ComputeHosts (or equivalent).

    If you have already modified your default ComputeHosts resource group or have created new resource groups, select the resource group that has the compute hosts you want.

  5. Click Expand All at the bottom of the page to expand the resource group tree.

    Only the consumers that have the ComputeHost (or equivalent) resource group selected in their properties display. If the consumer you want does not display, then it does not have this resource group selected in its properties.

  6. Specify the number of CPU slots required to perform computations on compute hosts for your consumer in Owned Slots.

    For example, the plan says that the SuperMortgage service needs 30 CPUs. In the resource plan for the consumer CSuperM (the leaf consumer that will have SuperMortgage registered to it), add 30 to Owned Slots.

    If your balance shows a deficit of CPUs, this means you allocated more CPUs to a leaf consumer than were available through its parent. Adjust the ownership throughout the entire branch until the balance is 0.

    If you need more CPUs than are ever available, add hosts to your cluster or adjust your resource groups.

  7. From the resource group menu, select ManagementHosts.
  8. For your consumer, specify 1 owned CPU slot from the ManagementHosts group.

    The 1 owned CPU slot that you specify is used to run and schedule workload.

    Since it is essential that this CPU slot be available for the service when it is required, it is essential to own the 1 CPU slot. Do not enable lending.

  9. When you are satisfied with your resource allocation and your balance is zero or positive, click Apply to make it active or Export to save it for later use.

Register a new service

You must be a cluster administrator to perform this task.Registered services may need to be enabled afterwards if the service is set up to be manually started.

  1. From the Cluster page, click Monitor Services.

    The Services page displays.

  2. In your consumer tree, navigate to the location where you want to register your service (for example, SuperMortgage).

    You can only register a service to a leaf location. If your consumer has any sub-consumers, then it is not a leaf.

  3. From Global Actions, select Register a new service.

    The Service Profile dialog displays showing the minimum required and preferred attributes for a new service, along with preconfigured default values.

  4. Click Table Preferences at the top of the dialog box, and check all boxes to add the corresponding information columns to the table.
  5. Modify the service profile with service-specific details.
    1. In the sc:ServiceDefinition section, click in the ServiceName value field and type new name for the service (for example, SuperMortgage).

      The name can only contain up to 40 characters, and can contain letters, numbers, and underscores ("_") . The first character of the name cannot be a number.

    2. Insert the parameter for ResourceGroupName and then specify the resource group from which compute hosts are to be selected for the consumer, following these steps:
      1. In the sc:sc:AllocationSpecification > sc:ResourceSpecification section, click Actions.

      2. From the drop-down list, select Insert "ego:ResourceGroupName".

      3. In the ResourceGroupName row, type a resource group name in the value field.

        If you have not created new resource groups, specify the default resource group ComputeHosts. Note that the resource group name must exactly match the compute host resource group specified in the consumer properties.

    3. In the sc:ActivityDescription > ego:ActivitySpecification section, select Insert "ego:ExecutionUser" from the drop-down list, and then specify the OS execution user account that runs the service (for example, egoadmin).
  6. Click Register.

Your service is registered. You may see the state change between DEFINED, INIT, ALLOCATING, and STARTED.

If your state remains DEFINED, you need to start your service.

Run your service to see instances

If you want to run your service from a machine that is not part of the cluster, you need to configure the machine to connect to the cluster.

  1. Run your service.
    1. From the Cluster page, click Monitor Services.

      The Services page displays.

    2. In your consumer tree, navigate to the location where the service you want to start is registered or see a list of all registered services by clicking the cluster name.
    3. For the service you want to start, select Actions > Start.
    4. Click OK to confirm that you want to start that service.
  2. Use the Platform Management Console to view service instances.
    1. From the Services page, locate your service.
    2. Click the name of your service, and then click Service Instances.

      A list of service instances display.

    3. For details about any service instances, click the service instance Sequence number.

      Service instance properties display.

  3. Repeat the process for all your new services.