This scenario has three phases:
All three queues have jobs running, and LSF assigns the number of slots to queues as expected: 8, 5, 2. Though queue Genova deserves 3 slots, the total slot assignment must be 15, so Genova is allocated only 2 slots:
When queue Verona has done its work, queues Roma and Genova get their respective shares of 8 and 3. This leaves 4 slots to be redistributed to queues according to their shares: 50% (2 slots) to Roma, 20% (1 slot) to Genova. The one remaining slot is assigned to queue Roma again:
When queues Roma and Verona have no more work to do, Genova can use all the available slots in the cluster:
poolA uses 15 slots and contains queues Roma (50% share, 8 slots), Verona (30% share, 5 slots), and Genova (20% share, 2 remaining slots to total 15).
poolB with 15 slots containing queues Pisa (30% share, 5 slots), Venezia (30% share, 5 slots), and Bologna (30% share, 5 slots).
Two other queues Milano and Parma do not belong to any pool, but they can use the hosts of poolB. The queues from Milano to Bologna all have the same priority.
The queues Milano and Parma run very short jobs that get submitted periodically in bursts. When no jobs are running in them, the distribution of jobs looks like this:
QUEUE_NAME PRIO STATUS MAX JL/U JL/P JL/H NJOBS PEND RUN SUSPRoma 50 Open:Active - - - - 1000 992 8 0Verona 48 Open:Active - - - - 1000 995 5 0Genova 48 Open:Active - - - - 1000 998 2 0Pisa 44 Open:Active - - - - 1000 995 5 0Milano 43 Open:Active - - - - 2 2 0 0Parma 43 Open:Active - - - - 2 2 0 0Venezia 43 Open:Active - - - - 1000 995 5 0Bologna 43 Open:Active - - - - 1000 995 5 0
When Milano and Parma have jobs, their higher priority reduces the share of slots free and in use by Venezia and Bologna:
QUEUE_NAME PRIO STATUS MAX JL/U JL/P JL/H NJOBS PEND RUN SUSPRoma 50 Open:Active - - - - 992 984 8 0Verona 48 Open:Active - - - - 993 990 3 0Genova 48 Open:Active - - - - 996 994 2 0Pisa 44 Open:Active - - - - 995 990 5 0Milano 43 Open:Active - - - - 10 7 3 0Parma 43 Open:Active - - - - 11 8 3 0Venezia 43 Open:Active - - - - 995 995 2 0Bologna 43 Open:Active - - - - 995 995 2 0
Pool poolA has 3 hosts each with 7 slots for a total of 21 slots to be shared. The first 3 queues are part of the pool poolA sharing the CPUs with proportions 50% (11 slots), 30% (7 slots) and 20% (3 remaining slots to total 21 slots).
The other 10 queues belong to pool poolB, which has 3 hosts each with 7 slots for a total of 21 slots to be shared. Each queue has 10% of the pool (3 slots).
The initial slot distribution looks like this:
bqueuesQUEUE_NAME PRIO STATUS MAX JL/U JL/P JL/H NJOBS PEND RUN SUSPRoma 50 Open:Active - - - - 15 6 11 0Verona 48 Open:Active - - - - 25 18 7 0Genova 47 Open:Active - - - - 460 455 3 0Pisa 44 Open:Active - - - - 264 261 3 0Milano 43 Open:Active - - - - 262 259 3 0Parma 42 Open:Active - - - - 260 257 3 0Bologna 40 Open:Active - - - - 260 257 3 0Sora 40 Open:Active - - - - 261 258 3 0Ferrara 40 Open:Active - - - - 258 255 3 0Napoli 40 Open:Active - - - - 259 256 3 0Livorno 40 Open:Active - - - - 258 258 0 0Palermo 40 Open:Active - - - - 256 256 0 0Venezia 4 Open:Active - - - - 255 255 0 0
Initially, queues Livorno, Palermo, and Venezia in poolB are not assigned any slots because the first 7 higher priority queues have used all 21 slots available for allocation.
As jobs run and each queue accumulates used slots, LSF favors queues that have not run jobs yet. As jobs finish in the first 7 queues of poolB, slots are redistributed to the other queues that originally had no jobs (queues Livorno, Palermo, and Venezia). The total slot count remains 21 in all queues in poolB.
bqueuesQUEUE_NAME PRIO STATUS MAX JL/U JL/P JL/H NJOBS PEND RUN SUSPRoma 50 Open:Active - - - - 15 6 9 0Verona 48 Open:Active - - - - 25 18 7 0Genova 47 Open:Active - - - - 460 455 5 0Pisa 44 Open:Active - - - - 263 261 2 0Milano 43 Open:Active - - - - 261 259 2 0Parma 42 Open:Active - - - - 259 257 2 0Bologna 40 Open:Active - - - - 259 257 2 0Sora 40 Open:Active - - - - 260 258 2 0Ferrara 40 Open:Active - - - - 257 255 2 0Napoli 40 Open:Active - - - - 258 256 2 0Livorno 40 Open:Active - - - - 258 256 2 0Palermo 40 Open:Active - - - - 256 253 3 0Venezia 4 Open:Active - - - - 255 253 2 0
The following figure illustrates the round-robin distribution of slot allocations between queues Livorno and Palermo:
In the following examples, job runtime is not equal, but varies randomly over time.
In this example, queue1 (the curve with the highest peaks) has the longer running jobs and so has less accumulated slots in use over time. LSF accordingly rebalances the load when all queues compete for jobs to maintain a configured 10% usage share.