scrubber

Removes data containers from VOB storage pools and removes DOs from VOB database

APPLICABILITY

Product	Command type
ClearCase	command
ClearCase LT	command

Platform
UNIX
Windows

SYNOPSIS

scrubber [ –e | –f | –o ] [ –p pool[,...] | –k kind[,...] ]: [ –a | vob-storage-dir-pname ... ]

DESCRIPTION

The scrubber program deletes (scrubs) data container files from the cleartext storage pools and derived object (DO) storage pools of one or more VOBs. It also deletes corresponding (DOs) from a VOB database. Only cleartext pools and DO pools are affected; scrubbing is not defined for source pools.

Note: DOs are associated with dynamic views only; they are not applicable to snapshot views.

Scrubbing Algorithms

scrubber implements the following scrubbing algorithms:

Heuristic scrubbing
By default or with the –o option, scrubber uses a free-space-analysis heuristic: it compares the current free-space level of a disk partition with a lower limit computed during its previous execution. This lower limit is stored in file /var/adm/rational/clearcase/cache/scrubber_fs_info (UNIX) or ccase-home-dir\var\cache\scrubber_fs_info (Windows).
- If the free-space level is still above the computed limit, scrubber does no scrubbing in that partition, regardless of the state of the storage pools within it. This performance optimization allows a quick check to take place frequently (for example, once an hour), without much system overhead.
- If the free-space level has fallen below the limit, scrubber performs parameter-driven scrubbing of each storage pool in the partition.

Parameter-driven scrubbing

With the –f option, scrubber removes data container files from a storage pool according to the pool's scrubbing parameter settings. (The heuristic scrubbing algorithm can also fall through to this algorithm.)

When a derived object pool or cleartext pool is created with mkvob or mkpool, its scrubbing parameters are set to user-specified or default values:

maximum size	Maximum pool size (specified in KB; default=0)
reclaim size	Size to which scrubber attempts to reduce the pool (specified in KB; default=0)
age	Threshold to prevent premature scrubbing of recently referenced objects (specified in hours; default=96)

Parameter-driven scrubbing proceeds as follows:

Files are removed from a pool only if its current size exceeds its maximum size setting. In this case, scrubber begins deleting data containers that have not been referenced within age hours, proceeding on a least-recently-referenced basis.
The data container for a derived object is deleted only if the DO's reference count is zero. In this case, the derived object in the VOB database is deleted, too. The associated configuration record is also deleted if no other derived object is associated with it.
Cleartext data containers do not have reference counts; they are deleted solely on the basis of recent use.
Scrubbing stops when the pool's size falls below its reclaim size setting. But in no case does scrubber delete any object that has been referenced within the last age hours.

A maximum size of zero is a special case: it instructs scrubber to delete all data containers that have not been referenced within age hours, regardless of the reclaim size setting.

Note: The scrubber considers access time rather than modification time. If your backup utility changes the access time on objects, scrubber does not delete the object if the backup utility ran within the period of time specified by age.

Everything-goes scrubbing
With the –e option, scrubber ignores a pool's scrubbing parameters and deletes these files:
- All files from each cleartext pool
- All files with zero reference counts from each derived object pool
  To avoid deleting files that are currently being used, scrubber does not delete any file that has been accessed in the preceding two minutes.

Automatic Scrubbing

By default, the scheduler runs scrubber periodically with the –f option, so that each pool is examined individually. For information about describing and changing scheduled jobs, see the schedule reference page.

You can scrub one or more pools manually at any time.

Scrubber Log File

scrubber documents its work in the host's scrubber log file:

UNIX—/var/adm/rational/clearcase/log/scrubber_log
Windows—ccase-home-dir\var\log\scrubber_log

For example, the following partial report describes the results of scrubbing a derived object pool.

04/27/99 08:03:00 Stats for VOB betelgeuse:/usr1/vobstorage/orange.vbs Pool ddft: 04/27/99 08:03:00 Get cntr tm 918.928979 04/27/99 08:03:00 Setup tm 10631.121127 04/27/99 08:03:00 Scrub tm 1207.099240 04/27/99 08:03:00 Total tm 12757.149346 04/27/99 08:03:00 Start size 404789 Deleted 3921 Limit size 0 04/27/99 08:03:00 Start files 20349 Deleted 121 Subdir dels 0 04/27/99 08:03:00 Statistics for scrub of DO Pool ddft: 04/27/99 08:03:00 DO's 3671 Scrubs 121 Strands 1760 04/27/99 08:03:00 Lost refs 1790 No DO's 20228 04/27/99 08:03:00 No fscntrs 2

The first six lines, which contain elapsed times and file statistics, are included in the report for every pool. The last four lines are specific to DO pools.

`Get cntr tm`	Elapsed time for first scrubbing phase: walk the file system tree to get pathname, size, and referenced-time information for each container in the pool.
`Setup tm`	Elapsed time for second scrubbing phase: perform setup processing specific to the kind of storage pool. For a cleartext pool, no setup is required. For a DO pool, setup is complicated; see “Processing of Derived Object Pools”.
`Scrub tm`	Elapsed time for third scrubbing phase: determine which containers to delete and then delete them.
`Start size`	Total size (KB) of all the container files in the storage pool directory before this scrubbing.
`Deleted`	Amount of storage (KB) reclaimed by this scrubbing.
`Limit size`	Desired size of the pool (KB), as specified by the pool's maximum size parameter.
`Start files`	Total number of container files in the storage pool directory before this scrubbing.
`Deleted`	Number of container files deleted by this scrubbing.
`Subdir dels`	Number of empty subdirectories of the storage pool directory deleted by this scrubbing.
`DO's`	Total number of zero-reference-count DOs in the VOB database before scrubbing.
`Scrubs`	Total number of shared zero-reference-count DOs deleted by this scrubbing. (This number equals the `Deleted` count, unless the scrubber removed shared zero-reference-count DOs that were missing their file system containers.)
`Strands`	Total number of stranded DOs deleted by this scrubbing. (These are described below.)
`Lost refs`	Total number of lost DO reference counts deleted by this scrubbing. (These are described below.)
`No DO's`	Total number of containers in the DO pool before scrubbing that are not associated with a zero-reference-count shared DO. (Each is presumably associated with a DO that is still referenced by some view, and hence cannot be scrubbed).
`No fscntrs`	Total number of shared zero-reference-count DOs that were missing their file system containers. This statistic is printed only when this condition occurs; also, the scrubber_log displays warning messages like this one: `04/21/99 10:11:17 scrubber: Warning: Unable to remove “d/do_pool2/21/5/73f1f66679f611cea15c080009935288”: No such file or directory.`

Processing of Derived Object Pools

For a DO pool, scrubber does more than delete old, unreferenced data containers.

It finds and deletes all stranded DOs from the VOB database: DOs that were never shared and whose data containers have been deleted from view-private storage. (The VOB database is not updated when the DO's data file is removed or overwritten in the view, because of implementation restrictions.) There are no data containers in the DO storage pool for such DOs, because they were never shared. This occurs during the second phase of scrubbing.
It finds and deletes all lost DO reference counts from the VOB database. Such entries are an implementation artifact; they correspond to files that were created during a build, but deleted before the build completed. This occurs during the second phase of scrubbing.
It deletes the derived object in the VOB database corresponding to the data container, and possibly its associated configuration record as well. This occurs during the third phase of scrubbing.
It finds and deletes all stranded configuration records: CRs that do not correspond to any existing derived object.

Derived Statistics

Some interesting results can be derived from these statistics:

Total number of derived object data containers in this pool after scrubbing:
Start files - scrubs
Total number of unreferenced data containers in this pool after scrubbing:
Start files - scrubs - No DO's
Total size (KB) of the storage pool after scrubbing:
Start size - deleted

Controlling the Size of the vista.tjf File

The file vista.tjf records updates to the VOB that result from scrubber operations. vista.tjf can grow very large. For information about limiting its size, read about the file db.conf in the config_ccase reference page.

OPTIONS AND ARGUMENTS

Specifying the Scrubbing Algorithm

Default: Invokes the free-space-analysis heuristic described above, instead of examining pools individually.
–f: Examines all specified pools individually, using the parameter-driven algorithm. This does not guarantee that any objects will be removed from the pools.
–e: Examines all specified pools individually (as with –f), using the everything-goes scrubbing algorithm.
–o: Same as default.

Specifying the Pools

Default: All of a VOB's cleartext and derived object pools are scrubbed.
–p pool[,...]: Restricts scrubbing to pools with the specified names, which may occur in multiple VOBs. The list of pool names must be comma-separated with no white space.
–k kind[,...]: Restricts scrubbing to pools of the specified kinds. Valid kinds are do and cltxt. The list of kinds must be comma-separated with no white space.

Specifying the VOBs

Default: None.
–a: Scrubs all VOBs listed in the storage registry whose storage directories reside on the local host. An error occurs if a VOB is listed in the registry, but cannot be found on the local host.
vob-storage-dir-pname ...: One or more pathnames of VOB storage directories, indicating the particular VOBs to be scrubbed.

EXAMPLES

Force scrubbing of all mounted VOBs with a storage directory on the local host.
scrubber –f –a
Scrub cleartext pools in the VOB whose storage directory is /usr/vobstore/project.vobs, using the free-space analysis heuristic.
scrubber –o –k cltxt /usr/vobstore/project.vobs
Force scrubbing of the default derived object pool (ddft) and the pool named do_staged in all mounted VOBs with a storage directory on the local host.
scrubber –f –p ddft,do_staged –a