FileNet P8 Platform, Version 5.2.1            

Configuring sorting for Oracle Directory Server Enterprise Edition

You can configure your Oracle Directory Server Enterprise Edition or your Sun Java System Directory Server to better support FileNet® P8, especially in the area of sorting.

Sun Java System Directory Server glossary definitions, for easy reference
All IDs Threshold
A size limit which is globally applied to every index key managed by the server. When the size of an individual ID list reaches this limit, the server replaces that ID list with an All IDs token.
browsing index
Otherwise known as the virtual list view index, speeds up the display of entries in a targeted directory branch. Browsing indexes can be created on any branch in the directory tree to improve search performance.
filter
A constraint applied to a directory query that restricts the information returned.
nsslapd-sizelimit (Size Limit)
A globally applied size limit that specifies the maximum number of entries to return from a search operation. If this limit is reached, ns-slapd returns any entries it has located that match the search request, as well as an exceeded size limit error.
nsLookthroughLimit
Specifies the maximum number of entries that the directory will check when examining candidate entries in response to a search request. If entries to be checked exceeds this limit, an error is returned.
substring index
Allows for efficient searching against substrings within entries.
virtual list view index
Otherwise known as a browsing index, speeds up the display of entries in a targeted directory branch. Virtual list view indexes can be created on any branchpoint in the directory tree to improve search performance.
Recommendations

Consult the Sun Java System Directory Server documentation for more information on the features mentioned here, particularly the chapter on Directory Server Indexing in the Administration Guide.

  • There is a performance cost associated with a wide range search such as using a single character search pattern or no-pattern. When possible, use a narrowed search pattern. For example, use a more complete searchPattern string: "stan" instead of "s".
  • Create substring indexes for the uid and cn fields that typically map to the Content Platform Engine User Short Name Attribute, Group Short Name Attribute, User Display Name Attribute, and Group Display Name Attribute properties. (To view these properties in Administration Console for Content Platform Engine, click the Domain node, click the Directory Configuration tab, then click the directory server listed under Name.)
  • Increase the All Ids Threshold from its default of 4000, as appropriate. For example, if its value is 4,000 and if there are more than 4,000 entries in the Directory Server that start with "sys", then the Directory Server will no longer maintain that index. This means that Directory Server will return an error if you do a sorted query for "StartsWith sys", even if you set the maximum results parameter to less than 4,000. According to Sun's documentation, you should try to maintain this threshold at no less than 5% of your total directory. That is, the default of 4,000 assumes an authentication directory containing approximately 80,000 entries. If your directory contained 150,000 entries, you would probably want to increase the All Ids Threshold to at least 7500.
  • Increase the Look Through Limit from its default of 5000 to -1 (no limit) so that there is no maximum number of accounts that will be checked in any given search. An alternative would be to set this value to such a large number that virtually no search would reach the maximum.
    Important: The Directory Server might return an error when the Directory Server entry count is over this limit, regardless of whether the query is sorted.
  • Increase the Size Limit parameter (default of 2000) to a value larger than the maximum-results entry set for any associated FileNet P8 user interface. This ensures that the expected maximum number of results will be returned. Otherwise, users could get back the number of records determined by the size limit parameter and not be aware that there are in fact more entries that meet the search criteria.
  • Set a browsing index in your Sun Java System directory to improve the performance of a no-pattern query.
    Important: Queries without a search pattern are used to retrieve all entries. if there is no matching browsing index and the resultant entries count is greater than the All IDs Threshold Limit, Directory Server returns an error such as:
    [LDAP: error code 12 - Sort Response Control]
More information about the recommendations
Simply increasing the All IDs Threshold by itself is not advisable, given the Sun Java System Directory Server recommendation to keep the All IDs Threshold value at about 5% of the Directory Server total entry count. Also, handling single-character searches without using a substring index could incur performance problems. Therefore it is recommended that you select an approach combining browsing index, substring index, and manipulating the All IDs Threshold. The following sections describe some of the issues to be resolved while implementing the recommendations.
Create a substring index for sorting attributes

For each attribute to be sorted, create a substring index in order to support the FileNet P8 pattern search. Remember that the Directory Server will still return an error if the entry count for a specific index is over the setting of All IDs Threshold.

You should enable a substring index for each attribute you assigned as the Content Platform Engine properties UserShortNameAttribute, GroupShortNameAttribute, UserDisplayNameAttribute, and GroupDisplayNameAttribute. They are shown on the Directory Configuration tab of the Domain node in Administration Console for Content Platform Engine. For Content Platform Engine, these attributes are normally uid and cn.

The substring index does not support the case of querying all users (which is done by leaving the FileNet P8 search string empty). This problem can be resolved by creating a browsing index, because the filter is static so only a few browsing indexes are needed.

Change the All IDs Threshold value

With the substring index enabled, a multi-character pattern search works as long as the targeted pattern's entry count is less than All IDs Threshold.

Change the All IDs Threshold to a number greater than the Directory Server entry count to ensure that single-character pattern searches work. There are, however, some costs of doing so. For example, when an index for a specific attribute value is over the All IDs Threshold value, the Directory Server will not maintain the index list for that value. In order to resolve this problem, the All IDs Threshold should be increased. The Sun recommendation is to keep the All IDs Threshold value at about 5% of the Directory Server total entry count, but even this percentage might have to be adjusted.

Example

Assume the Directory Server has 80,000 entries and the threshold is 4000. Also assume there are 5000 entries that start with "pw" and 1000 entries that start with "au". When you query for "au*" you get 1000 entries, no error. But when you query "pw*" you would get an error because there are more than 4000 entries that start with "pw" (in fact there are 5000 entries) and Sun Java System Directory Server stops sorting for this case. You would have to increase the threshold to a number over 5000 in order to get back all entries that fulfill the query's specifications.

Effects of changing the All IDs Threshold without enabling a substring index
  • When the Directory Server entry count is over the setting of All IDs Threshold, Directory Server returns an error such as
    [LDAP: error code 12 - Sort Response Control]

    This could happen when searching using a broad search scope such as with a single-character search pattern .

  • All substring queries work as long as the All IDs Threshold is greater than the Directory Server entry count.
  • If you do not use a substring index, the search might complete without error because the entries are within the limit, but you might experience poor search performance.
Recommendation
  • Change sizeLimit at the global config entry (dn: cn=config) to -1 (no limit).
  • Change nsslapd-lookthroughlimit to -1 (no limit).
  • Set the All IDs Threshold to a value no less than 5% of the Directory Server entry count, but monitor and test user and group account searches carefully to determine whether a higher value might be more appropriate.
How to create a browsing index on Sun One Directory Server (For Sun Java System Directory Server, see procedure below.)

The substring index does not help queries without pattern. A query without pattern returns an error when the Directory Server entry count is over the All IDs Threshold value. You can avoid this error by creating a browsing index for that specific search.

Consult the Administration Guide for Sun One Directory Server for more details on managing browsing indexes.

Here are steps to create a browsing index. The following example uses the following assumptions:
  • dc=eng,dc=filenet,dc=com is a Directory suffix and is the base DN
  • eng is the database name for this Directory suffix.
  • the Short name and Display name of user entries is uid
  • the Short name and Display name of group entries is cn
  1. Create a file BrowsingIndex.txt with the following contents:
    dn: cn="dc=eng,dc=filenet,dc=com:(objectClass=person)", 
       cn=eng,cn=ldbm database,cn=plugins,cn=config 
    
    objectClass: top 
    objectClass: vlvSearch 
    cn: "dc=eng,dc=filenet,dc=com:(objectClass=person)"
    vlvbase: dc=eng,dc=filenet,dc=com
    vlvscope: 2
    vlvfilter: (&(objectClass=person)(uid=*))
    
    dn: cn=sort uid,cn="dc=eng,dc=FileNet,
       dc=com:(objectClass=person)", cn=eng, cn=ldbm database, 
          cn=plugins,cn=config
    
    objectClass: top
    objectClass: vlvIndex
    cn: sort uid
    vlvSort: uid
    
    dn: cn=rev sort uid, cn="dc=eng,dc=filenet, 
       dc=com:(objectClass=person)", cn=eng, cn=ldbm database, 
          cn=plugins, cn=config
    
    objectClass: top
    objectClass: vlvIndex
    cn: rev sort uid
    vlvSort: -uid
    
    dn: cn="dc=eng,dc=filenet,
       dc=com:(objectClass=groupOfUniqueNames)", cn=eng, 
          cn=ldbm database, cn=plugins, cn=config
    
    objectClass: top
    objectClass: vlvSearch
    cn: "dc=eng,dc=filenet,dc=com:(objectClass=groupOfUniqueNames)"
    vlvbase: dc=eng,dc=filenet,dc=com
    vlvscope: 2
    vlvfilter: (&(objectClass=groupOfUniqueNames)(cn=*))
    
    dn: cn=sort cn,cn="dc=eng,dc=filenet, 
       dc=com:(objectClass=groupOfUniqueNames)", cn=eng, 
          cn=ldbm database,cn=plugins,cn=config
    
    objectClass: top
    objectClass: vlvIndex
    cn: sort cn
    vlvSort: cn
    
    dn: cn=rev sort cn,cn="dc=eng,dc=filenet,
       dc=com:(objectClass=groupOfUniqueNames)", cn=eng, 
          cn=ldbm database, cn=plugins, cn=config
    
     objectClass: top
     objectClass: vlvIndex
     cn: rev sort cn
     vlvSort: -cn
    
    dn: cn=sort_users_cn,cn="dc=eng,dc=filenet,
       dc=com:(objectClass=person)", cn=eng,
          cn=ldbm database,cn=plugins,cn=config
    
     objectClass: top
     objectClass: vlvIndex
     cn: sort_users_cn
     vlvSort: cn
    
    dn: cn=rev_sort_users_cn,cn="dc=eng,dc=filenet,
       dc=com:(objectClass=person)", cn=eng, 
          cn=ldbm database, cn=plugins, cn=config
    
     objectClass: top
     objectClass: vlvIndex
     cn: rev_sort_users_cn
     vlvSort: -cn

    The first three entries are browsing-index entries for querying users. The next three entries are browsing- index entries for querying groups. The last two are sort indexes on objectClass=person, based on cn instead of uid. Each vlvSearch entry is tied to a specific base DN and a specific filter.

    The vlvBase must be the same as specified in the User/Group Base DN in the Administration Console for Content Platform Engine Directory Configuration. The vlvFilter must match the User/Group Search Filter as well. For instance, if the following User Search Filter is specified in Directory Configuration:
    (&(objectClass=person)(uid={0}))

    then the vlvFilter should be (&(objectClass=person)(uid=*)).

    The valid values for vlvScope are:
    Value Indication
    0 The base entry alone
    1 The immediate children of the base
    2 The entire subtree rooted at the base

    However, only 2 should be used since the Content Platform Engine search query always implies a subtree search.

    Both vlvSearch and vlvIndex entries should be named. In this example, the vlvSearch name is formed by concatenating the base DN and the filter as a unique name. For the user case, one vlvIndex is named sort uid and the second one is named rev sort uid . It is assumed that the user Short name attribute and Display name attribute are the same attribute: uid. If they are not the same, then two more vlvIndex entries should be added for another attribute. In this example, these vlvSearch and vlvIndex entries are specific to the base DN dc=eng,dc=filenet,dc=com. If there are more realms on the server, a new set of entries should be added for each realm.

  2. Run the ldapmodify utility to create vlvSearch and vlvIndex entries using the above file as input:
    <SUNONE_INSTALL_DIR>\shared\bin\ldapmodify -a -h 
       <SUNONE_HOST_NAME> -p <SUNONE_PORT_NUMBER> -D 
          <USER_ID> -w <PASSWORD> -f <full path of BrowsingIndex.txt>
    For example,
    Ldapmodify -a -h hq-sunds -p 1389 -D "cn=Directory Manager" 
       -w -Directory Manager" -f C:\temp\BrowsingIndex.txt
  3. Stop the directory server:

    In the Sun One Directory Server console, under Tasks tab, click Stop Directory Server.

  4. Create browsing indexes on Sun One Directory Server:
    <SUNONE_INSTALL_DIR>\bin\slapd\server\slapd db2index -D 
       "<SUNONE_INSTALL_DIR>\slapd- <SUNONE_SERVER_NAME>" 
          -n <DB_NAME> -T "sort uid"
    
     <SUNONE_INSTALL_DIR>\bin\slapd\server\slapd db2index -D 
       "<SUNONE_INSTALL_DIR>\slapd- <SUNONE_SERVER_NAME>" 
          -n <DB_NAME> -T "rev sort uid"
    
     <SUNONE_INSTALL_DIR>\bin\slapd\server\slapd db2index -D 
       "<SUNONE_INSTALL_DIR>\slapd- <SUNONE_SERVER_NAME>" 
          -n <DB_NAME> -T "sort cn"
    
     <SUNONE_INSTALL_DIR>\bin\slapd\server\slapd db2index -D 
       "<SUNONE_INSTALL_DIR>\slapd- <SUNONE_SERVER_NAME>" 
          -n <DB_NAME> -T "rev sort cn" 

    Repeat the process above if there are more indexes added to the BrowsingIndex.txt file.

    Note that <DB_NAME> is the cn name under cn=idbm database,cn=plugins,cn=config. For example,
    Slapd db2index -D "C:\Program Files\Sun\MPS\slapd-myserver" 
       -n eng -T "sort uid"
  5. Start the directory server.
  6. Run the Sun One Directory Server Console, and open a server window.
  7. Click the Directory tab.
  8. Expand config in the left pane, then expand plugins and ldbm database.
  9. Click <DB_Name> entry, such as eng in this example and verify that there are vlvSearch entries (for both user and group) created by the earlier step.
  10. Right-click the <DB_NAME> entry, and select Set Access Permissions.
  11. Add a new ACI as anonymous user. This will allow all users to access these vlvSearch entries in the database.
How to create a browsing index on Sun Java System Directory Server

The substring index does not help queries without pattern. A query without pattern returns an error when the Directory Server entry count is over the All IDs Threshold value. You can avoid this error by creating a browsing index for that specific search.

Consult the Administration Guide for Sun Java System Directory Server for more details on managing browsing indexes.

Here are steps to create a browsing index. The following example uses the following assumptions:
  • dc=eng,dc=filenet,dc=com is a Directory suffix and is the base DN
  • eng is the database name for this Directory suffix.
  • the Short name and Display name of user entries is uid
  • the Short name and Display name of group entries is cn
  1. Create a file BrowsingIndex.txt with the following contents:
    dn: cn="dc=eng, dc=filenet, dc=com:(objectClass=person)", 
       cn=eng, cn=ldbm database, cn=plugins, cn=config 
    
    objectClass: top 
    objectClass: vlvSearch 
    cn: "dc=eng,dc=filenet,dc=com:(objectClass=person)"
    vlvbase: dc=eng,dc=filenet,dc=com
    vlvscope: 2
    vlvfilter: (&(objectClass=person)(uid=*))
    
    dn: cn=sort uid, cn="dc=eng, dc=FileNet, dc=com:(objectClass=person)", 
       cn=eng, cn=ldbm database, cn=plugins, cn=config
    
    objectClass: top
    objectClass: vlvIndex
    cn: sort uid
    vlvSort: uid
    
    dn: cn=rev sort uid, cn="dc=eng,dc=filenet,dc=com:(objectClass=person)",
       cn=eng, cn=ldbm database, cn=plugins, cn=config
    
    objectClass: top
    objectClass: vlvIndex
    cn: rev sort uid
    vlvSort: -uid
    
    dn: cn="dc=eng, dc=filenet, dc=com:(objectClass=groupOfUniqueNames)", 
       cn=eng, cn=ldbm database, cn=plugins, cn=config
    
    objectClass: top
    objectClass: vlvSearch
    cn: "dc=eng,dc=filenet,dc=com:(objectClass=groupOfUniqueNames)"
    vlvbase: dc=eng,dc=filenet,dc=com
    vlvscope: 2
    vlvfilter: (&(objectClass=groupOfUniqueNames)(cn=*))
    
    dn: cn=sort cn, cn="dc=eng, dc=filenet, 
       dc=com:(objectClass=groupOfUniqueNames)", 
          cn=eng, cn=ldbm database, cn=plugins, cn=config
    
    objectClass: top
    objectClass: vlvIndex
    cn: sort cn
    vlvSort: cn
    
    dn: cn=rev sort cn,cn="dc=eng,dc=filenet, 
       dc=com:(objectClass=groupOfUniqueNames)", 
          cn=eng, cn=ldbm database, cn=plugins, cn=config
    
     objectClass: top
     objectClass: vlvIndex
     cn: rev sort cn
     vlvSort: -cn
    
    dn: cn=sort_users_cn, cn="dc=eng, dc=filenet, 
       dc=com:(objectClass=person)", cn=eng, cn=ldbm database, 
          cn=plugins, cn=config
    
     objectClass: top
     objectClass: vlvIndex
     cn: sort_users_cn
     vlvSort: cn
    
    dn: cn=rev_sort_users_cn, cn="dc=eng, dc=filenet, 
       dc=com:(objectClass=person)", cn=eng, 
          cn=ldbm database, cn=plugins, cn=config
    
     objectClass: top
     objectClass: vlvIndex
     cn: rev_sort_users_cn
     vlvSort: -cn

    The first three entries are browsing-index entries for querying users. The next three entries are browsing- index entries for querying groups. The last two are sort indexes on objectClass=person, based on cn instead of uid. Each vlvSearch entry is tied to a specific base DN and a specific filter.

    The vlvBase should be the same as specified in the User/Group Base DN in the Administration Console for Content Platform Engine Directory Configuration. The vlvFilter should match the User/Group Search Filter as well. For instance, if the following User Search Filter is specified in Directory Configuration:
    (&(objectClass=person)(uid={0}))

    then the vlvFilter should be (&(objectClass=person)(uid=*)).

    The valid values for vlvScope are:
    Value Indication
    0 The base entry alone
    1 The immediate children of the base
    2 The entire subtree rooted at the base

    However, only 2 should be used since the Content Platform Engine search query always implies a subtree search.

    Both vlvSearch and vlvIndex entries should be named. In this example, the vlvSearch name is formed by concatenating the base DN and the filter as a unique name. For the user case, one vlvIndex is named sort uid and the second one is named rev sort uid. It is assumed that the user Short name attribute and Display name attribute are the same attribute: uid. If they are not the same, then two more vlvIndex entries should be added for another attribute. In this example, these vlvSearch and vlvIndex entries are specific to the base DN dc=eng,dc=filenet,dc=com. If there are more realms on the server, a new set of entries should be added for each realm.

  2. Run the ldapmodify utility to create vlvSearch and vlvIndex entries using the above file as input:
    <SUNONE_INSTALL_DIR>\shared\bin\ldapmodify -a -h 
        <SUNONE_HOST_NAME> -p <SUNONE_PORT_NUMBER> -D 
           <USER_ID> -w <PASSWORD> -f <full path of BrowsingIndex.txt>
    For example,
    Ldapmodify -a -h hq-sunds -p 1389 -D "cn=Directory Manager" 
       -w -Directory Manager" -f C:\temp\BrowsingIndex.txt
  3. Stop the directory server:

    In the Sun Directory Service Control Center, navigate to the Server Operation > Main tab and click Stop.

  4. Create browsing indexes on Sun Java System Directory Server:
    <SJDS_INSTALL_DIR>\bin\dsadm reindex –l –b –t <vlvIndex Name> 
       <LDAP_SERVER_INSTANCE> <SJDS_SERVER_NAME>

    Repeat the command above for all vlvIndexes. For example,

    Note that <DB_NAME> is the cn name under cn=idbm database,cn=plugins, cn=config. For example,
    Dsadm reindex –l –b –t “sort uid” ..\local\instance1 hq-sunds
  5. Start the directory server.
  6. Run the Sun Directory Server Control Center, then select the directory server Entry Management > Access Control.
  7. Click New ACI from Wizard and follow the steps to create an ACI that grants access rights to all users to search using the vlvSearch entries just created. On the Choose Target step, use the browse button to select the entry that is the parent of the vlvSearch entries created earlier. For example,
    cn=eng,cn=ldbm database,cn=plugins,cn=config

    In the Options dropdown box, select ACI Applies to All Entries Below This Entry.



Last updated: October 2015
p8psd032.htm

© Copyright IBM Corporation 2015.