Instructions for IBM OmniFind Enterprise Edition, Version 8.4 Fix Pack 1 Hot Fix 6 ---------------------------------------------------------------------------------- IBM(R) OmniFind(TM) Enterprise Edition, Version 8.4 Fix Pack 1 Hot Fix 6 includes an installation program to upgrade version 8.4 Fix Pack 1, including any previously installed hot fixes, to the latest hot fix level. You must install Version 8.4 Fix Pack 1 before you install this hot fix. Version 8.4 Fix Pack 1 Hot Fix 6 addresses the following APARs: --------------------------------------------------------------- IC55387: The Web crawler fails to access URLs that are protected by form-based authentication (FBA) Code was changed to handle HTTP messages after receiving HTTP code 100/Continue. IC55497: An error FFQR0122E is returned for search requests during index switching Code was changed to accept search requests during index switching. IC55536: Multiple values in some Lotus(R) Notes(R) document fields are not picked up if the document contains attachments Code was changed in the Notes crawler to copy all values in a multi-valued metadata field from a parent Notes document to an attachment. IC55724: Search fails with FFQR0121E when the requested search results page size is more than 100 and number of results is 100 * n. Code was changed to avoid this error. IC55351: If the network becomes unavailable when crawling the Windows(R) file system, documents are removed from index. Code was changed to detect network errors and Windows file system crawlers will be stopped when target network folders are inaccessible. IC55800: An error FFQG0015E occurs in the enterprise search administration console when viewing URI details report Code was changed not to allocate a large memory buffer for this operation. APARs addressed by previous hot fixes: -------------------------------------- Version 8.4 Fix Pack 1 Hot Fix 5 addressed the following APARs: --------------------------------------------------------------- IC54367: Index builds fail with FFQI0387W because of a corrupted store IC54624: The installation program displays incorrect required disk space information in the installation summary screen IC54789: The Stellent parser cannot parse some Microsoft(R) Excel files IC54842: Queries fail with FFQR0121E when dictionary lookup is done with n-gram collections IC54876: Multiple values in Reader and Author fields are not picked up by crawlers IC54883: The customcommunication process crashes if the Lotus Notes C API returns a NULL error message during NRPC security processing IC54899: A discovery session of the Notes crawler occasionally fails IC54906: Extended highlighting is incorrect when the document language is detected incorrectly IC54939: Notes databases are not correctly added to collections IC55020: The QuickPlace(R) crawler does not support Lotus Quickr(TM) sources on a Domino(R) server IC55184: Increasing the request buffer size of the search runtime does not resolve error FFQG0009E IC55194: Quick links are not displayed for a "keyword1 keyword2" query in Japanese characters IC55195: Scheduled crawls do not start if you click Refresh at the time the crawl is scheduled to start Internal: After the e-mail option is configured, new configuration changes cannot be made Internal: The parser stops responding under a rare timing condition Version 8.4 Fix Pack 1 Hot Fix 4 addressed the following APARs: --------------------------------------------------------------- IC54344: The ESSearchApplication can not display a last page of search results when the result set is larger than 500 IC54359: The Windows file system crawler fails to crawl documents with Arabic characters in the path or filename IC54444: The Parser hangs during processing a document written in Japanese in which 1 million of a same letter repeat without new line IC54448: Both of delta and main index builds fail intermittently IC54477: When a crawler starts, it fails with FFQC5202E and NullPointerException intermittently IC54480: The MinWordsPerSentence property value is not effective for search result summary IC54500: The Federator does not sort Quick Link results from federated collections properly IC54535: The Web Content Management crawler does not crawl all of the crawl space represented in a seed list from Workplace Web Content Management(TM) 6.0.1.1 or later IC54543: Authentication is requested for a user who was already authenticated in a single sign-on enabled system Version 8.4 Fix Pack 1 Hot Fix 3 addressed the following APARs: --------------------------------------------------------------- IC53777: Re-configuration of the Web crawler to forbid many URLs causes the crawler to stop after 100 URLs are removed IC53778: The parser stops responding if another server process is listening on 127.0.0.1: IC53830: Failure to build an index when a document in a store is corrupted IC53834: Sorting search results from text metadata fields generated by a Web crawler plug-in does not work IC53978: A duplicate request ID in CCL caused the other session operation to fail IC53980: The embedded HTML parser does not work for XML files after crawling binary files such as .doc or .pdf IC54066: User validation and group extraction by the NRPC crawler takes a long time IC54147: The CASProcessor becomes out of the control when a custom annotator stops responding IC54176: Changes in the Web Content Management crawler's plug-in properties are not saved in the configuration IC54391: The NNTP or Notes crawler falls into an infinite loop when deleting negative documents in the directory mode Internal: Support CIFS drives for the collection data store Version 8.4 Fix Pack 1 Hot Fix 2 addressed the following APARs: --------------------------------------------------------------- IC53077: A synchronization problem occurs when dumping the parserservice javacore from the parserdriver IC53365: The parser does not parse attached archive files(tar, zip, and so on) correctly IC53157 and IC53635: The document language is not properly determined by the parser IC53566: The Stellent process dumps core when processing large PDF files IC53440: A loop in the initialization of a custom annotator prevents the system from correctly starting IC53482: The weekly crawler scheduler cannot calculate the next crawl date IC53381: Not all document types can be used as criteria for retrieval IC53632: The timestamp column is indexed twice IC53908: The Web crawler cannot crawl a site if a robots.txt is protected by form-based authentication IC53239: Query response times reported in the administration console are incorrect IC53511: Different HTML tag replacement by the parser Internal: Cannot use characters other than alphanumeric in a scope name Version 8.4 Fix Pack 1 Hot Fix 1 addressed the following APARs: --------------------------------------------------------------- IC52876: Users are unable to browse to the next page of results when issuing searches from the category tree page IC52865: ArrayOutOfBoundsException is encountered when retrieving the last page of results when searching multiple secured collections IC52888: "Cannot find bean / links in any scope" error occurs when the user's search session times out IC52991: Documents in the store that contain a corrupted inverted term vector cause the index build to fail IC53081: When you configure a DB2 crawler, error SQL122N, SQLSTATE 55032 occurs IC52926: Preventing a recrawl of pages that return error 404 (not found) IC52964: The Web crawler fails to handle the configuration of form-based authentication if a long parameter is defined IC52934: The parser does not start after reading a field with mixed case characters IC53077: The parserdriver session stops responding IC53015: A race condition causes the parserdriver session to stop abnormally when the parser is stopped IC53026: The ccl.log file contains information reported by code that runs in the server framework IC52786: The xml_mapping.properties file is corrupted when mappings are deleted through the administration console IC53177: The index build fails to complete IC52608: Normalization rules for Arabic content are not correct. Installation instructions for Windows: ----------------------------------------- IMPORTANT: You must stop the enterprise search system completely before starting the installation process. Before you install Version 8.4 Fix Pack 1 Hot Fix 6: 1. Ensure that Version 8.4 Fix Pack 1 is installed on the enterprise search server. The value of a variable called VRCF in %ES_NODE_ROOT%\nodeinfo\bldinfo.txt should be 8.4.0.150 or higher. 2. Ensure that at least 1.9 GB of temporary disk space is available on the disk that is hosting the temporary directory of the installer. This temporary disk space is required to unpack the image before the product is installed. The temporary directory of the installer can be determined by checking the value of the TEMP environment variable. If you cannot have the required space on the desired disk, do one of the following actions: -> Modify the value of the TEMP environment variable to specify a directory path that has the required disk space. -> Run the installation executable (install-win32.exe) with the -is:tempdir argument, where identifies a directory path that has the required disk space. For example: install-win32.exe -is:tempdir Z:\temp In this option, free space in a directory specified by TEMP environment variable should be larger than 700 MB and that of a directory specified by -is:tempdir argument should be larger than 1.2 GB 3. Ensure that at least 800 MB of free disk space is available on the disk that is hosting the %ES_INSTALL_ROOT% directory. 4. Ensure that at least 100 MB of free disk space is available on the disk that is hosting the \wstemp directory of the WebSphere Application Server. 5. Installation of hot fix replaces files below, files in directories and in sub-directories of them %ES_NODE_ROOT%\nodeinfo %ES_INSTALL_ROOT%\installedAppls %ES_INSTALL_ROOT%\configurations\interfaces %ES_INSTALL_ROOT%\configurations\parserservice\jediidata %ES_INSTALL_ROOT%\configurations\ccl.properties All changes you made on these files will be overwritten and need to be set again. If you have updated these files by manual, please make a backup copy of those files and apply same changes on files after installing this hot fix. Installing Version 8.4 Fix Pack 1 Hot Fix 6: 1. On the index server, log in as the enterprise search administrator. 2. At the command prompt, enter "esadmin stop" to stop the enterprise search system. 3. At the command prompt, enter "stopccl" to stop the Common Communication Layer. 4. At the command prompt, enter "startccl" to restart the Common Communication Layer. 5. Log in as a Windows administrator to install the hot fix. 6. As a Windows administrator, run the executable install-win32.exe to install the hot fix. Note: The installation program prompts you for the "data directory" of the current installation. Be sure to specify the correct value. 7. For a multiple server installation, run steps 3 - 6 on all of your enterprise search servers. Note: If security is enabled on the search servers, the program prompts you for the WebSphere Application Server global security user ID and password. 8. On all servers, log in as the enterprise search administrator and enter "startccl" to start the Common Communicator Layer. 9. On the index server, enter "esadmin start" to start the enterprise search system. Note: After the search servers are restarted, ensure that the application servers 'server1' and 'ESSearchServer' are running in WebSphere Application Server. After you install Version 8.4 Fix Pack 1 Hot Fix 6: Run the crawler setup scripts again to set up crawler-related configuration files. The scripts make required changes on configuration files that are overridden during installation. For details, see http://www.ibm.com/support/docview.wss?rs=63&uid=swg21264441 Uninstallation instructions for Windows: ------------------------------------------- 1. On the index server, log in as the enterprise search administrator. 2. Change to the uninstall directory, which is normally located at the installation root directory, and run the uninstall program, uninstall.exe. For version 8.4 Fix Pack 1 Hot Fix 6, the location is %ES_INSTALL_ROOT%\_uninst_8.4.0.156 3. The program prompts you to specify what you want to remove: - You can uninstall the most recent hot fix or fix pack (or) - You can uninstall OmniFind Enterprise Edition (WebSphere Information Integrator OmniFind Edition) Note: If security is enabled on a search server, the program prompts you for the WebSphere Application Server global security user ID and password. 4. The results of the uninstallation program are displayed. Review the information and then click 'Finish'. 5. Restart the server. 6. For a multiple server installation, repeat these steps on the other servers. Installation instructions for AIX(R), Linux(R), and Solaris: ----------------------------------------------------------- IMPORTANT: You must stop the enterprise search system completely before starting the installation process. Before you install Version 8.4 Fix Pack 1 Hot Fix 6: 1. Ensure that Version 8.4 Fix Pack 1 is installed on the enterprise search server. The value of a variable called VRCF in $ES_NODE_ROOT/nodeinfo/bldinfo.txt should be 8.4.0.150 or higher. 2. - For the AIX platform, ensure that at least 1.6 GB of temporary disk space is available on the disk that is mounting the /tmp directory. The temporary disk space is required to unpack the image before the product is installed. If you cannot have the required space on the desired disk, run the installation executable with the -is:tempdir argument, where identifies a directory path that is mounted on a disk that has the required disk space. In this option, free space in the /tmp directory should be larger than 700 MB and free space of a directory specified by -is:tempdir argument should be larger than 900 MB - For Linux and Solaris platforms, ensure that at least 1.9 GB of temporary disk space is available on the disk that is mounting the /tmp directory. The temporary disk space is required to unpack the image before the product is installed. If you cannot have the required space on the desired disk, run the installation executable with the -is:tempdir argument, where identifies a directory path that is mounted on a disk that has the required disk space. In this option, free space in the /tmp directory should be larger than 700 MB and free space of a directory specified by -is:tempdir argument should be larger than 1.2 GB 3. Ensure that at least 800 MB of free disk space is available on the disk that is hosting the $ES_INSTALL_ROOT directory. 4. Ensure that at least 100 MB of free disk space is available on the disk that is hosting the /wstemp directory of the WebSphere Application Server. 5. Ensure that the maximum number of files that can be opened by a process (ulimit value) is large enough. See: http://www.ibm.com/support/docview.wss?rs=63&uid=swg21255312 6. Installation of hot fix replaces files below, files in directories and in sub-directories of them $ES_NODE_ROOT/nodeinfo $ES_INSTALL_ROOT/installedAppls $ES_INSTALL_ROOT/configurations/interfaces $ES_INSTALL_ROOT/configurations/parserservice/jediidata $ES_INSTALL_ROOT/configurations/ccl.properties All changes you made on these files will be overwritten and need to be set again. If you have updated these files by manual, please make a backup copy of those files and apply same changes on files after installing this hot fix. Installing Version 8.4 Fix Pack 1 Hot Fix 6: 1. On the index server, log in as the enterprise search administrator. 2. At the command prompt, enter "esadmin stop" to stop the enterprise search system. 3. At the command prompt, enter "stopccl.sh" to stop the Common Communication Layer. 4. At the command prompt, enter "startccl.sh" to restart the Common Communication Layer. 5. Log in as the root user to install the hot fix. 6. As the root user, run the executable to install the hot fix: install-aix.bin, install-linux.bin, or install-solaris.bin. Note: The installation program prompts you for the "data directory" of the current installation. Be sure to specify the correct value. 7. For a multiple server installation, run steps 3 - 6 on all of your enterprise search servers. Note: If security is enabled on the search servers, the program prompts you for the WebSphere Application Server global security user ID and password. 8. On all servers, log in as enterprise search administrator and enter "startccl.sh" to start the Common Communication Layer. 9. On the index server, enter "esadmin start" to start the enterprise search system. Note: After the search servers are restarted, ensure that the application servers 'server1' and 'ESSearchServer' are running in WebSphere Application Server. After you install Version 8.4 Fix Pack 1 Hot Fix 6: Run the crawler setup scripts again to set up crawler-related configuration files. The scripts make required changes on configuration files that are overridden during installation. For details, see http://www.ibm.com/support/docview.wss?rs=63&uid=swg21264441 Uninstallation instructions for AIX, Linux, and Solaris: ------------------------------------------------------------- 1. On the index server, log in as the enterprise search administrator. 2. Change the user to be the root user. 3. Change to the uninstall directory, which is normally located at the installation root directory, and enter "uninstall" to run the uninstall program. For Version 8.4 Fix Pack 1 Hot Fix 6, the location is $ES_INSTALL_ROOT/_uninst_8.4.0.156. 4. The installation program prompts you to specify what you want to remove: - You can uninstall the most recent hot fix or fix pack (or) - You can uninstall OmniFind Enterprise Edition (WebSphere Information Integrator OmniFind Edition) Note: If security is enabled on a search server, the program prompts you for the WebSphere Application Server global security user ID and password. 5. The results of the uninstallation program are displayed. Review the information and then click 'Finish'. 6. For a multiple server installation, repeat these steps on the other servers. Note: In the installation summary page, 700MB of free space in /tmp (UNIX platforms) or in the %TEMP% directory (Windows platform) and 800MB of free space in $ES_INSTALL_ROOT (%ES_INSTALL_ROOT%) directory are requested. Before this message is displayed, /tmp directory or %TEMP% directory or a directory specified by -is:tempdir option is used for expanding files in the installation package. This message requests it needs more 700MB in /tmp or %TEMP% directory for the remainder of the installation process. In total, installation needs 1.5GB free space for AIX and 1.9GB for other platforms in the /tmp or %TEMP% directory, as described above. Known problems of installation/uninstallation from a console: ------------------------------------------------------------- Problem: When WebSphere Application Server 6.1 is installed on a Linux machine, the default location of the WebSphere Application Server plug-in is displayed incorrectly. Workaround: The installation program shows "/opt/IBM/WebSphere/Plugins" as the IBM HTTP Server and plug-in directory, but the correct value is "/opt/IBM/HTTPServer/Plugins". Be sure to specify the correct value for the installation directory of the IBM HTTP Server. Problem: After uninstalling Version 8.4 Fix Pack 1 Hot Fix 6 from a machine on which the WebSphere Application Server 6.1 is installed, the search server (ESSearchServer) is not accessible. Workaround: Run the WebSphere Application Server setup script manually. Enter "/opt/IBM/WebSphere/AppServer/bin/ws_ant.sh -DWSConnType=NONE -DOFDir=/opt/IBM/es/installedApps -DWSNDProfile Dir= -DWSNDHostName= -Duser=root -Dpassword=XXXX -DwebServerName=webserver1 -DwebServerNodeName=rhas4u2Node01 -DmemConf=medium -buildfile was60_install.xml install" where "/opt/IBM/WebSphere" should be replaced by the installation directory of WebSphere Application Server in your environment and "/opt/IBM/es" should be replaced by $ES_INSTALL_ROOT. For "-Dpassword=XXX", a password of the root account should be supplied. Problem: On the Linux platform, available free space displayed in the installation summary page is twice as large as the actual size. Workaround: Ensure that enough free space, as described above, is available before you start the installation program. Special instructions: ---------------------- To resolve problems addressed by the APARs listed below, you must re-crawl and rebuild a searchable index. If you do not have the problems addressed by these APARs, a re-crawl nor rebuild is not required. New APARs in this hot fix: IC55387, IC55536 APARs from previous hot fixes: IC54789, IC54876, IC54906, IC55020, IC54359, IC54444, IC54535, IC53777, IC53834, IC53980, IC54359, IC53365, IC53632, IC52964, IC53511, IC52608. Notice: ------- Do not install WebSphere(R) Application Server, Version 6.1 Fix Pack 15. If you install this fix pack, the enterprise search administration console becomes unusable. Mapping collection data to CIFS drives: --------------------------------------- On Windows, you can configure the system to use Common Internet File System (CIFS) network drives to store enterprise search collection data. To divide collection data among multiple locations, you can create an environment variable for enterprise search and map the shared CIFS network drives that you want to use. - Ensure that the enterprise search administrator ID can access the network drives. The network drives must be accessible even when the enterprise search administrator is not interactively logged in. - In a multiple server installation, different enterprise search servers must use same network drive letter but map to different physical locations. You cannot map to the same physical location from different enterprise search servers. - The number of available mapped drives that you configure is limited by your operating system. - The mapped drive letter should not already exist. That is, do not use a drive letter that is already used by a physical device, such a high-density drive or CD-ROM drive. - The drives that you map for enterprise search are not displayed when the enterprise search administrator is logged in. - The Universal Naming Convention (UNC) path cannot end with a backward slash (\) character. To map CIFS network drives for storing collection data, do the following steps after you install enterprise search. In a multiple server configuration, repeat these steps on all enterprise search servers. 1. Create the following system environment variable: ES_MAP_DRIVE=drive_letter:UNC_path [;drive_letter:UNC_path ...] Where: drive_letter Specifies the network drive letter to be mapped. UNC_path Specifies the path to be used for storing collection data, such as \\jkenterprises\directory1. To map multiple network drives, type a semicolon (;) followed by the additional drive letter and UNC path statements. The following example shows two directory paths mapped to two drive letters: ES_MAP_DRIVE=T:\\jkenterprises1\esdata;U:\\jkenterprises2\esdata 2. Restart the computer. Examples: 1. Correct: Different servers map to different physical locations - Index server: ES_MAP_DRIVE=T:\\server1\dir1 - Search server: ES_MAP_DRIVE=T:\\server1\dir2 2. Correct: Different servers map to the same top-level path but different subdirectories - Index server: ES_MAP_DRIVE=T:\\server1\dir1\esdata1 - Search server: ES_MAP_DRIVE=T:\\server1\dir1\esdata2 3. Incorrect: Different servers map to the same physical location - Index server: ES_MAP_DRIVE=T:\\server1\dir1 - Search server: ES_MAP_DRIVE=T:\\server1\dir1 4. Incorrect: Different servers share the same physical path - Index server: ES_MAP_DRIVE=T:\\server1\dir1 - Search server: ES_MAP_DRIVE=T:\\server1\dir1\dir2 Limitation: 1. For some CIFS drives, depending on the version of Windows of a CIFS server or the version of Network Attached Storage (NAS), this function does not work. 2. The log of dropped documents cannot be seen from the administration console. To see the contents of the log: 1) On the index server or the search server, log in as the enterprise search administrator. 2) Map network drives in the same mapping as specified by the ES_MAP_DRIVE environment variable. 3) Type esviewlog -f //dropped_doc_logs/dropped_doc__.log Related technotes: ------------------ To learn about technotes and other information, go to the OmniFind Enterprise Edition Support site, http://www.ibm.com/software/data/enterprise-search/omnifind-enterprise/support.html, and search for technotes and APARs.