FTP List Files activity

During run time, the FTP List Files activity connects to an FTP Server and generates a list of files from the specified directory location using the specified regular expression. The FTP List Files activity returns the list of files to the orchestration using either the directoryListing or fileListing output parameter.

If the List Only Files (NLST) check box is cleared, a full directory listing including directory permissions, timestamps, and ownership is returned in the directoryListing output parameter as a single string. When a full directory listing is returned as a string, you must parse the directory listing using a Flat File Schema with the Read Flat File activity. If the List Only Files (NLST) check box is selected and the FTP List Files activity supports the format of the directory listing for your FTP Server, a repeating structure of just filenames is returned in the fileListing output parameter. When a repeating structure of filename is returned, you do not need to parse the directory listing and can just iterate through the list using a For Each or While Loop activity.

For some cases, you must use the FTP List Files activity in conjunction with the FTP Get File, FTP Put File, FTP Delete File, and FTP Rename File activities. to process a set of files found in an FTP directory, while in other cases you can just use the FTP Poll Directory activity.

Use the FTP List Files activity in conjunction with the FTP Get File, FTP Put File, FTP Delete File, and FTP Rename File activities, to process a set of files found in an FTP directory if any of the following conditions are true:

Symbolic links

The support for symbolic links by the FTP List Files activity during run time is dependant on the following factors:   To determine the specific symbolic link support for a specific FTP Server on a specific operating system, see your operating system and FTP Server documentation.

Processing subdirectories with the Microsoft FTP Endpoint

For the Microsoft IIS (Internet Information Services) FTP Endpoint, the run time behavior of the FTP List Files activity when processing subdirectories and the files in subdirectories depends on the regular expression specified in the File Named field of the Configure task of the activity, as described in the following cases:

Adding an FTP List Files activity to the orchestration

  1. Create or open an orchestration. A graphical representation of the orchestration is displayed.
  2. Select the Activities tab and expand the FTP folder.
  3. Drag the List Files activity onto the orchestration.
  4. Select the activity. The Checklist is displayed.

Creating, selecting, or editing an FTP endpoint

  1. Click Pick Endpoint task from the Checklist and choose from one of the following actions:
    • Select an existing endpoint:
      1. Select Browse. The Project Explorer is displayed.
        Attention: Specifying a new FTP Endpoint with a different directory structure than the original FTP Endpoint can cause the orchestration job to fail during run time. You must either specify a new FTP Server that matches the directory structure of the originally specified FTP Server or change the configured directory of the In Directory field by clicking Browse as described below in the "Configuring the activity" procedure.
      2. Select an existing endpoint and click OK. Skip to the "Configuring the activity" procedure.
    • Select an existing endpoint and edit the endpoint:
      1. Select Browse. The Project Explorer is displayed.
      2. Select an existing endpoint and click OK.
        Attention: Specifying a new FTP Endpoint with a different directory structure than the original FTP Endpoint can cause the orchestration job to fail during run time. You must either specify a new FTP Server that matches the directory structure of the originally specified FTP Server or change the configured directory of the In Directory field by clicking Browse as described below in the "Configuring the activity" procedure.
      3. Click Edit. The Edit Endpoint pane is displayed.
    • Create a new endpoint: Select New. The Create Endpoint pane is displayed.
  2. Create or edit an FTP endpoint: Creating or Editing an FTP endpoint

Configuring the activity

  1. Select the Configure task from the Checklist. The Configure pane is displayed.
  2. Configure all the fields in this pane as described in the following table:
    Table 2.
    Field Description
    In Directory Specifies the directory on the FTP server where the Integration Appliance deletes the specified file. You can specify a directory using one of the following options:
    • Directly enter directory path by clicking in the field and typing in one of the following directory paths:
      • Relative directory path, for example: /process.  (Assumes that FTP home has been configured for the user account that connects to the FTP Server. See the note below for more information.)
      • Complete directory path, for example on a UNIX machine: /home/user/process.
    • Click Browse... to populate the drown-down list of available directories. Select the appropriate directory from the drop-down list.
    Note: UNIX: If you specify a relative directory path when connecting to an FTP Server on a UNIX machine, verify that the FTP home directory has been correctly specified in the /etc/password file for the user account connecting to the FTP Server.
    File Named Specifies a pattern or regular expression that defines the filenames to get from the FTP Server. The processing of the regular expression is done by the FTP Server. For more information, see the FTP Server documentation on regular expressions. Typically, FTP Servers support the following wildcards in their regular expressions:
    • * - zero or more characters
    • ? - zero or one character
    Note: Because the processing of the regular expression is done on the FTP Server, the case sensitivity is determined by FTP Server. If the FTP Server is case sensitive, the file matching done by the FTP Poll Directory activity is also case sensitive.
    List Only Files (NLST) The List Only Files (NLST) check box determines the type of directory listing returned by the Test button and during run time. In addition, this check box defines if the fileListing or the directoryListing output parameter is displayed in the Map Outputs task and returned by the activity during run time.
      If the List Only Files (NLST) check box is selected, just the filenames are returned when the Test button is clicked, as shown in Figure 1. In addition, when the List Only Files (NLST) check box is selected, the fileListing output parameter is listed in the Map Outputs task. During run time, the set of fileNames are returned in the fileName repeating node of the fileListing output parameter. For this example, the first instance of the fileListing/fileName repeating element is equal to bidAcmeInc.txt, the second the first instance of the fileListing/fileName repeating element is equal to bidBudgetManufacture.txt, etc.
      If the List Only Files (NLST) check box is cleared, a full directory listing including directory permissions, timestamps, and ownership are returned when the Test button is clicked, as shown in Figure 2. In addition, when the List Only Files (NLST) check box is selected, the directoryListing output parameter is listed in the Map Outputs task. During run time, the full directory listing is provided as a single string and returned using the directoryListing output parameter.
    Figure 1. Example listing when List Only Files (NLST) is selected.
    bidAcmeInc.txt
    bidBudgetManufacture.txt
    bidCrankCo.txt
    bidSamAndSons.txt
    bidWidgetRUs.txt
    Figure 2. Example listing when List Only Files (NLST) is cleared.
    -rw-r--r--  1 551    551      65 May  3 11:39 bidAcmeInc.txt
    -rw-r--r--  1 551    551     43 May  3 11:39 bidBudgetManufacture.txt
    -rw-r--r--  1 551    551       0 May  3 11:39 bidCrankCo.txt
    -rw-r--r--  1 551    551      73 May  3 11:39 bidSamAndSons.txt
    -rw-r--r--  1 551    551      33 May  3 11:39 bidWidgetRUs.txt

Specifying the retry options

  1. Select the Retry task from the Checklist. The Retry pane is displayed.
  2. Configure the retry options for connecting to an FTP Server. The following table defines the retry options for a connection to an FTP Server:
    Table 3.
    Field Description
    Retry  
    1) Wait ___ second(s) between each retry. The number of seconds that the Integration Appliance waits before trying to establish a connection to the FTP Server.
    2) Try to connect ___ times before failing. Specifies the retry count - the maximum number of times the Integration Appliance attempts to establish a connection to the FTP Server before issuing an error.

    If an orchestration that starts with an FTP List Files activity is deployed and the Integration Appliance cannot connect to the specified FTP Server, the Integration Appliance logs the connections errors as warnings in the system log until the retry count value is reached. When the retry count is reached, the Integration Appliance logs an error in the system log, resets the connection error count to zero, and continues to attempt to establish a connection to the FTP Server.

    For example, you set the retry count to 3. The first, second, and third connection errors appear in the system log as warnings. The Integration Appliance logs the fourth connection error as an error and resets the connection error count to zero. Therefore, the fifth connection error generates a warning in the system log. The Integration Appliance continues to attempt to establish a connection to the FTP Server.

Mapping the input of the activity

  1. Select the Map Inputs task in the Checklist. The Retry pane is displayed. The input parameters of the activity are displayed in the To Activity pane.
  2. It is optional to map the input parameters of the FTP List Files activity. The FTP List Files activity has the following optional input parameters:    
    • directory
    • connection

    If the input parameter listed in the following table are not displayed in the To Activity pane of the Map Inputs pane, select Map > Show Optional Parameters from the toolbar menu or right-click in the To Activity pane and select the Show Optional Parameters option. If the Map > Show Optional Parameters option is not selectable in the toolbar menu, click on a node in the Map Inputs pane to make this option active.

    You can optionally override the settings you specified in the Configure and Pick Endpoint task of the Checklist dynamically during run time, using the input parameters defined in the following table.

    For example, the directory on the FTP Server where the Integration Appliance gets the list of files, could be set in the Configure task to the directory called input. During run time, the directory name: inputXML could be passed into the activity using the directory input parameter. The directory name specified dynamically during run time overrides the original setting specified in the Configure pane. In this example, the list of files is generated from the inputXML directory.

Table 4.
Input Parameter Name/Node Description Overrides the Setting in Task?
directory Specifies the directory on the FTP server where the Integration Appliance places the file generated in this activity. The directory input parameter overrides the In Directory field of the Configure task.
connection/timeout Specifies the amount of time in seconds before a timeout occurs between the Integration Appliance attempting to establish a connection to the endpoint and the time it takes for the endpoint to respond to the connection request. For the FTP Delete File activity, the amount of time the Integration Appliance attempts to establish a connection to the FTP Server before timing out.   The timeout node of the connection input parameter overrides the Connection Timeout -Time out after ___ second(s) when establishing a connection to the Endpoint field of the FTP endpoint.  
connection/host Specifies the Host Name or IP address for the FTP server. The host node of the connection input parameter overrides the Host Name field of the FTP endpoint.
connection/port Specifies the port number for the specified host name. The standard port for FTP traffic is 21. The default port for SFTP traffic is 22. The port node of the connection input parameter overrides the Port field of the FTP endpoint.
connection/username Specify a user name for the account used to connect to the FTP Server. The username node of the connection input parameter overrides the User Name field of the FTP endpoint.
connection/password Specify a password for the account used to connect to the FTP Server. The password node of the connection input parameter overrides the Password field of the FTP endpoint.

Mapping the output of the activity

  1. Select the Map Outputs task in the Checklist. The directoryListing or fileListing output parameters is displayed in the From Activity pane.
  2. Map the directoryListing or fileListing output parameter to a variable. See Creating a map for general instructions on mapping.

During run time, the FTP List Files activity returns the list of files to the orchestration using the directoryListing or fileListing output parameter.

If the List Only Files (NLST) check box is cleared in the Configure task, a full directory listing including directory permissions, timestamps, and ownership is returned in the directoryListing output parameter as a single string. When a full directory listing is returned as a string, you must parse the directory listing using a Flat File Schema with the Read Flat File activity. You can create a Flat File Schema using the Flat File Schema Editor. For an example orchestration parsing a directory listing using a Flat File Schema, see the Parse Directory Listing Example: Parse directory listing example.

If the List Only Files (NLST) check box is selected and the FTP List Files activity supports the format of the directory listing for your FTP Server, a repeating structure of just filenames is returned in the fileListing output parameter. When a repeating structure of filename is returned, you do not need to parse the directory listing and can just iterate through the list using a For Each or While Loop activity.