Version

    ListFiles

    ListFiles 64x64

    Short Description

    Ports

    Metadata

    ListFiles Attributes

    Details

    Examples

    Compatibility

    See also

    Short Description

    ListFiles lists directory contents including detailed information about individual files, e.g. size or modification date. Subdirectories may be listed recursively.

    Inputs Outputs Auto-propagated metadata

    0-1

    1-2

    Ports

    Port type Number Required Description Metadata

    Input

    0

    Input data records to be mapped to component attributes.

    Any

    Output

    0

    One record per each entry in the target directory

    Any

    1

    Errors

    Any

    Metadata

    ListFiles does not propagate metadata from left to right or from right to left.

    The component has metadata templates available. See Details or general info on metadata templates.

    ListFiles Attributes

    Attribute Req Description Possible values

    Basic

    File URL

    yes[1]

    The path to the file or directory to be listed (see Supported URL Formats for File Operations).

    Recursive

    no

    List subdirectories recursively. (Has no effect if List directory contents is set to false.)

    false (default) | true

    Input mapping

    [2]

    Defines mapping of input records to component attributes.

    Output mapping

    [2]

    Defines mapping of results to the standard output port.

    Error mapping

    [2]

    Defines mapping of errors to the error output port.

    Redirect error output

    no

    If enabled, errors will be sent to the standard outputport instead of the error port.

    false (default) | true

    Advanced

    List directory contents

    no

    If set to false, returns information about the directory, not its contents.

    true (default) | false

    Stop processing on fail

    no

    By default, a failure causes the component to skip all subsequent input records and send the information about skipped input records to the error output port. This behavior can be turned off by this attribute. Note: this function works only if an edge is connected to the component’s error port.

    true (default) | false

    1

     The attribute is required, unless specified in the Input mapping.

    2

     Required if the corresponding edge is connected.

    Details

    Editing any of the Input, Output or Error mapping opens the Transform Editor.

    Input mapping

    The editor allows you to override selected attributes of the component with the values of the input fields.

    Field Name Attribute Type Possible values

    fileURL

    File URL

    string

    recursive

    Recursive

    boolean

    true | false

    Output mapping

    The editor allows you to map the results and the input data to the output port.

    If output mapping is empty, fields of input record and result record are mapped to output by name.

    Field Name Type Description

    URL

    string

    URL of the file or directory.

    name

    string

    File name.

    canRead

    boolean

    True if the file can be read.

    canWrite

    boolean

    True if the file can be modified.

    canExecute

    boolean

    True if the file can be executed.

    isDirectory

    boolean

    True if the file exists and is a directory.

    isFile

    boolean

    True if the file exists and is a regular file.

    isHidden

    boolean

    True if the file is hidden.

    lastModified

    date

    The time that the file was last modified.

    size

    long

    True size of the file in bytes.

    result

    boolean

    True if the operation has succeeded (can be false when Redirect error output is enabled).

    errorMessage

    string

    If the operation has failed, the field contains the error message (used when Redirect error output is enabled).

    stackTrace

    string

    If the operation has failed, the field contains the stack trace of the error (used when Redirect error output is enabled).

    Error mapping

    The editor allows you to map the errors and the input data to the error port.

    If Error mapping is empty, fields of input record and result record are mapped to output by name.

    Field Name Type Description

    result

    boolean

    Will always be set to false.

    errorMessage

    string

    The error message.

    stackTrace

    string

    The stack trace of the error.

    Listing Archive Files

    When listing archive files with the archive URL (see Supported URL Formats for File Operations), the Recursive and List directory contents attributes apply to the archive content. Without archive URL, the Recursive and List directory contents attributes do not descend into archive files.

    Archive URL of the zip archive file zip:(sandbox://data/archive.zip) will list zip file content according to the Recursive and List directory contents attribute settings. Non-archive URLs like sandbox://data/archive.zip or file://folder/* (when it encounters an archive file) will always list only the archive file itself.

    That applies to the nested archives too. Archive URL of the inner tar archive file tar:(zip:(sandbox://data/archive.zip)#innerarchive.tar) will list tar file content according to the Recursive and List directory contents attribute settings. Non-archive URLs of the inner file like zip:(sandbox://data/archive.zip)#innerarchive.tar or zip:(sandbox://data/archive.zip)#folder/* (when it encounters an inner archive file) will always list only the inner archive file itself.

    Some archives may not have directory entries - they have only file entries with full paths. In these archives the listing of directory itself will not work.
    Nested Path Separator

    When listing nested archives, output field URL has archive content paths separated by ! character instead of # character, except for innermost file path. Example of the returned URL format is:

    • tar:(zip:(tgz:(outerfile.tar.gz)!innerfile.zip)!innerfile.tar)#innermostfile.txt

    In File URL attribute you can use both formats of URL, with ! or with # in all paths:

    • tar:(zip:(tgz:(outerfile.tar.gz)#innerfile.zip)#innerfile.tar)#innermostfile.txt

    • tar:(zip:(tgz:(outerfile.tar.gz)!innerfile.zip)!innerfile.tar)#innermostfile.txt

    Examples

    Listing Files

    Working with directories

    Listing Files

    List files in ${DATATMP_DIR} directory. Do not list content of subdirectories.

    Solution

    Use File URL and Input mapping attributes.

    Attribute Value

    File URL

    ${DATATMP_DIR}

    Input mapping

    See the code below.

    function integer transform() {
        $out.0.* = $in.1.*;
    
        return ALL;
    }

    You need an edge connected to the first output port.

    Working with directories

    We have the following directory tree structure:

    listfiles directory tree
    1. Check if the Clients directory exists.

    2. List the contents of the Clients directory.

    3. List the contents of any existing subdirectories in the Clients directory.

    Solution

    Use the File URL, Recursive and List directory contents attributes.

    Attribute Value

    File URL

    ${DATAIN_DIR}/Clients/

    List directory contents

    false

    Recursive

    false (default)

    Enable debugging on edges and run the graph. The graph has confirmed the directory exists:

    listfiles directories 01

    Next, set the List directory contents to true (default) and run the graph again. The graph has now listed the contents of the Clients directory:

    listfiles directories 02

    Now, set the Recursive attribute to true and run the graph.

    The graph has now listed the contents of the Clients directory and all subdirectories:

    listfiles directories 03

    Compatibility

    Version Compatibility Notice

    4.6.0-M1

    The List directory contents attribute has been implemented.

    6.3.0

    The support for listing archive content has been implemented.