SystemExecute

Port type	Number	Required	Description	Metadata
Input	0	⨯	For standard input of the specified system command (input of the process)	Any1
Output	0	^[1]	For a standard output of the specified system command (output of the process)	Any2

Port type

Number

Required

Description

Metadata

Input

⨯

For standard input of the specified system command (input of the process)

Any1

Output

^[1]

For a standard output of the specified system command (output of the process)

Any2

¹	A standard output must be written to an output port or output file. If both an output port is connected and output file is specified, an output is sent to output port only.

Metadata

SystemExecute does not propagate metadata.

SystemExecute has no metadata template.

SystemExecute Attributes

Attribute Req Description Possible values

Attribute	Req	Description	Possible values
Basic
System command	yes	A command to be executed by the system. The command is always saved to a tmp file as a script. In case an interpreter is specified, it executes that script. If the command requires an input, it has to be sent to the command through the optional input port. For details, see How it works.
Process input/output charset		Encoding used for formatting/parsing data for input/from output of system process. The default encoding depends on DEFAULT_CHARSET_DECODER in defaultProperties.	UTF-8 \| <any other>
Output file URL	^[1]	Name of a file, including the path, to which output of the process (together with errors) should be written if the output edge is not connected and if System command creates the standard output. For more information, see URL File Dialog.
Append		By default, the contents of an output file is always deleted and overwritten by new data. If set to `true`, new output data is appended to the output file without deleting the file contents.	false (default) \| true
Command interpreter		An interpreter that should execute the command. If specified, System command is saved as a script to a temporary file and executed by this interpreter. Its form must be the following: `<interpreter name> [parameters] ${} [parameters]`. If an interpreter is defined, System command is saved to a temporary file and executed as a script. In such a case, the component replaces this `${}` expression by the name of this temporary script file. See the list of well known interprets.
Working directory		A working directory of the component.	current directory^[2] (default) \| other directory
Advanced
Number of error lines		The number of lines that are printed if a command finishes with errors.	2 (default) \| 1-N
Environment		System-dependent mapping from variables to values. Mappings are separated by a colon, semicolon or pipe. By default, the new value is appended to the environment of the current process. Both `PATH=/home/user/mydir` and `PATH=/home/user/mydir!true` means that `/home/user/mydir` will be appended to the existing `PATH`. Whereas `PATH=/home/user/mydir!false` replaces the old `PATH` value by the new one (`/home/user/mydir`).	For example: PATH=/home/user/mydir[!true] (default) \| PATH=/home/user/mydir!false
Timeout for producer/consumer workers (ms)		Timeout; by default in milliseconds, but other time units may be used. For details, see Timeout.	0 (without limitation) \| 1-n
Ignore exit value		In case the executed system command returns non-zero value, the component fails. This option can change this behavior, the exit value can be ignored.	true \| false (default)

Basic

System command

yes

A command to be executed by the system. The command is always saved to a tmp file as a script. In case an interpreter is specified, it executes that script. If the command requires an input, it has to be sent to the command through the optional input port. For details, see How it works.

Process input/output charset

Encoding used for formatting/parsing data for input/from output of system process.

The default encoding depends on DEFAULT_CHARSET_DECODER in defaultProperties.

UTF-8 | <any other>

Output file URL

^[1]

Name of a file, including the path, to which output of the process (together with errors) should be written if the output edge is not connected and if System command creates the standard output. For more information, see URL File Dialog.

Append

By default, the contents of an output file is always deleted and overwritten by new data. If set to true, new output data is appended to the output file without deleting the file contents.

false (default) | true

Command interpreter

An interpreter that should execute the command. If specified, System command is saved as a script to a temporary file and executed by this interpreter. Its form must be the following: <interpreter name> [parameters] ${} [parameters]. If an interpreter is defined, System command is saved to a temporary file and executed as a script. In such a case, the component replaces this ${} expression by the name of this temporary script file.

See the list of well known interprets.

Working directory

A working directory of the component.

current directory^[2] (default) | other directory

Advanced

Number of error lines

The number of lines that are printed if a command finishes with errors.

2 (default) | 1-N

Environment

System-dependent mapping from variables to values. Mappings are separated by a colon, semicolon or pipe. By default, the new value is appended to the environment of the current process. Both PATH=/home/user/mydir and PATH=/home/user/mydir!true means that /home/user/mydir will be appended to the existing PATH. Whereas PATH=/home/user/mydir!false replaces the old PATH value by the new one (/home/user/mydir).

For example: PATH=/home/user/mydir[!true] (default) | PATH=/home/user/mydir!false

Timeout for producer/consumer workers (ms)

Timeout; by default in milliseconds, but other time units may be used. For details, see Timeout.

0 (without limitation) | 1-n

Ignore exit value

In case the executed system command returns non-zero value, the component fails. This option can change this behavior, the exit value can be ignored.

true | false (default)

¹	If the output port is not connected, a standard output can only be written to a specified output file. If the output port is connected, an output file will not be created and a standard output will be sent to the output port.

A working directory defaults to current process working directory only if the graph runs on a local CloverDX engine. If it runs on the CloverDX Server, then the default working directory is the sandbox root. In the case of a Clustered Server environment, the default working directory is the sandbox root only if the sandbox is of a "shared" type, or it is of a "local" type and is located on the node running the SystemExecute component. Otherwise the Server node cannot access the sandbox locally, i.e. the sandbox is of type "local" but is located on a remote node, in which case the working directory defaults to some arbitrary temporary directory supplied by the Temp Space Management subsystem of the Server node. The component prints a message about which working directory was actually used into a graph run log at INFO log level.

Details

SystemExecute executes commands and arguments specified in the component itself as a separate process. The commands receive a standard input through the input port and send a standard output to the output port (if the command creates any output).

How it works

SystemExecute runs the command specified in the System command and creates two threads.

The first thread (producer) reads records from the input edge, serializes them and sends them to stdin of the command.
The second thread (consumer) reads stdout of the command, parses it and sends it to the output edge.

Timeout

When the command ends, the component still waits until both the producer and the consumer also finish their work. The time is defined in the Timeout attribute.
By default, timeout is unlimited. In the case of an unexpected deadlock, you can set the timeout to any number of milliseconds.

Well known command interprets

The following list contains possible interprets that can be used in SystemExecute component. You are not limited to the items from this list.

Windows

cmd /c ${}
powershell ${}

Linux

/bin/sh ${}
/bin/bash ${}
/bin/tcsh ${}
/usr/bin/perl ${}
/usr/bin/python ${}

Difference between SystemExecute and ExecuteScript

SystemExecute uses a data-oriented approach. It allows you to stream data from and to a script.

ExecuteScript executes scripts in steps. It allows you to receive scripts from an input edge and execute them one by one.

Examples

Run command on Linux

Run command on Windows

Run command that saves its output to file

Calling external filter

Run command on Linux

This example shows execution of command on Linux using SystemExecute.

Run the uptime command using the SystemExecute component. Send the results through an edge to the next component.

Solution

Connect the output port of SystemExecute with the next component. Metadata of the edge should contain just one string field.

Configure SystemExecute.

Attribute	Value
System command	uptime
Command interpreter	/bin/bash ${}

Attribute

Value

System command

uptime

Command interpreter

/bin/bash ${}

Run command on Windows

This example shows execution of command on MS Windows using SystemExecute

Run tasklist using SystemExecute component. Send the results trough an edge to the next component.

Solution

Connect the output port of SystemExecute with the next component. Metadata of the edge should contain just one string field.

Configure SystemExecute.

Attribute	Value
System command	tasklist
Process input/output charset	windows-1252
Command interpreter	cmd /c ${}

Attribute

Value

System command

tasklist

Process input/output charset

windows-1252

Command interpreter

cmd /c ${}

If process input/output charset is not set or it is set incorrectly, you can encounter the following error message in the graph log: Character decoding error occurred. Set correct charset. Current charset is UTF-8.

Run command that saves its output to file

This example shows the way to run a command with SystemExecute and save the output from the command to a file.

Run command who and save its output to the file logged_users.txt for further processing.

Solution

Attribute	Value
System command	who
Output file URL	${DATAOUT_DIR}/logged_users.txt
Command interpreter	/bin/bash ${}

Attribute

Value

System command

who

Output file URL

${DATAOUT_DIR}/logged_users.txt

Command interpreter

/bin/bash ${}

Calling external filter

This example shows the way to use an external shell filter from SystemExecute: the data from a graph is sent to input stream of the filter and the output of the script is sent back to the graph.

Call sed to replace A with B.

Solution

Connect the input port of SystemExecute with the component producing the data and the output port of SystemExecute with the component consuming the data. Assign metadata to the edges. The metadata on both edges should be the same, unless the script changes the number of columns or delimiters.

Attribute	Value
System command	sed 's_A_B_g'
Command interpreter	/bin/bash ${}

Attribute

Value

System command

sed 's_A_B_g'

Command interpreter

/bin/bash ${}

Best Practices

We recommend users to explicitly specify Process input/output charset.

SystemExecute

Short Description

Ports

Metadata

SystemExecute Attributes

Details

How it works

Timeout

Well known command interprets

Windows

Linux

Difference between SystemExecute and ExecuteScript

Examples

Run command on Linux

Solution

Run command on Windows

Solution

Run command that saves its output to file

Solution

Calling external filter

Solution

Best Practices

See also