MySQLDataWriter

Short Description
Ports
Metadata
MySQLDataWriter Attributes
Details
See also

Short Description

MySQLDataWriter is a high-speed MySQL table loader. It uses MySQL native client.

Component Data output Input ports Output ports Transformation Transf. required Java CTL Auto-propagated metadata
MySQLDataWriterdatabase0-10-1
no
no
no
no
no

Ports

Port typeNumberRequiredDescriptionMetadata
Input0 [1] Records to be loaded into the databaseAny
Output0
no
For information about incorrect recordsError Metadata for MySQLDataWriter

[1]  If no file containing data for loading (Loader input file) is specified, input port must be connected.

Metadata

MySQLDataWriter does not propagate metadata.

Error Metadata cannot use Autofilling Functions.

Table 56.5. Error Metadata for MySQLDataWriter

Field numberField nameData typeDescription
0<any_name1>integerThe number of incorrect record (records are numbered starting from 1).
1<any_name2>integerThe number of incorrect column.
2<any_name3>stringError message

MySQLDataWriter Attributes

AttributeReqDescriptionPossible values
Basic
The path to MySQL utilityyesThe name of MySQL utility, including the path. Must be installed and configured on the same machine where CloverDX runs. The MySQL command line tool must be available.  
Host Host where the database server is located.localhost (default) | other host
DatabaseyesThe name of the database into which the records should be loaded. 
Database tableyesThe name of the database table into which the records should be loaded. 
User nameyesDatabase user. 
PasswordyesThe password for a database user. 
Advanced
Path to control script The name of a command file containing the LOAD DATA INFILE statement, including the path. For more information, see Path to Control Script. 
Lock database table  By default, a database is not locked and multiuser access is allowed. If set to true, the database table is locked to ensure exclusive access and possibly faster loading. false (default) | true
Ignore rows  The number of rows of a data file that should be skipped. By default, no records are skipped. Valid only for input file with data. 0 (default) | 1-N
Column delimiter The delimiter used for each column in the data. Field values must not include this delimiter as their part. By default, a tabulator is used. \t (default) | other character
Loader input file [1] The name of an input file to be loaded, including path. For more information, see Loader input file.  
Parameters All parameters that can be used as parameters by the load method. These values are contained in a sequence of pairs of the following form: key=value, or key only (if the key value is the boolean true) separated from each other by a semicolon, colon, or pipe. If the value of any parameter contains the delimiter as its part, such value must be double quoted.  

[1]  If the input port is not connected, Loader input file must be specified and contain data. For more information, see Loader input file.

Details

MySQLDataWriter loads data into a database table using native MySQL database client.

It reads data either from the input port or from an input file.

You can attach the optional output port and read records which have been reported as rejected.

Reading from input port (input port connected) dumps the data into a temporary file which is then used by mysql utility. You can set the temporary file explicitly by setting the Loader input file attribute or leave it blank to use default.

Reading from a file (no input connected) uses the Loader input file attribute as a path to your data file. The attribute is mandatory in this case. The file needs to be in a format recognized by the mysql utility (see MySQL LOAD DATA).

This component executes the MySQL native command-line client (bin/mysql or bin/mysql.exe). The client must be installed on the same machine as the graph is running on.

MySQLDataWriter is a bulk loader suitable for uploading many records to database. To insert several records, you can also use DBOutputTable, which does not require mysql utility.

Path to Control Script

The name of a command file containing the LOAD DATA INFILE statement (See MySQL LOAD DATA), including path.

  • If it is not set, the command file is created in CloverDX temporary directory and it is deleted after the load finishes.

  • If it is set, but the specified command file does not exist, a temporary command file is created with the specified name and path and it is not deleted after the load finishes.

  • If it is set and the specified command file exists, this file is used instead of command file created by CloverDX.

Loader input file

The name of an input file to be loaded, including the path.

  • If it is not set, a loader file is created in CloverDX or OS temporary directory (on Windows) or stdin is used instead of temporary file (on Unix). The file is deleted after the load finishes.

  • If it is set, a specified file is created. It is not deleted after data is loaded and it is overwritten on each graph run.

  • If the input port is not connected, this file must be specified, must exist and must contain data that should be loaded into the database. The file is not deleted nor overwritten.

Notes and Limitations

You should not write maps and lists.

See also

DBOutputTable
Common Properties of Components
Specific Attribute Types
Common Properties of Writers
Writers Comparison