Version

    InfobrightBulkWriter

    InfobrightDataWriter 64x64

    Short Summary

    Ports

    Metadata

    InfobrightBulkWriter Attributes

    Details

    Best Practices

    Compatibility

    See also

    Short Summary

    InfobrightBulkWriter loads data into an Infobright database.

    When to use InfobrightBulkWriter

    This component requires installation and configuration of a database native client. The client must be running on the same machine as the database server. Due to this overhead, we recommend using this component only if you require significantly higher loading performance than with DatabaseWriter which should be used in typical scenarios.

    Component Data output Input ports Output ports Transformation Transf. required Java CTL Auto-propagated metadata

    InfobrightBulkWriter

    database

    1

    0-1

    Ports

    Port type Number Required Description Metadata

    Input

    0

    Records to be loaded into the database

    Any

    Output

    0

    For records as they were loaded into the database

    A corresponding part of metadata on Input 0[1]

    1

     Only mapped Clover field values can be sent out through the optional output port. A comma must be set as a delimiter for each field, System.getProperty("line.separator") ("\n" for Unix, "\r\n" for Windows) must be set as a record delimiter. Date fields must strictly have the yyyy-MM-dd format for dates and the yyyy-MM-dd HH:mm:ss format for dates with time.

    Metadata

    InfobrightBulkWriter does not propagate metadata.

    It has no metadata template.

    InfobrightBulkWriter Attributes

    Attribute Req Description Possible values

    Basic

    DB connection

    yes

    The ID of the DB connection object to access the database.

    Database table

    yes

    The name of the DB table where data will be loaded.

    Charset

    Charset used for encoding string values to VAR, VARCHAR column types.

    The default encoding depends on DEFAULT_CHARSET_DECODER in defaultProperties.

    UTF-8 | other encoding

    Data format

    bh_dataformat supported by Infobright. Options are: txt_variable or binary (this option is faster, but works with IEE only).

    Text (default) | Binary

    Advanced

    Clover fields

    A sequence of Clover fields separated by a semicolon. Only Clover fields listed in this attribute will be loaded into database columns. The position of both Clover field and database column will be the same. Their number should equal to the number of database columns.

    Log file

    A file for records loaded into database, including the path. If this attribute is specified, no data goes to the output port even if it is connected.

    Append data to log file

    By default, new records overwrite the older ones. If set to true, new records are appended to the older ones.

    false (default) | true

    Execution timeout

    Timeout for the load command (in seconds). Has effect only on the Windows platform.

    15 (default) | 1-N

    Check string’s and binary’s sizes

    By default, sizes are not checked before data is passed to the database. If set to true, sizes are checked - should be set to true if debugging is supported.

    false (default) | true

    Remote agent port

    A port to be used when connecting to the server.

    5555 (default) | otherportnumber

    Details

    InfobrightBulkWriter loads data into an Infobright database. Only the root user can insert data into the database with this component. To run this component on Windows, infobright_jni.dll must be present in the Java library path.

    If the hostname is localhost or 127.0.0.1, the load will be done using a local pipe. Otherwise, it will use a remote pipe. The external IP address of the server is not recognized as a local server.

    For loading to a remote server, you need to start the Infobright remote load agent on the server where Infobright is running. This should be done by executing the command java -jar infobright-core-3.0-remote.jar [-p PortNumber] [-l all | debug | error | info]. The output can be redirected to a log file. By default, the server is listening at port 5555. The infobright-core-3.0-remote.jar is distributed with CloverDX.

    By default, root is only allowed to connect from localhost. You need to add an additional user root@% to connect from other hosts. It is recommended to create a different user (not root) for loading data. The user requires the FILE privilege in order to be able to load data or use the connector:

    grant FILE on *.* to 'user'@'%';
    Notes and Limitations

    Writing maps and lists is not supported as database has no lists and maps.

    Best Practices

    We recommend users to explicitly specify Charset.

    Compatibility

    Version Compatibility Notice

    5.3.0

    InfobrightDataWriter was renamed to InfobrightBulkWriter.