Version

    MSSQLBulkWriter

    Short Description
    Ports
    Metadata
    MSSQLBulkWriter Attributes
    Details
    Examples
    Compatibility
    See also

    Short Description

    MSSQLBulkWriter loads data into MSSQL database.

    [Important]When to use MSSQLBulkWriter

    This component requires installation and configuration of a database native client. The client must be running on the same machine as the database server. Due to this overhead, we recommend using this component only if you require significantly higher loading performance than with DatabaseWriter which should be used in typical scenarios.

    ComponentData outputInput portsOutput portsTransformationTransf. requiredJavaCTLAuto-propagated metadata
    MSSQLBulkWriterdatabase0-10-1⨯⨯⨯⨯⨯

    Ports

    Port typeNumberRequiredDescriptionMetadata
    Input0[1]Records to be loaded into the databaseAny
    Output0⨯For information about incorrect recordsInput 0 (plus three Error Fields for MSSQLBulkWriter

    [1]  If no file containing data for loading (Loader input file) is specified, the input port must be connected.

    This component has one optional input port and one optional output port.

    Metadata

    MSSQLBulkWriter does not propagate metadata.

    Metadata on the output port 0 contains three additional fields at their end: number of incorrect row, number of incorrect column, error message.

    Table 56.5. Error Fields for MSSQLBulkWriter

    Field numberField nameData typeDescription
    LastInputField + 1<anyname1>integerThe number of the incorrect row.
    LastInputField + 2<anyname2>integerThe number of the incorrect column.
    LastInputField + 3<anyname3>stringThe error message.

    MSSQLBulkWriter Attributes

    AttributeReqDescriptionPossible values
    Basic
    Path to bcp utilityyes

    The name of the bcp utility, including path. SQL Server Client Connectivity Components must be installed and configured on the same machine where CloverDX runs. The bcp command line tool must be available.

    		 
    DatabaseyesThe name of the database where the destination table or view resides. 
    Server name 

    The name of the server to which bcp utility should connect.

    If the bcp utility connects to local named or remote named instance of server, Server name should be set to serverName\instanceName.

    If the bcp utility connects to a local default or remote default instance of server, Server name should be set to serverName. If it is not set, bcp connects to the local default instance of server on localhost. The same meaning is true for the serverName which can be set in Parameters. However, if both Server name attribute and the serverName parameter are set, serverName in Parameters is ignored.

    		 
    Database table[1]Name of the destination table. 
    Database view[1]Name of the destination view. All columns of the view must refer to the same table. 
    Database schema 

    Owner of table or view. It does not need to be specified if the user performing the operations is the owner. If it is not set and the user is not the owner, SQL Server returns an error message and the process is cancelled.

    		 
    User nameyes

    The login ID to be used when connecting to the server. The same can be set by specifying the value of the userName parameter in the Parameters attribute. If set, userName in Parameters is ignored.

    		 
    Passwordyes

    The password for the login ID to be used when connecting to the server. The same can be set by specifying the value of the password parameter in the Parameters attribute. If set, password in Parameters is ignored.

    		 
    Advanced
    Column delimiter 

    The delimiter used for each column in data. Field values cannot have the delimiter within them. The same can be set by specifying the value of the fieldTerminator parameter in the Parameters attribute. If set, fieldTerminator in Parameters is ignored.

    		\t (default) | any other character
    Loader input file[2]

    Name of the input file to be loaded, including path. For more information, see Loader input file.

    		 
    Parameters 

    All parameters that can be used as parameters by the bcp utility. These values are contained in a sequence of pairs of the following form: key=value, or key only (if the key value is the boolean true) separated from each other by a semicolon. If the value of any parameter contains a semicolon as its part, such value must be double quoted. For more information, see Parameters.

    		 

    [1]  One of these must be specified.

    [2]  If the input port is not connected, Loader input file must be specified and contain data. For more information, see Loader input file.

    Details

    MSSQLBulkWriter loads data into a MSSQL database using the MSSQL database client. SQL Server Client Connectivity Components must be installed and configured on the same machine where CloverDX runs. The bcp command line tool must be available.

    MSSQLBulkWriter reads data through the input port or from a file. If the input port is not connected to any other component, data must be contained in file that should be specified in the component using Loader input file attribute.

    If you connect some other component to the optional output port, it can serve to log the rejected records and information about errors. Metadata on this error port must have the same metadata fields as the input records plus three additional fields at its end: number of incorrect row (integer), number of incorrect column (integer), error message (string).

    MSSQLBulkWriter is a bulk loader suitable for uploading many records to database. To insert several records, you can also use DatabaseWriter which does not require the bcp utility.

    Loader input file

    Depending on an edge being connected to the input port, you either can or you must specify another attribute (Loader input file). It is the name of an input file with data to be loaded, including its path.

    • If it is not set, a loader file is created in CloverDX or OS temporary directory (on Windows) or named pipe is used instead of temporary file (on Unix). The file is deleted after the load finishes.

    • If it is set, a specified file is created. It is not deleted after data is loaded and it is overwritten on each graph run.

    • If an input port is not connected, the file must exist, must be specified and must contain data that should be loaded. It is not deleted nor overwritten.

    Parameters

    Parameters serve to pass additional parameters to the bcp utility. All of the parameters must have the form of key=value or key only (if its value is true). Individual parameters must be separated from each other by a colon, semicolon or pipe. Note that a colon, semicolon or pipe can also be a part of some parameter value, but in this case the value must be double quoted.

    Among the optional parameters, you can also set userName, password or fieldTerminator for User name, Password or Column delimiter attributes, respectively. If some of the three attributes (User name, Password and Column delimiter) are set, corresponding parameter value will be ignored.

    If you also need to specify the server name, you should do it within parameters. The pattern is as follows: serverName=[msServerHost]:[msServerPort]. For example, you can specify both server name and user name in the following way: serverName=msDbServer:1433|userName=msUser.

    Notes and Limitations

    MSSQLBulkWriter does not write lists and maps.

    Examples

    Loading data to MSSQL database

    Load records (productID, amount) to table products of MSSQL database sales on mssql.example.com. Database schema is dbo, username is smithj and password is smithy.

    Solution

    Install the bcp utility.

    Set up the following attributes of MSSQLBulkWriter.

    AttributeValue
    Path to bcp utilityC:/Program Files/Microsoft SQL Server/Client SDK/ODBC/110/Tools/Binn/bcp.exe
    Databasesales
    Server namemssql.example.com
    Database tableproducts
    Database schemadbo
    Usersmithj
    Passwordsmithy

    In case the server uses a non-standard port number, append the port number after Server name. 

    mssql.example.com,1437

    The port number is separated by a comma, not by a semicolon.

    Troubleshooting

    To test that the connection to database works, you can read some database table with the bcp utility directly: 

    bcp [database_name].[database_schema].[tablename] out outputfile -U user -S serverHostName,port -c

    e.g. 

    bcp master.dbo.products out products.txt -U doejohn -S mssql.example.com,1435 -c

    You can use the utility to check that you can insert records into database table: 

    bcp [database_name].[database_schema].[tablename] in inputfile -U user -S serverHostName,port -c

    e.g. 

    bcp master.dbo.products in inputfile.txt -U doejohn -S mssql.example.com,1435 -c

    The inputfile.txt should be a tab-delimited csv file.

    See also documentation on bcp.

    Compatibility

    VersionCompatibility Notice
    5.3.0MSSQLDataWriter was renamed to MSSQLBulkWriter.