Version
MSSQLBulkWriter
Short Description
MSSQLBulkWriter loads data into MSSQL database.
|
When to use MSSQLBulkWriter
This component requires installation and configuration of a database native client. The client must be running on the same machine as the database server. Due to this overhead, we recommend using this component only if you require significantly higher loading performance than with DatabaseWriter which should be used in typical scenarios. |
| Component | Data output | Input ports | Output ports | Transformation | Transf. required | Java | CTL | Auto-propagated metadata |
|---|---|---|---|---|---|---|---|---|
MSSQLBulkWriter |
database |
0-1 |
0-1 |
⨯ |
⨯ |
⨯ |
⨯ |
⨯ |
Ports
| Port type | Number | Required | Description | Metadata |
|---|---|---|---|---|
Input |
0 |
Records to be loaded into the database |
Any |
|
Output |
0 |
⨯ |
For information about incorrect records |
Input 0 (plus three Error Fields for MSSQLBulkWriter |
| 1 |
If no file containing data for loading (Loader input file) is specified, the input port must be connected. |
This component has one optional input port and one optional output port.
Metadata
MSSQLBulkWriter does not propagate metadata.
Metadata on the output port 0 contains three additional fields at their end: number of incorrect row, number of incorrect column, error message.
| Field number | Field name | Data type | Description |
|---|---|---|---|
LastInputField + 1 |
<anyname1> |
integer |
The number of the incorrect row. |
LastInputField + 2 |
<anyname2> |
integer |
The number of the incorrect column. |
LastInputField + 3 |
<anyname3> |
string |
The error message. |
MSSQLBulkWriter Attributes
| Attribute | Req | Description | Possible values |
|---|---|---|---|
Basic |
|||
Path to bcp utility |
yes |
The name of the bcp utility, including path. SQL Server Client Connectivity Components must be installed and configured on the same machine where CloverDX runs. The bcp command line tool must be available. |
|
Database |
yes |
The name of the database where the destination table or view resides. |
|
Server name |
The name of the server to which bcp utility should connect. If the bcp utility connects to local named or remote named instance of server, Server name should be set to If the bcp utility connects to a local default or remote default instance of server, Server name should be set to |
||
Database table |
Name of the destination table. |
||
Database view |
Name of the destination view. All columns of the view must refer to the same table. |
||
Database schema |
Owner of table or view. It does not need to be specified if the user performing the operations is the owner. If it is not set and the user is not the owner, SQL Server returns an error message and the process is cancelled. |
||
User name |
yes |
The login ID to be used when connecting to the server.
The same can be set by specifying the value of the |
|
Password |
yes |
The password for the login ID to be used when connecting to the server.
The same can be set by specifying the value of the |
|
Advanced |
|||
Column delimiter |
The delimiter used for each column in data.
Field values cannot have the delimiter within them.
The same can be set by specifying the value of the fieldTerminator parameter in the Parameters attribute.
If set, |
\t (default) | any other character |
|
Loader input file |
Name of the input file to be loaded, including path. For more information, see Loader input file. |
||
Parameters |
All parameters that can be used as parameters by the bcp utility.
These values are contained in a sequence of pairs of the following form: |
||
| 2 |
If the input port is not connected, Loader input file must be specified and contain data. For more information, see Loader input file. |
Details
MSSQLBulkWriter loads data into a MSSQL database using the MSSQL database client.
SQL Server Client Connectivity Components must be installed and configured on the same machine where CloverDX runs.
The bcp command line tool must be available.
MSSQLBulkWriter reads data through the input port or from a file. If the input port is not connected to any other component, data must be contained in file that should be specified in the component using Loader input file attribute.
If you connect some other component to the optional output port, it can serve to log the rejected records and information about errors.
Metadata on this error port must have the same metadata fields as the input records plus three additional fields at its end: number of incorrect row (integer), number of incorrect column (integer), error message (string).
MSSQLBulkWriter is a bulk loader suitable for uploading many records to database.
To insert several records, you can also use DatabaseWriter which does not require the bcp utility.
Loader input file
Depending on an edge being connected to the input port, you either can or you must specify another attribute (Loader input file). It is the name of an input file with data to be loaded, including its path.
-
If it is not set, a loader file is created in CloverDX or OS temporary directory (on Windows) or
named pipeis used instead of temporary file (on Unix). The file is deleted after the load finishes. -
If it is set, a specified file is created. It is not deleted after data is loaded and it is overwritten on each graph run.
-
If an input port is not connected, the file must exist, must be specified and must contain data that should be loaded. It is not deleted nor overwritten.
Parameters
Parameters serve to pass additional parameters to the bcp utility. All of the parameters must have the form of key=value or key only (if its value is true). Individual parameters must be separated from each other by a colon, semicolon or pipe.
Note that a colon, semicolon or pipe can also be a part of some parameter value, but in this case the value must be double quoted.
Among the optional parameters, you can also set userName, password or fieldTerminator for User name, Password or Column delimiter attributes, respectively.
If some of the three attributes (User name, Password and Column delimiter) are set, corresponding parameter value will be ignored.
If you also need to specify the server name, you should do it within parameters.
The pattern is as follows: serverName=[msServerHost]:[msServerPort].
For example, you can specify both server name and user name in the following way: serverName=msDbServer:1433|userName=msUser.
Notes and Limitations
MSSQLBulkWriter does not write lists and maps.
Examples
Loading data to MSSQL database
Load records (productID, amount) to table products of MSSQL database sales on mssql.example.com.
Database schema is dbo, username is smithj and password is smithy.
Solution
Install the bcp utility.
Set up the following attributes of MSSQLBulkWriter.
| Attribute | Value |
|---|---|
Path to bcp utility |
C:/Program Files/Microsoft SQL Server/Client SDK/ODBC/110/Tools/Binn/bcp.exe |
Database |
sales |
Server name |
mssql.example.com |
Database table |
products |
Database schema |
dbo |
User |
smithj |
Password |
smithy |
In case the server uses a non-standard port number, append the port number after Server name.
mssql.example.com,1437The port number is separated by a comma, not by a semicolon.
Troubleshooting
To test that the connection to database works, you can read some database table with the bcp utility directly:
bcp [database_name].[database_schema].[tablename] out outputfile -U user -S serverHostName,port -ce.g.
bcp master.dbo.products out products.txt -U doejohn -S mssql.example.com,1435 -cYou can use the utility to check that you can insert records into database table:
bcp [database_name].[database_schema].[tablename] in inputfile -U user -S serverHostName,port -ce.g.
bcp master.dbo.products in inputfile.txt -U doejohn -S mssql.example.com,1435 -cThe inputfile.txt should be a tab-delimited csv file.
See also documentation on bcp.
Compatibility
| Version | Compatibility Notice |
|---|---|
5.3.0 |
MSSQLDataWriter was renamed to MSSQLBulkWriter. |