Version
OracleBulkWriter
Short Description
OracleBulkWriter loads data into Oracle database.
|
When to use OracleBulkWriter
This component requires installation and configuration of a database native client. The client must be running on the same machine as the database server. Due to this overhead, we recommend using this component only if you require significantly higher loading performance than with DatabaseWriter which should be used in typical scenarios. |
| Component | Data output | Input ports | Output ports | Transformation | Transf. required | Java | CTL | Auto-propagated metadata |
|---|---|---|---|---|---|---|---|---|
OracleBulkWriter |
database |
0-1 |
0-1 |
⨯ |
⨯ |
⨯ |
⨯ |
⨯ |
Ports
| Port type | Number | Required | Description | Metadata |
|---|---|---|---|---|
Input |
0 |
Records to be loaded into the database. |
Any |
|
Output |
0 |
⨯ |
Rejected records |
Input 0 |
Metadata
OracleBulkWriter does not propagate metadata.
It has no metadata templates.
Both ports must have the same metadata.
Input metadata has to have the same names as database table names. Otherwise, BD column names has to be used.
OracleBulkWriter Attributes
| Attribute | Req | Description | Possible values |
|---|---|---|---|
Basic |
|||
Path to sqlldr utility |
yes |
The name of sqlldr utility, including the path. |
|
TNS name |
yes |
TNS name identifier. |
E.g. db.example.com |
User name |
yes |
Username to be used when connecting to the Oracle database. |
|
Password |
yes |
Password to be used when connecting to the Oracle database. |
|
Oracle table |
yes |
The name of the database table into which the records should be loaded. |
|
Advanced |
|||
Control script |
Control script for the sqlldr utility. For more information, see Control Script. |
||
Append |
Specifies what should be done with the database table. For more information, see Append Attribute. |
append (default) | insert | replace | truncate |
|
Log file name |
The name of the file where the process is logged. |
${PROJECT}/loaderinputfile.log |
|
Bad file name |
The name of the file where the records causing errors are written. |
${PROJECT}/loaderinputfile.bad |
|
Discard file name |
The name of the file where the records not meeting selection criteria are written. |
${PROJECT}/loaderinputfile.dis |
|
DB column names |
The names of all columns in the database table. |
E.g. f1;f2;f3 |
|
Loader input file |
The name of an input file to be loaded, including the path. For more information, see Loader Input File. |
||
Max error count |
The maximum number of allowed insert errors. When this number is exceeded, the graph fails. If no errors are to be allowed, the attribute should be set to 0. To allow all errors, set this attribute to a very high number. |
50 (default) | 0-N |
|
Max discard count |
The number of records that can be discarded before the graph stops. If set to 1, even single discarded record stops the graph run. |
all (default) | 1-N |
|
Ignore rows |
The number of rows of the data file that should be skipped when loading data to a database. |
0 (default) | 1-N |
|
Commit interval |
Conventional path loads only: Commit interval specifies the number of rows in the bind array. Direct path loads only: Commit interval identifies the number of rows that should be read from the data file before the data is saved. By default, all rows are read and data is all saved at once, at the end of the load. |
64 (default for conventional path) | all (default for direct path) | 1-N |
|
Use file for exchange |
On Unix, a pipe transfer is used, by default.
If it is set to |
false (default on Unix) | true (default on Windows) |
|
Parameters |
All parameters that can be used as parameters by the sqlldr utility.
These values are contained in a sequence of pairs of the following form: |
||
Fail on warnings |
By default, the component fails on errors.
By switching the attribute to |
false (default) | true |
|
Details
OracleBulkWriter loads data into a database using Oracle database client. It can read data through the input port or from an input file. If the input port is not connected to any other component, data must be contained in an input file that should be specified in the component. If you connect some other component to the optional output port, rejected records are sent to it.
OracleBulkWriter's functionality depends on the sqlldr utility. Oracle sqlldr database utility must be installed on the computer where CloverDX runs.
See details on sqlldr utility: Loading data into an Oracle database with SQL*Loader
|
One of the common issues with the component functionality is the missing |
OracleBulkWriter is a bulk loader suitable for uploading many records to a database.
To insert several records, you can also use DatabaseWriter, which does not require the sqlldr utility.
Control Script
Control script for the sqlldr utility.
-
If specified, both the Oracle table and the Append attributes are ignored. Must be specified if input port is not connected. In such a case, Loader input file must also be defined.
-
If Control script is not set, default control script is used.
Example 34. Example of a Control script
LOAD DATA
INFILE *
INTO TABLE test
append
(
name TERMINATED BY ';',
value TERMINATED BY '\n'
)Append Attribute
-
Append (default)
Specifies that data is simply appended to a table. Existing free space is not used.
-
Insert
Adds new rows to the table/view with the
INSERTstatement. TheINSERTstatement in Oracle is used to add rows to a table, the base table of a view, a partition of a partitioned table or a subpartition of a composite-partitioned table, or an object table or the base table of an object view.An
INSERTstatement with aVALUESclause adds to the table a single row containing the values specified in theVALUESclause.An
INSERTstatement with a subquery instead of aVALUESclause adds to the table all rows returned by the subquery. Oracle processes the subquery and inserts each returned row into the table. If the subquery selects no rows, Oracle inserts no rows into the table. The subquery can refer to any table, view, or snapshot, including the target table of theINSERTstatement. -
Update
Changes existing values in a table or in a view’s base table.
-
Truncate
Removes all rows from a table or cluster and resets the
STORAGEparameters to the values when the table or cluster was created.
Loader Input File
Name of input file to be loaded, including path.
-
If it is not set, a loader file is created in CloverDX or OS temporary directory (on Windows) (unless Use file for exchange is set to
false) ornamed pipeis used instead of temporary file (in Unix). The created file is deleted after the load finishes. -
If it is set, specified file is created. The created file is not deleted after data is loaded and it is overwritten on each graph run.
-
If the input port is not connected, this file must be specified, must exist and must contain data that should be loaded into database. At the same time, Control script must be specified. The file is not deleted nor overwritten.
Notes and Limitations
OracleBulkWriter does not support writing lists and maps.
Example
Load records (fields username, surname) into database table users2.
The Oracle database is installed on bd.example.com and listens on 1521.
Database login smithj and password MySecretPassword.
Solution
Install sqlldr utility, if it is not installed.
Set up the following attributes of the component:
| Attribute | Value |
|---|---|
Path to sqlldr utility |
/app/product/12.1.0/client_1/bin/sqlldr |
TNS name |
db.example.com |
User name |
smithj |
Password |
MySecretPassword |
Oracle table |
users2 |
Metadata field names have to match database table field names.
Case 2: database table has columns user and surn.
Set up attribute DB Column names to user;surn.
Compatibility
| Version | Compatibility Notice |
|---|---|
5.3.0 |
OracleDataWriter was renamed to OracleBulkWriter. |