Version

    MongoDBWriter

    Short Description
    Ports
    Metadata
    MongoDBWriter Attributes
    Details
    Mapping
    Error Handling in Bulk Operations
    Examples
    See also

    Short Description

    MongoDBWriter stores, removes, or updates data in the MongoDB database using the Java driver.

    MongoDBWriter can manipulate with documents in a MongoDB collection. New documents can be inserted, existing documents can be updated or removed.

    ComponentData outputInput portsOutput portsTransformationTransf. requiredJavaCTLAuto-propagated metadata
    MongoDBWriterdatabase10-2✓>✓>⨯>✓>✓

    Ports

    Port typeNumberRequiredDescriptionMetadata
    Input0✓Input data records to be mapped to component attributes.any
    Output0⨯Resultsany
    1⨯Errorsany

    Metadata

    MongoDBWriter does not propagate metadata.

    This component has metadata templates. The templates are described in the documentation of MongoDBReader in the section Metadata.

    MongoDBWriter Attributes

    AttributeReqDescriptionPossible values
    Basic
    Connection✓The ID of the MongoDB connection to be used. 
    Collection name✓  [ 1]The name of the target collection. 
    Operation 

    The operation to be performed. MongoDBWriter can perform:

    		See the description.
    Query 

    Selects a subset of documents from a collection. The selection criteria may contain query operators.

    Ignored by the insertOne, insert and save operations.

    		BSON document
    New value✓  [ 1]

    Specifies the document to be stored in the database.

    Ignored by delete operations (deleteOne, deleteMany and remove).

    For update operations, New value may contain update operators.

    		BSON document
    Input mapping✓Defines the mapping of input records to component attributes. 
    Output mapping Defines mapping of results to standard output port. 
    Error mapping Defines the mapping of errors to the error output port. 
    Advanced
    Batch size 

    Number of records that can be sent to database in one batch.

    Bulk write operations may significantly increase performance. However, the whole batch is stored in memory, so increasing Batch size also increases memory requirements.

    		bulk write: 100 (default) | basic: 1 (default)
    Stop processing on fail 

    If true, a failure causes the component to skip all subsequent operations and send the information about skipped executions to the error output port. Note: this function works only if the edge is connected to the component's error port.

    		true (default) | false
    Field pattern 

    Specifies the format of placeholders that can be used within the Query and New value attributes. The value of the attribute must contain "field" as a substring, e.g. "<field>", "#{field}", etc.

    During the execution, each placeholder is replaced using a simple string substitution with the value of the respective input field, e.g. the string "@{name}" will be replaced with the value of the input field called "name" (assuming the default format of the placeholders).

    		@{field} (default) | any string containing "field" as a substring.

    [ 1] The attribute is required, unless specified in the Input mapping.

    Details

    Operations
    Mapping
    Error Handling in Bulk Operations
    Notes and Limitations
    Format of the date Field Value

    Operations

    There are two types of operations available for this component: bulk write and basic operations. Bulk write operations are supported since driver and DB version 3.2.

    Table 56.7. List of operations

    OperationDescription
    Bulk write operations (recommended)
    insertOne

    Adds the value of the New value attribute as a new document to the target Collection. If the document does not contain the _id field, a generated one will be added.

    See also db.collection.insertOne().

    updateOne

    Updates a single document matching the Query, with the values specified in the New value attribute, which must contain update operators.

    The Upsert parameter is available for this operation.

    See also db.collection.updateOne().

    updateMany

    Updates multiple documents matching the Query, with the values specified in the New value attribute, which must contain update operators.

    The Upsert parameter is available for this operation.

    See also db.collection.updateMany().

    replaceOne

    Replaces a single document matching the Query.

    The Upsert parameter is available for this operation.

    See also db.collection.replaceOne().

    deleteOne

    Removes a single document matching the Query.

    See also db.collection.deleteOne().

    deleteMany

    Removes all documents matching the Query.

    See also db.collection.deleteMany().

    Parameters for bulk write operations
    Upsert

    Only applicable to updateOne, updateMany and replaceOne. If enabled, the operation inserts a new document into the collection if no document matches the Query.

    Generated object IDs for upserted documents will be returned as the objectId field in the output mapping.

    Ordered

    Executes operations in the order they arrive within every batch. If enabled, the first error causes the following operations in the same batch to be skipped.

    Basic operations
    insert

    Adds the value of the New value attribute as a new document to the target Collection. If the document does not contain the _id field, a generated one will be added.

    See also db.collection.insert().

    remove

    Removes objects that match the Query from the Collection.

    See also db.collection.remove().

    save

    Similar to insert. Adds the document specified as the New value attribute to the Collection or replaces an existing document with the same _id.

    See also db.collection.save().

    update

    Updates at most one document that matches the Query, with the values specified in the New value attribute which may contain update operators.

    See also db.collection.update().

    update_multi

    Updates multiple documents matching the Query with the values specified in the New value attribute which must contain update operators.

    See also db.collection.update() - multi.

    upsert

    If no existing document matches the Query, inserts a new document into the Collection, otherwise performs an update. The New value attribute may contain update operators.

    See also db.collection.update() - upsert.


    Mapping

    Editing any of the Input, Output or Error mapping opens the Transform Editor.

    Input mapping

    The editor allows you to override selected attributes of the component with the values of the input fields.

    Field NameAttributeTypePossible values
    collectionCollectionstring 
    queryQuerystring 
    newValueProjectionstring 
    Output mapping

    The editor allows you to map the results and the input data to the output port.

    If Output mapping is empty, fields of input record and result record are mapped to output by name.

    Field NameTypeDescription
    numAffectedinteger

    The number of affected documents, only set by the update, update_multi and upsert operations.

    objectIdstring

    The object ID of the document.

    Bulk write operations: set by the insertOne operation, and the updateOne, updateMany and replaceOne operations for upsert.

    Basic operations: set by the insert and save operations. (Not populated in the bulk insert mode.)

    batchNumberlongThe sequence number of the current batch, starting from 0.
    deletedCountintegerThe number of documents deleted by the current batch.
    insertedCountintegerThe number of documents inserted by the current batch.
    matchedCountintegerThe number of documents matched by the current batch.
    modifiedCountintegerThe number of documents modified by the current batch.
    Error mapping

    The editor allows you to map the errors and the input data to the error port.

    If Error mapping is empty, fields of input record and result record are mapped to output by name.

    Field NameTypeDescription
    errorMessagestringThe error message.
    stackTracestringThe stack trace of the error.
    batchNumberlongThe sequence number of the current batch, starting from 0.
    deletedCountintegerThe number of documents deleted by the current batch.
    insertedCountintegerThe number of documents inserted by the current batch.
    matchedCountintegerThe number of documents matched by the current batch.
    modifiedCountintegerThe number of documents modified by the current batch.

    Error Handling in Bulk Operations

    Each input record produces one output record either on the standard, or error output port. A record is sent to the error output port if an error occurs or the operation is skipped. In such a case, see the errorMessage field in Error mapping for details and possible solution.

    Notes and Limitations

    MongoDBWriter does not write maps and lists. It converts maps and lists to string and writes the string.

    Format of the date Field Value

    A date value is in an ISO-8601 date format. 

    {
    withTZ : { "$date": "2018-11-22T14:25:11.541+02:00" },
    millis: { "$date": "2018-11-22T14:25:11.541" },
    seconds: { "$date": "2018-11-22T14:25:11" },
    dateLocal: { "$date": "2018-11-22" },
    dateUTC: { "$date": "2018-11-22+00:00" }
}

    Examples

    Writing records to MongoDB

    Insert records (productID, productName, description) to collection newProducts.

    Solution

    Create MongoDB Connection to target database.

    Set up the following attributes:

    AttributeValue
    ConnectionMyMongoDBConnection
    Collection namenewProducts
    OperationinsertOne
    New value{ productID : @{productID}, productName : "@{productName}", description: "@{description}"}