CloverDX Designer
CloverDX Overview
Family of CloverDX Products
Products Introduction
CloverDX Designer
CloverDX Server
CloverDX Cluster
Online Resources
Installation
System Requirements
Downloading
Installing
Windows
Mac OS
Linux
Starting
Activating
License Manager
CloverDX License Dialog
Troubleshooting
Windows
Windows SmartScreen
User Account Control
Windows 10 Firewall
Linux
Designer Stops Responding
Welcome Page not Displayed
Hints on Edges Have no Content
Component Editor Not Working
Others
Subclipse
Online Activity
Logs
IBM InfoSphere MDM Plugin Installation
Downloading
Installation into Designer
Troubleshooting
Installation into Server
Troubleshooting
Upgrading
Configuration
Configuration
Runtime Configuration
Logging
Master Password
User Classpath
CloverDX Server Integration
Ignored Files
Execution Monitoring
Java Configuration
Engine Configuration
Refresh Operation
Using Designer
Designer User Interface
Designer Panes
Graph Editor with Palette of Components
Project Explorer Pane
Outline Pane
Tabs Pane
Execution Tab
Keyboard Shortcuts
Projects
Types of CloverDX Projects
CloverDX (Local) Project
CloverDX Server Project
Creating CloverDX Projects
CloverDX Project
CloverDX Server Project
CloverDX Examples Project
Converting CloverDX Projects
Converting Local Project to Server Project
Converting Server Projects to Local Project
Structure of CloverDX Projects
Standard Structure of All CloverDX Projects
The .classpath File
Workspace.prm File
Versioning of Server Project Content
Initial Check-Out of Project from Repository
Adding Server Project to Version Control
Connecting Server Project to Existing Repository
Getting Changes from Repository
Committing into Repository
Working with CloverDX Server Projects
Working Offline
Handling Conflicts
Project Configuration
CloverDX Connection
Ignored Files
Graphs
Creating an Empty Graph
Creating a Simple Graph
Execution
Successful Graph Execution
Run Configuration
Run Configuration vs CloverDX Runtime
Main Tab
Parameters Tab
Refresh Tab
Connecting to a Running Job
Graph States
Common Dialogs
URL File Dialog
Local Files
Workspace View
Hadoop HDFS
Remote Files
Port
Dictionary
Filtering Files and Tips
Edit Value Dialog
Open Type Dialog
Import
Import CloverDX Projects
Import CloverDX Project from URL
Import CloverDX Projects
Import from CloverDX Server Sandbox
Import Graphs
Import Metadata
Metadata from XSD
Metadata from DDL
Import Wrangler Mapping
Reject metadata
Export
Convert Graph to Jobflow
Convert Jobflow to Graph
Convert Subgraph to Graph
Export Graphs to HTML
Export to CloverDX Server Sandbox
Export Image
Export Project as Library
Graph Tracking
Changing Record Count Font Size
Search Functionality
Working with CloverDX Server
CloverDX Server Project Basic Principles
Connecting via HTTP
Connecting via HTTPS
Designer has its Own Certificate
Designer does not have its Own Certificate
Connecting via Proxy Server
Graphs
Components
Adding Components
Finding Components
Edit Component Dialog
Commands
Attributes
Enable/Disable Component
Enabling Component
Disabling Component
Enabling by Graph Parameter
Enabling by Connected Input Port
Disable as Trash
Passing Data Through Disabled Component
Common Properties of Components
Component Name
Phases
Component Allocation
Specific Attribute Types
Time Intervals
Group Key
Sort Key
Metadata Templates
Edges
Connecting Components with Edges
Types of Edges
Assigning Metadata to Edges
Colors of Edges
Debugging Edges
Selecting Debug Data
Viewing Debug Data
Turning Off Debug
Edge Memory Allocation
Metadata
Records and Fields
Record Types
Data Types in Metadata
Data Formats
Locale and Locale Sensitivity
Time Zone
Autofilling Functions
Metadata Types
Internal Metadata
External (Shared) Metadata
SQL Query Metadata
Reading Metadata from Special Sources
Auto-propagated Metadata
Priorities of Metadata
Propagation of SQL Query Metadata
Creating Metadata
Extracting Metadata from a Flat File
Extracting Metadata from an XLS(X) File
Extracting Metadata from a Database
Extracting Metadata from a DBase File
Extracting Metadata from Salesforce
SQL Query Metadata
User Defined Metadata
Merging Existing Metadata
Creating Database Table from Metadata and Database Connection
Metadata Editor
Basics of Metadata Editor
Record Pane
Field Name vs. Label vs. Description
Details Pane
Changing and Defining Delimiters
Changing Record Delimiter
Changing Default (Field) Delimiter
Defining Non-Default Delimiter for a Field
Editing Metadata in the Source Code
Multivalue Fields
Lists and Maps Support in Components
Joining on multivalue fields (Comparison Rules)
Connections
Database Connections
Internal Database Connections
External (Shared) Database Connections
Database Connection Properties
Encryption of Access Password
Browsing Database and Extracting Metadata from Database Tables
Windows Authentication on Microsoft SQL Server
Snowflake Connection
Hive Connection
Troubleshooting
JMS Connections
Internal JMS Connections
External (Shared) JMS Connections
Edit JMS Connection Wizard
Encrypting the Authentication Password
QuickBase Connections
Hadoop Connections
Connecting to YARN
Libraries Needed for Hadoop
Kerberos Authentication for Hadoop
Kafka Connections
Creating Kafka Connection
Details
OAuth2 Connections
Creating OAuth2 Connection
Authorize connection
Details
MongoDB Connections
Salesforce Connections
Creating Salesforce Connection
Important Details
Lookup Tables
Lookup Tables in Cluster Environment
Internal Lookup Tables
Creating Internal Lookup Tables
Externalizing Internal Lookup Tables
Exporting Internal Lookup Tables
External (Shared) Lookup Tables
Creating External (Shared) Lookup Tables
Linking External (Shared) Lookup Tables
Internalizing External (Shared) Lookup Tables
Types of Lookup Tables
Simple Lookup Table
Database Lookup Table
Range Lookup Table
Persistent Lookup Table
Aspell Lookup Table
Sequences
Persistent Sequences
Non-Persistent Sequences
Internal Sequences
Creating Internal Sequences
Externalizing Internal Sequences
Exporting Internal Sequences
External (Shared) Sequences
Creating External (Shared) Sequences
Linking External (Shared) Sequences
Internalizing External (Shared) Sequences
Editing a Sequence
Sequences in Cluster Environment
Parameters
Internal Parameters
Creating Internal Parameters
Externalizing Internal Parameters
External (Shared) Parameters
Creating External Parameters
Linking External Parameters
Internalizing External (Shared) Parameters
XML Schema of External Parameters
Graph Parameter Editor
Secure Graph Parameters
Parameters with CTL2 Expressions (Dynamic Parameters)
Environment Variables
Canonicalizing File Paths
Using Parameters
Internal/External Graph Elements
Internal Graph Elements
External (Shared) Graph Elements
Working with Graph Elements
Advantages of External (Shared) Graph Elements
Advantages of Internal Graph Elements
Changing Form of Graph Elements
Dictionary
Creating a Dictionary
Dictionary Entry Types
Dictionary Entry Properties
Using a Dictionary in Graphs
Accessing Dictionary from Readers and Writers
Accessing Dictionary with Java
Accessing Dictionary with CTL2
Execution Properties
Notes in Graphs
Placing Notes into Graph
Resizing Notes
Editing Notes
Formatted Text
Links from Notes
Folding Notes
Notes Properties
Compatibility
Transformations
Defining Transformations
Components Allowing Transformation
Java or CTL
Internal or External Definition
Return Values of Transformations
Error Actions and Error Log (deprecated since 3.0)
Transform Editor
Common Java Interfaces
Debugging Java Transformations
Debugging Java Transformations in Local Projects
Debugging Java Transformation in Server Projects
Data Partitioning (Parallel Running)
Data Partitioning in Cluster
High Availability
Scalability
Transformation Requests
Parallel Data Processing
Graph Allocation Examples
Example of Distributed Execution
Details of the Example Transformation Design
Scalability of the Example Transformation
Remote Edges
Subgraphs
Subgraphs Overview
Subgraphs Introduction
Design & Execution
Subgraphs vs. Jobflow
Using Subgraphs
Using Subgraphs
Configuring Subgraphs
Developing Subgraphs
Wrapping
Creating from Scratch
Making Subgraph Configurable
Developing and Testing Subgraphs
Filling Required Parameters
Metadata Propagation
Design Patterns
Readers
Writers
Transformers
Executors
Jobflow
Jobflow Overview
Jobflow Introduction
What is CloverDX Jobflow?
Design and Execution
Anatomy of the Jobflow Module
Important Concepts
Dynamic Attribute Setting
Parameter Passing
Pass-Through Mapping
Execution Status Reporting
Error Handling
Jobflow Execution Model: Single Token
Jobflow Execution Model: Multiple Tokens
Stopping on Error
Synchronous vs. Asynchronous Execution
Logging
Advanced Concepts
Daemon Jobs
Killing Jobs
Jobflow Design Patterns
Data Services
Data Services Introduction
Architecture
Development
Data Service Job Editor
Endpoint Configuration
Data Service REST Job Logic
Anatomy of Data Service Jobs
Input and Output Components
HTTP Request Payload
HTTP Request Parameters
HTTP Headers
HTTP Response
Multiple Edges
HTTP Status Code and Headers
Execution Steps of Data Service Jobs
Exceptions and Error Handling
Auto-generated Documentation and Swagger/OpenAPI Definition
Testing
Testing Service Logic in Designer
Testing Services Deployed on Server
Use cases
Creating REST APIs
Custom Serialization
Using Static File as Response
Sending a File Generated by .rjob
Publishing a Static File
Using CTL2 Functions in Data Services
Data Service that Receives a File or Text in Body Part
Converting Graph to Data Service
Converting Graph to Data Service in Server Projects
Converting Graph to Data Service in Local Projects
Publishing Data Service
Publishing Multiple Data Services at Once
Unpublishing Data Service
Unpublishing Multiple Data Services at Once
Example
Troubleshooting
Server Returns Error Code 404
Server Returns Error Code 500
Server Returns Error Code 503
Libraries Development
Libraries
Introduction
Usage
Subgraphs
Metadata
Graphs, Jobflows
Data Services
Library File Structure
Enhancing Library Look and Feel
Parameters
Global Library Parameters
Connector Parameters
Initialization Jobs (pre-generating metadata)
User Credentials and Secrets
Database/JDBC Connections
OAuth2 Connections
Library Dependencies
Using Java Code
Live Debugging and Testing
Job Documentation Tips
Documentation for Developers
Documentation for Administrators
Data Source Connector Development
Data Source Connector Requirements
Data Source Connector Best Practices
Data Target Connector Development
Data Target Connector Requirements
Data Target Connector Best Practices
Component Reference
Readers
Common Properties of Readers
Supported File URL Formats for Readers
Viewing Data on Readers
Input Port Reading
Incremental Reading
Selecting Input Records
Data Policy
XML Features
CTL Templates for Readers
Java Interfaces for Readers
CloverDataReader
ComplexDataReader
CustomJavaReader
DatabaseReader
DataGenerator
DBFDataReader
EDIFACTReader
EmailReader
FlatFileReader
HadoopReader
HL7Reader
JavaBeanReader
JMSReader
JSONExtract
JSONReader
KafkaCommit
KafkaReader
LDAPReader
MongoDBReader
MultiLevelReader
ParallelReader
ParquetReader
QuickBaseRecordReader
QuickBaseQueryReader
SalesforceBulkReader
SalesforceReader
SpreadsheetDataReader
UniversalDataReader
X12Reader
XMLExtract
XMLReader
XMLXPathReader
Writers
Common Properties of Writers
Supported File URL Formats for Writers
Viewing Data on Writers
Output Port Writing
Appending or Overwriting
Creating Directories
Selecting Output Records
Partitioning Output into Different Output Files
Excluding Fields
Java Interfaces for Writers
CloverDataWriter
CustomJavaWriter
DatabaseWriter
DB2BulkWriter
DBFDataWriter
EDIFACTWriter
EmailSender
FlatFileWriter
HadoopWriter
InformixBulkWriter
JavaBeanWriter
JavaMapWriter
JMSWriter
JSONWriter
KafkaWriter
LDAPWriter
MongoDBWriter
MSSQLBulkWriter
MySQLBulkWriter
OracleBulkWriter
ParquetWriter
PostgreSQLBulkWriter
QuickBaseImportCSV
QuickBaseRecordWriter
SalesforceBulkWriter
SalesforceWriter
SalesforceEinsteinWriter
SnowflakeBulkWriter
SpreadsheetDataWriter
StructuredDataWriter
Trash
UniversalDataWriter
X12Writer
XMLWriter
Transformers
Common Properties of Transformers
CTL Templates for Transformers
Java Interfaces for Transformers
Aggregate
Concatenate
CustomJavaTransformer
DataIntersection
DataSampler
Dedup
Denormalizer
ExtSort
FastSort
Filter
LoadBalancingPartition
Merge
MetaPivot
Normalizer
Partition
Pivot
Map
Rollup
SimpleCopy
SimpleGather
SortWithinGroups
XSLTransformer
Joiners
Common Properties of Joiners
Join Types
Slave Duplicates
CTL Templates for Joiners
Java Interfaces for Joiners
Combine
CrossJoin
DBJoin
ExtHashJoin
ExtMergeJoin
LookupJoin
RelationalJoin
Job Control
Common Properties of Job Control
Barrier
Condition
ExecuteGraph
ExecuteJobflow
ExecuteWranglerJob
ExecuteMapReduce
ExecuteScript
Fail
GetJobInput
KillGraph
KillJobflow
Loop
MonitorGraph
MonitorJobflow
SetJobOutput
Sleep
Subgraph
Success
TokenGather
File Operations
Common Properties of File Operations
Common Attributes of File Operation Components
Supported URL Formats for File Operations
CopyFiles
CreateFiles
DeleteFiles
ListFiles
MoveFiles
Data Partitioning
Common Properties of Data Partitioning Components
ParallelLoadBalancingPartition
ParallelMerge
ParallelPartition
ParallelRepartition
ParallelSimpleCopy
ParallelSimpleGather
Data Quality
Common Properties of Data Quality
EmailFilter
ProfilerProbe
Metrics
List of Metrics
Validator
List of Rules
Others
Common Properties of Others
CheckForeignKey
CustomJavaComponent
DBExecute
HTTPConnector
LookupTableReaderWriter
SequenceChecker
WebServiceClient
Deprecated
ApproximativeJoin
InfobrightBulkWriter
JavaExecute
MongoDBExecute
RunGraph
SystemExecute
TableauWriter
CTL2 - CloverDX Transformation Language
CTL Overview
Basic Features of CTL
CTL History
Language Reference
Program Structure
Comments
Import
Metadata Import in CTL2
Data Types in CTL2
byte
cbyte
date
decimal
integer
long
number (double)
string
list
map
variant
record
Literals
Variables
Constants
Dictionary in CTL2
Operators
Arithmetic Operators
Relational Operators
Logical Operators
Assignment Operator
Ternary Operator
Conditional Fail Expression
Simple Statement and Block of Statements
Control Statements
Conditional Statements
Iteration Statements
Jump Statements
Error Handling
Try-Catch Statement
OnError() Functions
Functions
Message Function
Conditional Fail Expression
Accessing Data Records and Fields
Mapping
Parameters
Regular Expressions
Regex basics
Regex for advanced users
CTL Debugging
Debug Perspective
Importing and Exporting Breakpoints
Exporting Breakpoints
Importing Breakpoints
Inspecting Variables and Expressions
Inspect Action
Expressions View and Watch Action
Examples
Basic Example
Using Hit Count
Conditional Breakpoint
Detecting Changes of the Value
Functions Reference
Conversion Functions
Date Functions
Mathematical Functions
String Functions
Mapping Functions
Container Functions
Record Functions (Dynamic Field Access)
Miscellaneous Functions
Lookup Table Functions
Sequence Functions
Subgraph Functions
Data Service HTTP Library Functions
Custom CTL Functions
List of All CTL2 Functions
CTL2 Appendix - List of National-specific Characters
Tutorial
Creating a Transformation Graph
Terminology
Creating a Project
Creating a New Data File
Creating a Graph
Placing Components in the Graph Editor Pane
Connecting Components by an Edge
Extracting Metadata from the Input File
Assigning Metadata to the Edges
Setting Up Readers (FlatFileReader)
Setting Up Writers (SpreadsheetDataWriter)
Running the Graph
Opening the Output File
What to do next
Filtering the records
Inserting the Filter
Setting Up the Filter Component
Two data streams
Sorting the Records
Adding ExtSort
Setting Up the ExtSort Component
CloverDX Wrangler
Getting started
How Wrangler works (video)
Quick tutorial (video)
Source: Choosing data for your job
Combining multiple data sources
Lookup configuration
Lookup hints
Job anatomy: Understanding steps and repeatability
Moving between steps
What to consider before adding a column
Column data types
Formulas and calculations
Comparing values
Calculations and functions
Modify values based on a condition
if function
Ternary operator
Working with strings (text)
Working with decimals (including currency)
Working with dates
Errors in your data
No data or some missing?
No value: Working with null and empty values
Replacing empty value with a default value
Filtering data
Getting results from your job
Target: Configuring job output
Mapping your data to specific target
Data sources and targets
Introduction
Data sources
My Sources
Adding data sources to My Sources
Updating data source in an existing job
Data source connectors
Adding connectors to My Sources
CSV data sources
Configuring CSV data source
Where are my CSV files?
Limitations of the CSV data source
Excel data sources
Configuring Excel data source
Where are my Excel files?
Limitations of the Excel data source
Data targets
My Targets
Adding data targets to My Targets
Updating data target in existing job
Target mapping
Mapping editor
Data type compatibility
Unmapping columns
Data target preview
Downloading and viewing output files
Reject file format
Data target connectors
CSV data targets
CSV data target configuration
CSV data target limitations
Excel data targets
Excel data target configuration
Formatting of Excel files
Excel data target limitations
Data Catalog
Searching in the Data Catalog
Connector details
Transforming data
My Jobs screen
Creating your job
Duplicate job
Job Export and Import
Test run
Working with transformations
Data quality bar
Using Steps sidebar
Data types in Wrangler
Integer data type
Decimal data type
Date data type
String data type
Boolean data type
Empty values
Formatting your data
Integer formatting
Decimal formatting
Date formatting
Using formulas
Referencing data set columns in formulas
Autocomplete feature in Formula editor
Quick example
Mathematical operators
Comparison operators
Logical operators
Error handling in Wrangler
Errors in data
Validation errors
Errors occurring during step execution
Step configuration errors
Errors and output
Fixing errors
Fixing data errors
Fixing step execution errors
Fixing step configuration errors
Running jobs in Wrangler
Viewing job status
Job run details
How does Wrangler expression language differ from CTL?
Data types
if function
How to use CTL functions in Wrangler?
List of common functions
Date functions
String functions
Conversion functions
Transformation steps
Data set manipulation steps
Add column
Duplicate column
Delete column(s)
Rename column
Reorder columns
Merge columns
Split column
Filter rows based on formula
Sort
Lookup
Calculate formula
Text manipulation steps
Left substring
Right substring
Substring
Trim
Pad left
Pad right
Normalize whitespaces
Remove whitespaces
Remove accents
Lowercase
Propercase
Uppercase
Remove non-alphanumeric characters
Remove non-ASCII characters
Remove non-numeric characters
Remove non-printable characters
Replace text
Data validation (data quality) steps
Validate against list
Validate credit card
Validate email
Validate if not empty
Validate pattern match
Validate phone number
Validate text length
Validate value range
Validate with formula
Error handling steps
Remove rows with errors
Replace errors
Replace empty values
Clear error cells
Math steps
Absolute value
Round step
Truncate
Square root
Exponential
Raise number to a power
Natural logarithm
Common logarithm
Ceiling
Floor
Division remainder
Date manipulation steps
Current date and time
Date add/subtract
Date difference
Get part of date
Data conversion steps
Convert to boolean
Convert to date
Convert to decimal
Convert to integer
Convert to string
Convert Unix time to date
Data anonymization steps
Add noise to date
Add noise to number
Mask text
Step and group conditions
Conditional formulas logic
Step conditions
Enabling step conditions
Group conditions
Enabling group conditions
Important considerations for group conditions
Steps with condition support
CloverDX Server
Introduction
What is CloverDX Server?
CloverDX Server Architecture
CloverDX Core
CloverDX Worker
CloverDX Cluster
Installation
System Requirements
Installing
Evaluation Server
Installation
Production Server
Apache Tomcat
Activation
CloverDX Server Activation Alternatives
IBM InfoSphere MDM Plugin Installation
Installation into Server
Postinstallation Configuration
Memory Settings
Maximum Number of Open Files
Maximum Number of Processes or Threads
Reflective access
Firewall Exceptions
Garbage Collector for Worker
Reverse Proxy Configuration
Optional Installation Steps
Troubleshooting
Upgrading Server to Newer Version
Cloud Marketplaces
AWS Marketplace
Overview
Quickstart
Architecture
Configuration
Security
Deployment Into Existing Infrastructure
Upgrade
Azure Marketplace
Overview
Quickstart
Architecture
Configuration
Security
Deployment Into Existing Infrastructure
Upgrade
Docker and Kubernetes deployments
Configuration
Configuration Introduction
Configuration Sources
Configuration File on Specified Location
Alternative Configuration Sources
Priorities of Configuration Sources
Specifying the Path to the Configuration File
Example Configuration File
Properties Details
Setup
Before You Start
Using Setup
Configuration File
License
Database
Worker
Sandboxes
Encryption
E-Mail
LDAP
Cluster
System Database Configuration
Embedded Apache Derby
MySQL
Creating Database
CloverDX Server Setup
DB2
Creating Database
CloverDX Server Setup
DB2 on AS/400
Oracle
Creating Database
CloverDX Server Setup
Microsoft SQL Server
Creating Database
CloverDX Server Setup
PostgreSQL
Creating Database
CloverDX Server Setup
JNDI Configuration and Encryption
JNDI DB Datasource
JNDI Datasource Troubleshooting
Encrypted JNDI
List of Configuration Properties
General Configuration Properties
Worker - Configuration Properties
Worker Health Related Properties
Worker - JNDI Properties
JDBC Datasources
JMS Connections
Worker - SSL Properties
Properties on Worker’s Command Line
Job Execution Properties
List of all properties
Secure Configuration Properties
Logging
Main Logs
Logging Framework in CloverDX Server
Logs Directory
Default Logging Logic
Logging Customization
Graph Run Logs
Server Audit Logs
Designer-Server Integration Logs
Performance Log
Job Queue Log
Monitor Log
Engine Configuration
JMX Configuration
Instance Indicator
Branding of CloverDX Server
Administration
Monitoring
Monitoring (Operations Dashboard)
Overview
Quickstart
Using the Dashboard
Logs
Scenarios
API
Configuration
Limitations
Resources
Standalone Server View
Cluster View
Server Management
Status History
Server Logs
Temp Space Management
Temp Space Overview
Management
Secure Parameters
Secret Managers
Users and Groups
LDAP Authentication
Active Directory
SAML Authentication
SAML configuration examples
SAML troubleshooting
Users
User Profile
Groups
User Lockout
Sandboxes - Server Side Job Files
Referencing Files from the Graph or Jobflow
Sandbox Content Security and Permissions
Sandbox Content and Options
Download sandbox as ZIP
Upload ZIP to sandbox
Download file in ZIP
Create File
Create Directory
Upload File
Export Sandbox Configuration
Delete Sandbox
Sandbox Configuration Import
Execution Properties
OAuth2 Connections
WebDAV Access to Sandboxes
WebDAV Clients
WebDAV Authentication/Authorization
Libraries
Introduction
Development
Usage
Subgraphs
Graphs, Jobflows
Data Services
Installation
Single Library Upload
Library Repository
Installation Settings
Installation Logic
Multiple Library Instances
Overview Tab
Documentation Tab
Configuration Tab
Initialization Job
Health Check Job
Configurable Parameters
OAuth2 Connections
Dependencies
Incompatible library
Permissions Tab
Library-Level Permissions
Data Connectors Permissions
Data Connector Permissions Logic
Library Editing
Library Deleting
Library and Library Repository Configuration Exports
Library and Library Repository Configuration Import
Library Management through REST API
Wrangler
Wrangler Users
Wrangler Workspaces
Wrangler Jobs
Licensing: number of allowed Wrangler seats
Wrangler user limit exceeded
Decreasing number of used seats
Disabling Wrangler
Server Configuration Migration
Permissions for Configuration Migration
Server Configuration Export
Server Configuration Import
Environment Variables
Passwords
Support Package
Downloading Support Package
Diagnostics
Troubleshooting Worker
Worker Does Not Start
Restarting Worker
Worker does not start
Worker Crashes
Worker Hangs
Issues with Classloading
Alerts and Notification
Using Graphs
Job Queue
Overview
Quickstart
Architecture
Load Metrics
Emergency Mode
Job Queue Algorithm
Impact
Scenarios
Troubleshooting
Configuration
Limitations
Graph/Jobflow Parameters
Parameters by Execution Type
Adding Another Graph Parameters
Tasks
Send an Email
Placeholders
Execute Shell Command
Start a Graph
Start a Jobflow
Start a Library Health Check Job
Abort job
Archive Records
Send a JMS Message
Execute Groovy Code
Manual Task Execution
Scheduling
Timetable Setting
Allocations of Scheduled Task on Nodes
Scheduling the Tasks - Examples
Start a graph at specific time
Start a Jobflow once an hour
Complex Scheduling
Alerts and Notification
Viewing Job Runs - Execution History
Filtering and ordering
Job Inspector
Tracking
Log File
Job Inspector
Overview
Quickstart
Using the Job Inspector
Running Jobs Manually
Job Inspector in Separate Browser Tab
Configuration
Listeners
Graph Event Listeners
Graph Events
Listener
Tasks
Use Cases
Jobflow Event Listeners
Jobflow Events
Listener
Tasks
JMS Message Listeners
Kafka Message Listeners
Universal Event Listeners
Evaluation Criteria
File Event Listeners (remote and local)
Cluster environment
Supported filesystems and protocols
Observed file
File Events
Check Interval, Task and Use Cases
Howtos
Task Failure Listeners
Task Choice
Task Failed E-mail Template
Alerts and Notification
Extensibility - CloverDX Engine Plugins
API
Simple HTTP API
REST API
Data Services
Data Services Overview
User Interface
Endpoints
List of Data Services
Overview
Data App
State and History
Alerts and Notification
Configuration
Catalog of Services
Built-in Data Service Examples
HTTPS Connectors
Using Data Services
Deploying Data Service
Publishing and Unpublishing Data Service from Sandbox
Publishing Data Service Examples
Changing Data Service to Anonymous
Running Data Service on HTTPS
Running Data Service on HTTPS on Cluster
Monitoring Data Service
Testing Data Service
Performance Tuning
Exporting Data Service Configuration
Importing Data Service Configuration
Avoiding Premature Marking of Data Service as Failing
Looking up Particular Data Service
Resetting State of Failing Data Service Endpoint
Enabling CORS Filter
Custom HTTP Headers
Data Services on Cluster
Data Apps
Create new Data App
Using Data Apps
Data Apps catalog
Single Data App form
Responses
Debugging, Troubleshooting
Authentication, Anonymous mode
Permissions
Validation
Customizing Data Apps
Examples, How to, Best practices
SOAP WebService API
SOAP WS Client
SOAP WS API Authentication/Authorization
Cluster
Sandboxes in Cluster
Shared Sandbox
Local Sandbox
Partitioned Sandbox
Using a Sandbox Resource as a Component Data Source
Remote Edges
Cluster Configuration
Mandatory Cluster Properties
Optional Cluster Properties
Firewall Exceptions
Example of 2-Node Cluster Configuration
Basic 2-Node Cluster Configuration
A 2-Node Cluster with Load Balancer
Example of 3-Node Cluster Configuration
Jobs Load Balancing Properties
A 2-Node Cluster Load Balancing Example
Cluster Load Balancer Debugging
Running More Clusters
Recommendations for Cluster Deployment
Troubleshooting
NodeA Cannot Establish HTTP Connection to NodeB
NodeA Cannot Establish TCP Connection (Port 7800 by Default) to NodeB
NodeB is Killed or It Cannot Connect to the Database
Node cannot access the sandboxes home directory
Auto-Resuming in Unreliable Network
Long-Term Network Malfunction May Cause Jobs to Hang on
Security
Security Recommendations for CloverDX Server
Version
CloverDX Server
>
Introduction
Introduction
1. What is CloverDX Server?
2. CloverDX Server Architecture
3. CloverDX Core
4. CloverDX Worker
5. CloverDX Cluster
