CloverDX Server

User’s Guide

This User’s Guide refers to CloverDX Server 6.0.0 release.

Copyright © 2010-2023 CloverDX a.s. All rights reserved.

Do not copy or distribute without express permission of CloverDX a.s.

Feedback welcome:

If you have any comments or suggestions for this documentation, please send them by email to support@cloverdx.com.

Consider How to speed up communication with CloverCARE support before contacting the support.

Introduction
1. What is CloverDX Server?
2. CloverDX Server Architecture
3. CloverDX Core
4. CloverDX Worker
5. CloverDX Cluster
Installation
6. System Requirements
7. Installing
Evaluation Server
Installation
Production Server
Apache Tomcat
Activation
CloverDX Server Activation Alternatives
IBM InfoSphere MDM Plugin Installation
Installation into Server
8. Postinstallation Configuration
Memory Settings
Maximum Number of Open Files
Maximum Number of Processes or Threads
Firewall Exceptions
Garbage Collector for Worker
Reverse Proxy Configuration
9. Optional Installation Steps
Adding Libraries to the Worker’s Classpath
10. Troubleshooting
11. Upgrade
12. Cloud Marketplaces
AWS Marketplace
Overview
Quickstart
Architecture
Configuration
Security
Deployment Into Existing Infrastructure
Upgrade
Azure Marketplace
Overview
Quickstart
Architecture
Configuration
Security
Deployment Into Existing Infrastructure
Upgrade
13. CentOS/RHEL RPM package
Configuration
14. Configuration Introduction
15. Configuration Sources
Configuration File on Specified Location
Alternative Configuration Sources
Priorities of Configuration Sources
Specifying the Path to the Configuration File
16. Setup
Before You Start
Using Setup
Configuration File
License
Database
Worker
Sandboxes
Encryption
E-Mail
LDAP
Cluster
17. System Database Configuration
Embedded Apache Derby
MySQL
Creating Database
CloverDX Server Setup
DB2
Creating Database
CloverDX Server Setup
DB2 on AS/400
Oracle
Creating Database
CloverDX Server Setup
Microsoft SQL Server
Creating Database
CloverDX Server Setup
PostgreSQL
Creating Database
CloverDX Server Setup
JNDI Configuration and Encryption
JNDI DB Datasource
JNDI Datasource Troubleshooting
Encrypted JNDI
18. List of Configuration Properties
General Configuration Properties
Worker - Configuration Properties
Worker Health Related Properties
Worker - JNDI Properties
JDBC Datasources
JMS Connections
Worker - SSL Properties
Properties on Worker’s Command Line
Job Execution Properties
List of all properties
19. Secure Configuration Properties
20. Logging
Main Logs
Graph Run Logs
Server Audit Logs
Designer-Server Integration Logs
Performance Log
Job Queue Log
Monitor Log
21. Engine Configuration
22. Instance Indicator
Administration
23. Monitoring
Monitoring (Operations Dashboard)
Overview
Quickstart
Using the Dashboard
Logs
Scenarios
API
Configuration
Limitations
Resources
Standalone Server View
Cluster View
Server Management
Status History
Server Logs
24. Temp Space Management
Temp Space Overview
Management
25. Secure Parameters
26. Secret Managers
27. Users and Groups
LDAP Authentication
Active Directory
SAML Authentication
SAML configuration examples
SAML troubleshooting
Users
Groups
User Lockout
28. Sandboxes - Server Side Job Files
Referencing Files from the Graph or Jobflow
Sandbox Content Security and Permissions
Sandbox Content and Options
Download sandbox as ZIP
Upload ZIP to sandbox
Download file in ZIP
Create File
Create Directory
Upload File
Export Sandbox Configuration
Delete Sandbox
Sandbox Configuration Import
Execution Properties
OAuth2 Connections
WebDAV Access to Sandboxes
WebDAV Clients
WebDAV Authentication/Authorization
29. Libraries
Development and Installation
Usage
Graphs, Jobflows
Subgraphs
Data Connectors
Data Services
Metadata
Updating a Library
Library Documentation
Library Overview
Library Editing
Library Configuration
Library Repository
Library Export
Library Import
30. Wrangler
Wrangler Users
Wrangler Workspaces
Wrangler Jobs
Licensing
31. Server Configuration Migration
Server Configuration Export
Server Configuration Import
Environment Variables
Passwords
32. Upgrading Server to Newer Version
33. Support Package
34. Diagnostics
35. Troubleshooting Worker
36. Alerts and Notification
Using Graphs
37. Job Queue
Overview
Quickstart
Architecture
Load Metrics
Emergency Mode
Job Queue Algorithm
Impact
Scenarios
Troubleshooting
Configuration
Limitations
38. Graph/Jobflow Parameters
Parameters by Execution Type
Adding Another Graph Parameters
39. Tasks
Send an Email
Placeholders
Execute Shell Command
Start a Graph
Start a Jobflow
Abort job
Archive Records
Send a JMS Message
Execute Groovy Code
40. Manual Task Execution
41. Scheduling
Timetable Setting
Allocations of Scheduled Task on Nodes
Scheduling the Tasks - Examples
Start a graph at specific time
Start a Jobflow once an hour
Complex Scheduling
Alerts and Notification
42. Viewing Job Runs - Execution History
Filtering and ordering
Job Inspector
Tracking
Log File
43. Job Inspector
Overview
Quickstart
Using the Job Inspector
Running Jobs Manually
Job Inspector in Separate Browser Tab
Configuration
44. Listeners
Graph Event Listeners
Graph Events
Listener
Tasks
Use Cases
Jobflow Event Listeners
Jobflow Events
Listener
Tasks
JMS Message Listeners
Kafka Message Listeners
Universal Event Listeners
Evaluation Criteria
File Event Listeners (remote and local)
Cluster environment
Supported filesystems and protocols
Observed file
File Events
Check Interval, Task and Use Cases
Howtos
Task Failure Listeners
Task Choice
Task Failed E-mail Template
Alerts and Notification
45. Recommendations for Transformations Developers
46. Extensibility - CloverDX Engine Plugins
47. Troubleshooting
API
48. Simple HTTP API
49. REST API
Overview
50. Data Services
Data Services Overview
User Interface
Endpoints
List of Data Services
Overview
Data App
State and History
Alerts and Notification
Configuration
Catalog of Services
Built-in Data Service Examples
HTTPS Connectors
Using Data Services
Deploying Data Service
Publishing and Unpublishing Data Service from Sandbox
Publishing Data Service Examples
Changing Data Service to Anonymous
Running Data Service on HTTPS
Running Data Service on HTTPS on Cluster
Monitoring Data Service
Testing Data Service
Performance Tuning
Exporting Data Service Configuration
Importing Data Service Configuration
Avoiding Premature Marking of Data Service as Failing
Looking up Particular Data Service
Resetting State of Failing Data Service Endpoint
Enabling CORS Filter
Custom HTTP Headers
Data Services on Cluster
51. Data Apps
Create new Data App
Using Data Apps
Validation
Customizing Data Apps
Examples, How to, Best practices
Branding of Data Apps
52. SOAP WebService API
SOAP WS Client
SOAP WS API Authentication/Authorization
53. JMX mBean
JMX Configuration
How to configure JMX on Apache Tomcat
Possible Problems
Operations
Cluster
54. Sandboxes in Cluster
Using a Sandbox Resource as a Component Data Source
Remote Edges
55. Cluster Configuration
Mandatory Cluster Properties
Optional Cluster Properties
Firewall Exceptions
Example of 2 Node Cluster Configuration
Basic 2-nodes Cluster Configuration
2-nodes Cluster with Load Balancer
Example of 3 Node Cluster Configuration
Basic 3-nodes Cluster Configuration
Jobs Load Balancing Properties
Running More Clusters
56. Recommendations for Cluster Deployment
57. Troubleshooting
NodeA Cannot Establish HTTP Connection to NodeB
NodeA Cannot Establish TCP Connection (Port 7800 by Default) to NodeB
NodeB is Killed or It Cannot Connect to the Database
Node cannot access the sandboxes home directory
Auto-Resuming in Unreliable Network
Long-Term Network Malfunction May Cause Jobs to Hang on
Security
58. Security Recommendations for CloverDX Server