Version

    CloverDX Server

    User's Guide

    This User's Guide refers to CloverDX Server 5.11.x release.

    Copyright © 2010-2022 CloverDX a.s. All rights reserved.

    Do not copy or distribute without express permission of CloverDX a.s.

    Feedback welcome:

    If you have any comments or suggestions for this documentation, please send them by email to .

    Consider How to speed up communication with CloverCARE support before contacting the support.

    Table of Contents

    I. Introduction
    1. What is CloverDX Server?
    2. CloverDX Server Architecture
    3. CloverDX Core
    4. CloverDX Worker
    5. CloverDX Cluster
    II. Installation
    6. System Requirements
    7. Installing
    Evaluation Server
    Installation
    Production Server
    Apache Tomcat
    IBM WebSphere
    JBoss Enterprise Application Platform
    Oracle WebLogic Server
    Activation
    CloverDX Server Activation Alternatives
    IBM InfoSphere MDM Plugin Installation
    Installation into Server
    8. Postinstallation Configuration
    Memory Settings
    Maximum Number of Open Files
    Maximum Number of Processes or Threads
    Firewall Exceptions
    Garbage Collector for Worker
    Reverse Proxy Configuration
    9. Optional Installation Steps
    Adding Libraries to the Worker's Classpath
    10. Troubleshooting
    11. Upgrade
    12. Cloud Marketplaces
    AWS Marketplace
    Overview
    Quickstart
    Architecture
    Configuration
    Security
    Deployment Into Existing Infrastructure
    Upgrade
    Azure Marketplace
    Overview
    Quickstart
    Architecture
    Configuration
    Security
    Deployment Into Existing Infrastructure
    Upgrade
    III. Configuration
    13. Configuration Introduction
    14. Configuration Sources
    Configuration File on Specified Location
    Alternative Configuration Sources
    Priorities of Configuration Sources
    Specifying the Path to the Configuration File
    15. Setup
    Before You Start
    Using Setup
    Configuration File
    License
    Database
    Worker
    Sandboxes
    Encryption
    E-Mail
    LDAP
    Cluster
    16. System Database Configuration
    Embedded Apache Derby
    MySQL
    Creating Database
    CloverDX Server Setup
    DB2
    Creating Database
    CloverDX Server Setup
    DB2 on AS/400
    Oracle
    Creating Database
    CloverDX Server Setup
    Microsoft SQL Server
    Creating Database
    CloverDX Server Setup
    PostgreSQL
    Creating Database
    CloverDX Server Setup
    JNDI Configuration and Encryption
    JNDI DB Datasource
    JNDI Datasource Troubleshooting
    Encrypted JNDI
    17. List of Configuration Properties
    General Configuration Properties
    Worker - Configuration Properties
    Worker Health Related Properties
    Worker - JNDI Properties
    JDBC Datasources
    JMS Connections
    Worker - SSL Properties
    Properties on Worker's Command Line
    Job Execution Properties
    List of all properties
    18. Secure Configuration Properties
    19. Logging
    Main Logs
    Graph Run Logs
    Server Audit Logs
    Designer-Server Integration Logs
    Performance Log
    Job Queue Log
    Monitor Log
    20. Engine Configuration
    IV. Administration
    21. Operations Dashboard
    Overview
    Quickstart
    Using the Dashboard
    Logs
    Scenarios
    API
    Configuration
    Limitations
    22. Monitoring
    Resources
    Cluster Overview
    Node Detail
    Server Logs
    Using the Monitoring
    Restarting the Worker
    Showing Worker's Command Line Arguments
    Suspending the Server
    Resuming the Server
    Displaying List of Threads of the Server Core
    Dismissing Warning Messages
    23. Temp Space Management
    Temp Space Overview
    Management
    24. Secure Parameters
    25. Users and Groups
    LDAP Authentication
    Active Directory
    SAML Authentication
    Users
    Groups
    User Lockout
    26. Sandboxes - Server Side Job Files
    Referencing Files from the Graph or Jobflow
    Sandbox Content Security and Permissions
    Sandbox Content and Options
    Download sandbox as ZIP
    Upload ZIP to sandbox
    Download file in ZIP
    Create File
    Create Directory
    Upload File
    Export Sandbox Configuration
    Delete Sandbox
    Sandbox Configuration Import
    Execution Properties
    WebDAV Access to Sandboxes
    WebDAV Clients
    WebDAV Authentication/Authorization
    27. Server Configuration Migration
    Server Configuration Export
    Server Configuration Import
    Environment Variables
    Passwords
    28. Upgrading Server to Newer Version
    29. Support Package
    30. Diagnostics
    31. Troubleshooting Worker
    32. Alerts and Notification
    V. Using Graphs
    33. Job Queue
    Overview
    Quickstart
    Architecture
    Load Metrics
    Emergency Mode
    Job Queue Algorithm
    Impact
    Scenarios
    Troubleshooting
    Configuration
    Limitations
    34. Graph/Jobflow Parameters
    Parameters by Execution Type
    Adding Another Graph Parameters
    35. Tasks
    Send an Email
    Placeholders
    Execute Shell Command
    Start a Graph
    Start a Jobflow
    Start a Profiler Job
    Abort job
    Archive Records
    Send a JMS Message
    Execute Groovy Code
    36. Manual Task Execution
    37. Scheduling
    Timetable Setting
    Allocations of Scheduled Task on Nodes
    Scheduling the Tasks - Examples
    Start a graph at specific time
    Start a Jobflow once an hour
    Complex Scheduling
    Alerts and Notification
    38. Viewing Job Runs - Execution History
    Filtering and ordering
    Job Inspector
    Tracking
    Log File
    39. Job Inspector
    Overview
    Quickstart
    Using the Job Inspector
    Configuration
    40. Listeners
    Graph Event Listeners
    Graph Events
    Listener
    Tasks
    Use Cases
    Jobflow Event Listeners
    Jobflow Events
    Listener
    Tasks
    JMS Message Listeners
    Kafka Message Listeners
    Universal Event Listeners
    Evaluation Criteria
    File Event Listeners (remote and local)
    Cluster environment
    Supported filesystems and protocols
    Observed file
    File Events
    Check Interval, Task and Use Cases
    Howtos
    Task Failure Listeners
    Task Choice
    Task Failed E-mail Template
    Alerts and Notification
    41. Recommendations for Transformations Developers
    42. Extensibility - CloverDX Engine Plugins
    43. Troubleshooting
    VI. API
    44. Simple HTTP API
    45. REST API
    Overview
    46. Data Services
    Data Services Overview
    User Interface
    List of Data Services
    Overview
    Data App
    State and History
    Alerts and Notification
    Configuration
    Catalog of Services
    Built-in Data Service Examples
    HTTPS Connectors
    Using Data Services
    Deploying Data Service
    Publishing and Unpublishing Data Service from Sandbox
    Publishing Data Service Examples
    Changing Data Service to Anonymous
    Running Data Service on HTTPS
    Running Data Service on HTTPS on Cluster
    Monitoring Data Service
    Testing Data Service
    Performance Tuning
    Exporting Data Service Configuration
    Importing Data Service Configuration
    Avoiding Premature Marking of Data Service as Failing
    Looking up Particular Data Service
    Resetting State of Failing Data Service Endpoint
    Enabling CORS Filter
    Custom HTTP Headers
    Data Services on Cluster
    47. Data Apps
    Create new Data App
    Using Data Apps
    Validation
    Customizing Data Apps
    Examples, How to, Best practices
    Branding of Data Apps
    48. SOAP WebService API
    SOAP WS Client
    SOAP WS API Authentication/Authorization
    49. JMX mBean
    JMX Configuration
    How to configure JMX on Apache Tomcat
    How to Configure JMX on WebSphere
    Possible Problems
    Operations
    VII. Cluster
    50. Sandboxes in Cluster
    Using a Sandbox Resource as a Component Data Source
    Remote Edges
    51. Cluster Configuration
    Mandatory Cluster Properties
    Optional Cluster Properties
    Firewall Exceptions
    Example of 2 Node Cluster Configuration
    Basic 2-nodes Cluster Configuration
    2-nodes Cluster with Proxied Access to Database
    2-nodes Cluster with Load Balancer
    Example of 3 Node Cluster Configuration
    Basic 3-nodes Cluster Configuration
    Jobs Load Balancing Properties
    Running More Clusters
    52. Recommendations for Cluster Deployment
    53. Troubleshooting
    NodeA Cannot Establish HTTP Connection to NodeB
    NodeA Cannot Establish TCP Connection (Port 7800 by Default) to NodeB
    NodeB is Killed or It Cannot Connect to the Database
    Auto-Resuming in Unreliable Network
    Long-Term Network Malfunction May Cause Jobs to Hang on
    VIII. Security
    54. Security Recommendations for CloverDX Server