Version

    CloverDX Server

    User’s Guide

    This User’s Guide refers to CloverDX Server 6.3.0 release.

    Copyright © 2010-2023 CloverDX a.s. All rights reserved.

    Do not copy or distribute without express permission of CloverDX a.s.

    http://www.cloverdx.com

    Feedback welcome:

    If you have any comments or suggestions for this documentation, please send them by email to support@cloverdx.com.

    Consider How to speed up communication with CloverCARE support before contacting the support.

    Introduction
    1. What is CloverDX Server?
    2. CloverDX Server Architecture
    3. CloverDX Core
    4. CloverDX Worker
    5. CloverDX Cluster
    Installation
    6. System Requirements
    7. Installing
    Evaluation Server
    Installation
    Production Server
    Apache Tomcat
    Activation
    CloverDX Server Activation Alternatives
    IBM InfoSphere MDM Plugin Installation
    Installation into Server
    8. Postinstallation Configuration
    Memory Settings
    Maximum Number of Open Files
    Maximum Number of Processes or Threads
    Reflective access
    Firewall Exceptions
    Garbage Collector for Worker
    Reverse Proxy Configuration
    9. Optional Installation Steps
    10. Troubleshooting
    11. Upgrading Server to Newer Version
    12. Cloud Marketplaces
    AWS Marketplace
    Overview
    Quickstart
    Architecture
    Configuration
    Security
    Deployment Into Existing Infrastructure
    Upgrade
    Azure Marketplace
    Overview
    Quickstart
    Architecture
    Configuration
    Security
    Deployment Into Existing Infrastructure
    Upgrade
    13. CentOS/RHEL RPM package
    Overview
    Prerequisites
    Architecture
    Apache Tomcat 9
    CloverDX Server
    Installation
    Method 1: Fully automated (recommended)
    Method 2: Separate Java installation
    Post-installation checklist
    Configuration
    System Database Configuration
    Libraries and Classpath
    Sandboxes
    Memory
    License
    CloverDX Services
    Server logs
    JMX monitoring
    Upgrading CloverDX
    Operating systems without SystemD
    Configuration
    14. Configuration Introduction
    15. Configuration Sources
    Configuration File on Specified Location
    Alternative Configuration Sources
    Priorities of Configuration Sources
    Specifying the Path to the Configuration File
    16. Example Configuration File
    Properties Details
    17. Setup
    Before You Start
    Using Setup
    Configuration File
    License
    Database
    Worker
    Sandboxes
    Encryption
    E-Mail
    LDAP
    Cluster
    18. System Database Configuration
    Embedded Apache Derby
    MySQL
    Creating Database
    CloverDX Server Setup
    DB2
    Creating Database
    CloverDX Server Setup
    DB2 on AS/400
    Oracle
    Creating Database
    CloverDX Server Setup
    Microsoft SQL Server
    Creating Database
    CloverDX Server Setup
    PostgreSQL
    Creating Database
    CloverDX Server Setup
    JNDI Configuration and Encryption
    JNDI DB Datasource
    JNDI Datasource Troubleshooting
    Encrypted JNDI
    19. List of Configuration Properties
    General Configuration Properties
    Worker - Configuration Properties
    Worker Health Related Properties
    Worker - JNDI Properties
    JDBC Datasources
    JMS Connections
    Worker - SSL Properties
    Properties on Worker’s Command Line
    Job Execution Properties
    List of all properties
    20. Secure Configuration Properties
    21. Logging
    Main Logs
    Graph Run Logs
    Server Audit Logs
    Designer-Server Integration Logs
    Performance Log
    Job Queue Log
    Monitor Log
    22. Engine Configuration
    23. Instance Indicator
    Administration
    24. Monitoring
    Monitoring (Operations Dashboard)
    Overview
    Quickstart
    Using the Dashboard
    Logs
    Scenarios
    API
    Configuration
    Limitations
    Resources
    Standalone Server View
    Cluster View
    Server Management
    Status History
    Server Logs
    25. Temp Space Management
    Temp Space Overview
    Management
    26. Secure Parameters
    27. Secret Managers
    28. Users and Groups
    LDAP Authentication
    Active Directory
    SAML Authentication
    SAML configuration examples
    SAML troubleshooting
    Users
    User Profile
    Groups
    User Lockout
    29. Sandboxes - Server Side Job Files
    Referencing Files from the Graph or Jobflow
    Sandbox Content Security and Permissions
    Sandbox Content and Options
    Download sandbox as ZIP
    Upload ZIP to sandbox
    Download file in ZIP
    Create File
    Create Directory
    Upload File
    Export Sandbox Configuration
    Delete Sandbox
    Sandbox Configuration Import
    Execution Properties
    OAuth2 Connections
    WebDAV Access to Sandboxes
    WebDAV Clients
    WebDAV Authentication/Authorization
    30. Libraries
    Introduction
    Development
    Usage
    Subgraphs
    Graphs, Jobflows
    Data Services
    Installation
    Single Library Upload
    Library Repository
    Installation Settings
    Installation Logic
    Multiple Library Instances
    Overview Tab
    Documentation Tab
    Configuration Tab
    Initialization Job
    Health Check Job
    Configurable Parameters
    OAuth2 Connections
    Dependencies
    Permissions Tab
    Library-Level Permissions
    Data Connectors Permissions
    Data Connector Permissions Logic
    Library Editing
    Library Deleting
    Library and Library Repository Configuration Exports
    Library and Library Repository Configuration Import
    31. Wrangler
    Wrangler Users
    Wrangler Workspaces
    Wrangler Jobs
    Licensing: number of allowed Wrangler seats
    Wrangler user limit exceeded
    Decreasing number of used seats
    Disabling Wrangler
    32. Server Configuration Migration
    Permissions for Configuration Migration
    Server Configuration Export
    Server Configuration Import
    Environment Variables
    Passwords
    33. Support Package
    34. Diagnostics
    35. Troubleshooting Worker
    Worker Does Not Start
    Restarting Worker
    Worker does not start
    Worker Crashes
    Worker Hangs
    Issues with Classloading
    36. Alerts and Notification
    Using Graphs
    37. Job Queue
    Overview
    Quickstart
    Architecture
    Load Metrics
    Emergency Mode
    Job Queue Algorithm
    Impact
    Scenarios
    Troubleshooting
    Configuration
    Limitations
    38. Graph/Jobflow Parameters
    Parameters by Execution Type
    Adding Another Graph Parameters
    39. Tasks
    Send an Email
    Placeholders
    Execute Shell Command
    Start a Graph
    Start a Jobflow
    Start a Library Health Check Job
    Abort job
    Archive Records
    Send a JMS Message
    Execute Groovy Code
    40. Manual Task Execution
    41. Scheduling
    Timetable Setting
    Allocations of Scheduled Task on Nodes
    Scheduling the Tasks - Examples
    Start a graph at specific time
    Start a Jobflow once an hour
    Complex Scheduling
    Alerts and Notification
    42. Viewing Job Runs - Execution History
    Filtering and ordering
    Job Inspector
    Tracking
    Log File
    43. Job Inspector
    Overview
    Quickstart
    Using the Job Inspector
    Running Jobs Manually
    Job Inspector in Separate Browser Tab
    Configuration
    44. Listeners
    Graph Event Listeners
    Graph Events
    Listener
    Tasks
    Use Cases
    Jobflow Event Listeners
    Jobflow Events
    Listener
    Tasks
    JMS Message Listeners
    Kafka Message Listeners
    Universal Event Listeners
    Evaluation Criteria
    File Event Listeners (remote and local)
    Cluster environment
    Supported filesystems and protocols
    Observed file
    File Events
    Check Interval, Task and Use Cases
    Howtos
    Task Failure Listeners
    Task Choice
    Task Failed E-mail Template
    Alerts and Notification
    45. Recommendations for Transformations Developers
    46. Extensibility - CloverDX Engine Plugins
    47. Troubleshooting
    API
    48. Simple HTTP API
    49. REST API
    50. Data Services
    Data Services Overview
    User Interface
    Endpoints
    List of Data Services
    Overview
    Data App
    State and History
    Alerts and Notification
    Configuration
    Catalog of Services
    Built-in Data Service Examples
    HTTPS Connectors
    Using Data Services
    Deploying Data Service
    Publishing and Unpublishing Data Service from Sandbox
    Publishing Data Service Examples
    Changing Data Service to Anonymous
    Running Data Service on HTTPS
    Running Data Service on HTTPS on Cluster
    Monitoring Data Service
    Testing Data Service
    Performance Tuning
    Exporting Data Service Configuration
    Importing Data Service Configuration
    Avoiding Premature Marking of Data Service as Failing
    Looking up Particular Data Service
    Resetting State of Failing Data Service Endpoint
    Enabling CORS Filter
    Custom HTTP Headers
    Data Services on Cluster
    51. Data Apps
    Create new Data App
    Using Data Apps
    Data Apps catalog
    Single Data App form
    Responses
    Debugging, Troubleshooting
    Authentication, Anonymous mode
    Permissions
    Validation
    Customizing Data Apps
    Examples, How to, Best practices
    Branding of Data Apps
    52. SOAP WebService API
    SOAP WS Client
    SOAP WS API Authentication/Authorization
    53. JMX mBean
    JMX Configuration
    How to configure JMX on Apache Tomcat
    Possible Problems
    Operations
    Cluster
    54. Sandboxes in Cluster
    Shared Sandbox
    Local Sandbox
    Partitioned Sandbox
    Using a Sandbox Resource as a Component Data Source
    Remote Edges
    55. Cluster Configuration
    Mandatory Cluster Properties
    Optional Cluster Properties
    Firewall Exceptions
    Example of 2-Node Cluster Configuration
    Basic 2-Node Cluster Configuration
    A 2-Node Cluster with Load Balancer
    Example of 3-Node Cluster Configuration
    Jobs Load Balancing Properties
    A 2-Node Cluster Load Balancing Example
    Cluster Load Balancer Debugging
    Running More Clusters
    56. Recommendations for Cluster Deployment
    57. Troubleshooting
    NodeA Cannot Establish HTTP Connection to NodeB
    NodeA Cannot Establish TCP Connection (Port 7800 by Default) to NodeB
    NodeB is Killed or It Cannot Connect to the Database
    Node cannot access the sandboxes home directory
    Auto-Resuming in Unreliable Network
    Long-Term Network Malfunction May Cause Jobs to Hang on
    Security
    58. Security Recommendations for CloverDX Server