Version

    Automation

    The Automation section is intended to help you leverage the full power of CloverDX’s automation capabilities, focusing on automating jobs developed using CloverDX Designer. By automating these jobs and other processes, you can streamline data workflows, reduce manual intervention, and increase the efficiency and reliability of your data operations.

    This section covers several key areas of automation in CloverDX:

    • Tasks: This section provides a comprehensive overview of various tasks that can be set up within CloverDX Server to enhance workflow automation. Tasks include sending emails based on specific triggers, executing shell commands for system-level automation, starting graphs and jobflows to process data according to pre-defined logic, and conducting library health checks to ensure that libraries are functioning correctly. It also covers critical operations like aborting jobs based on conditions, archiving or deleting records like task history or obsolete log files to manage the temp disk space size, sending JMS messages to integrate with messaging systems, and executing Groovy code for custom scripting needs. These tasks are the building blocks of CloverDX automation, allowing for robust and flexible job management.

    • Scheduling: This section delves into the details of scheduling tasks within CloverDX, which is essential for ensuring that jobs run automatically at specified times or intervals without manual intervention. It explains how to configure timetable settings to define when tasks should execute, manage the allocation of scheduled tasks across multiple nodes for balanced workload distribution, and create schedules that fit your operational needs with practical examples. Additionally, it addresses how to set up alerts and notifications to keep you informed about the status and results of your scheduled tasks, helping you maintain control and oversight over automated processes.

    • Listeners: Listeners are a powerful feature in CloverDX that allows for event-driven automation. This section covers various types of listeners that can be configured to automatically respond to specific events. For example, Graph Event Listeners and Jobflow Event Listeners can trigger actions based on the start or completion of selected graphs or jobflows. JMS and Kafka Message Listeners enable integration with messaging systems, triggering workflows upon receiving new messages. Universal Event Listeners provide a flexible mechanism for handling a wide range of events, while File Event Listeners (both remote and local) automate processes based on file system changes. Task Failure Listeners specifically monitor for errors of other listeners and allow for automatic remediation or notification actions.

    • Manual task execution: While automation typically implies scheduled or event-driven actions, there are times when you need to execute tasks manually. This section explains how manual task execution can be set up and utilized in CloverDX, providing the flexibility to run tasks on-demand when immediate intervention is required. This capability is useful for testing, debugging, or handling special cases that fall outside the routine automation schedule.

    • Data Services: Data Services in CloverDX allow users to automate data operations through API endpoints developed in CloverDX Designer as Data Service .rjob files, making it possible to integrate CloverDX capabilities into broader IT systems and workflows. This section covers everything you need to know about deploying, and managing Data Services within CloverDX. It provides an overview of the user interface in CloverDX Server console, including how to manage endpoints, view and configure available Data Services, monitor service state and history, and set up alerts and notifications.

      The section also details the steps for deploying Data Services, publishing and unpublishing them from sandboxes, or running them over HTTPS. You will find information on monitoring and testing services, performance tuning, and troubleshooting, including handling custom HTTP headers and enabling CORS filters for cross-origin requests.

    • Data Apps: Data Apps are customizable applications within CloverDX that provide a user-friendly way to interact with data processes. Data Apps are developed as Data Services .rjob files. This section covers creating and managing a catalog of Data Apps, designing single-form interfaces for specific data operations, and handling different types of responses. It also includes information on authentication and permissions to control user access, and setting up alerts and notifications to monitor app usage and performance.