This chapter provides a comprehensive guide to the various components available within CloverDX for building and executing data transformation jobs. It provides information essential for understanding how to effectively use each component to build robust and scalable data workflows. Each component plays a crucial role in facilitating data integration, manipulation, and processing tasks, allowing users to design complex workflows with flexibility and efficiency.
-
Readers focus on extracting data from various sources.
-
Writers are used for outputting transformed data to different destinations, including databases, files, and external systems.
-
Transformers handle data manipulation tasks, applying transformations to datasets to cleanse, aggregate, or enrich data.
-
Joiners enable merging of datasets from multiple sources, facilitating relational operations across different data inputs.
-
Job Control components allow for the orchestration of complex workflows, providing tools to manage dependencies, triggers, and job execution.
-
File Operations components offer functionalities for interacting with file systems, handling file management tasks like moving, copying, or deleting files.
-
Data Partitioning components optimize performance by splitting large datasets into manageable parts for parallel processing.
-
Data Quality components are designed to ensure that data meets predefined quality standards, helping identify and correct anomalies or errors.
-
Others cover additional utility components that enhance the functionality of CloverDX.
-
Deprecated lists components that are still available but no longer recommended for use, usually because they have been replaced by more advanced alternatives.
-
Incubation showcases components currently in development or experimental stages, providing a glimpse into future features.