This section provides an in-depth guide for developers and advanced users who are looking to optimize and extend the capabilities of CloverDX. This section covers a range of topics that are essential for building robust, efficient, and scalable data transformations and integrations.
In Library development best practices, we offer comprehensive guidelines on creating and managing CloverDX libraries. A CloverDX library is a redistributable package, which can contain reusable graphs, subgraphs, jobflows, data services, or metadata. This chapter covers the fundamentals, such as library file structure and usage, and progresses to more complex topics like enhancing library aesthetics, managing parameters, and handling user credentials and secrets. It also provides instructions for establishing database/JDBC and OAuth2 connections, managing library dependencies, and integrating Java code. Additionally, this chapter includes best practices for live debugging and testing, job documentation, and the development of both Data Source and Data Target Connectors, ensuring that developers can create reusable, maintainable, and secure libraries.
The Data partitioning (parallel running) section, focuses on techniques for optimizing data processing performance through different kinds of parallelism. This chapter explains the concept of data partitioning and introduces the use of parallel components to enhance data throughput. It delves into the specifics of parallel processing on both standalone Server environments and Server clusters. Practical examples are provided to illustrate distributed execution and demonstrate the scalability of partitioned transformations. The chapter also discusses the use of partitioned sandboxes in a Server cluster environment to further optimize performance.
The Shared lookup tables in CloverDX Server section provides information about how you can create shared lookups that are available in the Data Manager.