Graph Allocation Examples

Basic component allocation

This example shows a graph with two components, where allocation ensures that the first component will be executed on cluster node1 and the second component will be executed on cluster node2.

Basic component allocation with remote data transfer

Two components connected with an edge can have a different allocation. The first is executed on node1 and the second is executed on node2. Cluster environment automatically ensures remote data records transfer.

Multiple execution

A graph with a multiple node allocation is executed in parallel. In this example, both components have same allocation, so three identical transformations will be executed on cluster node1, node2 and node3.

Cluster data partitioning

A graph with two allocations. The first component has a single node allocation, which is not specified and is automatically derived to ensure a minimal number of remote edges. The ParallelPartition component distribute records for further data processing on the cluster node1, node2 and node3.

Cluster data gathering

A graph with two allocations. Resulted data records of parallel data processing in the first component are collected in the ParallelSimpleGather component and passed to the cluster node4 for further single node processing.