Version
26. Graph Tracking
The CloverDX engine provides various tracking information about running graphs. The most important information is used to populate the Tracking view, located on bottom of the CloverDX perspective (see the designer’s tabs).
The same source of data is used for displaying decorations on graph elements. The number of transferred records appears along the edges of a running graph. The phase edges have two numbers, the left end of the edge shows the number of data records sent to the edge, and the right end of the edge shows the number of data records already read from the phase edge.
Figure 124. Edge tracking example
If the graph is running in the CloverDX Cluster environment with a multi-worker allocation, the in-graph tracking information can go into even more detail. Each component displays the number of instances of the component, i.e. parallel executions. Tracking information on edges is available in three levels of detail - low, medium and high. The level can be changed in . Or press 'D' to iterate over all levels of tracking details directly in the graph editor.
-
The low level of tracking detail shows the total number of transferred records over all workers/partitions.
-
The medium level shows the total number of transferred records as well as additional drill down information - the number of passed records and skew for each processing partition.
Figure 125. An example of a medium level of tracking informationThe example above shows a simple Clustered graph with a medium level of tracking information. The DataGenerator and ParallelPartition components are executed on a single worker, so the interconnecting edge is decorated only by the total number of transferred records. On the output side of ParallelPartition component, there is a partitioned edge, since the SimpleCopy component is executed three times. The label above this edge shows that 30% of the data records are sent to one instance of SimpleCopy and 34% to the other two instances.
-
The high level shows the most detailed information - the number of transferred records and Cluster node names where the partition is running (for example 'node1: 250 123'). Partitions where the edge is remote, the source Cluster node and target Cluster node are shown (for example 'node1~node2: 250 123').
Figure 126. An example of a high level tracking informationThe example above shows a simple Clustered graph with a high level of tracking information. The ParallelPartition component sends data to three different instances of the SimpleCopy component. The first instance runs on the same worker as the ParallelPartition component, so no Remote Edges are necessary (34,477 records have been transferred locally). The second and third instance run on different workers (and even different Cluster nodes). So 34,646 records have been moved from node1 to node2 and 30,877 records have been transferred to node3.