Version

    Chapter 44. Developing Subgraphs

    There are two ways how to create a subgraph.

    Wrapping

    Subgraph wrapping is a way to convert a section of existing graph into a subgraph. Wizard will let you copy additional graph elements (metadata, connections, lookups) from a parental graph to subgraph.

    Wrapping in Steps
    1. Select components you would like to move into a new subgraph.

    Original graph without subgraphs

    Figure 44.1. Original graph without subgraphs


    1. Use the right mouse button and choose Wrap As Subgraph.

    Wrapping components into a subgraph

    Figure 44.2. Wrapping components into a subgraph


    1. Enter the name of the subgraph file (*.sgrf) and order of its input and output ports.

    Wrapping Subgraph Wizard

    Figure 44.3. Wrapping Subgraph Wizard


    1. A new Subgraph component replaced the wrapped components in the parent graph.

    CloverDX Graph with the Subgraph Component

    Figure 44.4. CloverDX Graph with the Subgraph Component


    Continue with Making Subgraph Configurable.

    Creating from Scratch

    A new subgraph can be created from scratch. It has an initial structure - it contains a debug input component, SubgraphInput and SubgraphOutput components and a sample of the subgraph body. The initial structure is a template to help you design the subgraph.

    1. Choose in the main menu FileNewSubgraph.

    A new Subgraph

    Figure 44.5. A new Subgraph


    1. Design the subgraph body - implement the subgraph's logic in the central body section of the subgraph, using components, other subgraphs, etc.

    2. Connect the subgraph body with the SubgraphInput and SubgraphOutput components.

    Continue with Making Subgraph Configurable.

    Making Subgraph Configurable

    Optional Ports
    Color of Subgraph
    Icon of Subgraph

    Each attribute of a component in a subgraph can be exposed as an attribute of corresponding subgraph component using public parameter. It allows you to develop more generic subgraphs.

    Example 44.1. Using public parameter

    You have a subgraph filtering and aggregating records. You need to use the subgraph on several places but with different filter expression. Export the filter expression as a public parameter and let the user of the subgraph to set it up per subgraph component.


    Exporting an Attribute as Subgraph Parameter

    To export an attribute of a component as a parameter of subgraph, choose the attribute of a component of subgraph and use the Export as subgraph parameter button.

    Export as subgraph parameter button

    Figure 44.6. Export as subgraph parameter button


    The following window opens, where you can set the parameter properties.

    Public parameter appeared as a subgraph component attribute

    Figure 44.7. Public parameter appeared as a subgraph component attribute


    The public parameter then appears as a subgraph component attribute under its respective group of properties.

    One public parameter can be used in more components of the same subgraph. For example two Filter components can share the filter expression exported from a subgraph component as a public parameter.

    Using Existing Public Graph Parameter

    Any existing public graph parameter can be used as an attribute value of components in a subgraph.

    To use an existing public parameter as a value of an attribute of a component choose the attribute and use the Use parameter as value button.

    Use parameter as value button

    Figure 44.8. Use parameter as value button


    Optional Ports

    Input and output ports of a subgraph can be marked as optional. It lets you create a component with ports that do not require a connected edge.

    It can be set up in Outline within a subgraph. Right click the port in Outline and choose the corresponding option.

    Setting up an Optional Port

    Figure 44.9. Setting up an Optional Port


    You can set up optional ports from Context menu in the subgraph editor too. Move the mouse cursor on the optional port and right click to open the Context menu.

    This way is available only if there is an edge connected to the port.

    Setting up an Optional Port in Graph Editor

    Figure 44.10. Setting up an Optional Port in Graph Editor


    There are three options:

    • required

    • optional - edge receives zero records

    • optional - edge is removed

    You can use Optional ports to conditionally enable or disable a component within subgraph. See Enable/Disable Component.

    Required

    If a port of a subgraph is marked as required, an edge has to be connected to the port of a subgraph component.

    For example, the first output port of Filter is required and the port of a subgraph will work in the same way.

    Optional - Edge Receives Zero Records

    An edge does not have to be connected to a port of the subgraph component. If an edge is not connected to a port of the subgraph component, the subgraph itself assumes the port to be connected and zero records to be received through the port.

    This is similar to any input port of SimpleGather.

    This is useful, for example, in the case of merging input data streams within a subgraph.

    Optional - Edge Is Removed

    An edge does not have to be connected to a port of the subgraph component. If an edge is not connected to a port of the subgraph component, it works like there is no edge connected to the port within the subgraph.

    This is similar to optional input ports of readers as it changes behavior of the component.

    This case is useful,for example, in the case of wrapping reader with an optional port in a subgraph.

    Optional - Edge Discards All Records

    An edge does not have to be connected to an output port of the subgraph component. If an edge is connected to an output port of the subgraph component, records are sent out to the port from of the subgraph component. If an edge is not connected to an output port of the subgraph component, records to be sent out to the port are silently discarded within the subgraph.

    It is similar to the second output port of Filter.

    This is useful, if you convert records to several output formats using a subgraph and let the user of the subgraph to choose one or more output format to use.

    Color of Subgraph

    You can set up color for the subgraph component arising from the subgraph. The subgraph can be assigned to the category of components (Readers, Writers, etc.). The subgraph component will have the color of the assigned category.

    To set up the category, right click on the subgraph component, select Open subgraph, and in the Properties Tab select SubgraphCategory.

    Icon of Subgraph

    Subgraph can have own icon assigned. Three sizes of icons can be defined:

    • Small (16x16)

    • Medium (32x32)

    • Large (64x64)

    To define the path to the icon, right click on the subgraph component, select Open Subgraph and go to Subgraph section on the Properties.

    Suggested place for subgraph icons is in ${PROJECT}/icons.

    As an icon, .png and .gif files can be used.

    Continue with Developing and Testing Subgraphs.

    Developing and Testing Subgraphs

    Subgraph can be launched and tested as standalone, without being run from a parent graph. You can run and debug it the same as an ordinary graph.

    Useful tips:
    • You can run the subgraph as an ordinary graph via the Run As item in toolbar or right-click the context menu.

    • Debugging tools such as View Data can be used as in an ordinary graph.

    • You can use all the ordinary graph elements in the subgraph - connections, metadata, lookup tables, phases, etc.

    • Prepare test data in the Debug Input section of the subgraph - components that produce data to input ports of SubgraphInput are executed only when running the subgraph as standalone, not when it's used from a parent graph. These components are used to generate testing and development data for the subgraph for easy development and testing without the need to use it in the parent graph.

    • Parameterize subgraph components using public parameters if necessary.

    Note, that subgraph elements, like graph parameters, connections, etc., are independent of graph elements of parent graph. There is no connection between them like inheritance. If you need some elements of parent subgraph, map it using the Input mapping or Output mapping attributes.

    Filling Required Parameters

    If a subgraph contains required parameters, you are asked to fill them in using Dialog for Filling Required Parameters before a graph run. This way, you can easily test the subgraph with various values of required parameters.

    • Dialog opens just before a graph (subgraph) is run from Designer if the graph has any required parameters.

    • It shows only required parameters.

    • Values of parameters are shown in the dialog. If no value is defined for the parameter, it is prefilled with a value that was used last time the graph was run.

    Dialog for Filling Required Parameters

    Figure 44.11. Dialog for Filling Required Parameters


    Metadata Propagation

    You can use Auto-propagated Metadata in the same way as in the graphs. Metadata can be propagated between a parent graph and subgraph.

    Subgraph Providing Metadata

    A subgraph can define explicit metadata in its definition and propagate them to the SubgraphOutput component. When the subgraph is used in a parent graph, these metadata will be propagated via subgraph's output edge to the parent graph.

    Typical use-case is a reader subgraph that not only reads a data source, but also provides metadata of the data source (e.g. orders). In the example below, we define explicit metadata on the output of SpreadsheetDataReader component for records containing orders. Metadata on the output of the Filter component are set to be auto-propagated, which propagates the orders metadata to the output of the subgraph (as defined by SubgraphOutput). When using such a subgraph in a parent graph, the orders metadata are auto-propagated on the output of the subgraph.

    Subgraph providing metadata

    Figure 44.12. Subgraph providing metadata


    Metadata propagated from Subgraph component

    Figure 44.13. Metadata propagated from Subgraph component


    Subgraph Requiring Metadata

    Subgraph can require specific metadata when used in a parent graph by defining explicit metadata on the outputs of SubgraphInput. When the subgraph is used, its input metadata in the parent graph must match the metadata defined inside the subgraph.

    Typical use-case is a writer subgraph that requires some specific metadata (e.g. customers) to store records in a service. In the example below, we explicitly define customers metadata on the output of the SubgraphInput. When the subgraph is used, its input metadata in the parent graph must match the customers metadata.

    Subgraph explicitly defines input metadata for customers

    Figure 44.14. Subgraph explicitly defines input metadata for customers


    Using subgraph with matching metadata

    Figure 44.15. Using subgraph with matching metadata


    Metadata Acquired from Parent

    Subgraph can be quite generic and not specify any explicit metadata, only use auto-propagated metadata. The subgraph will acquire metadata from its parent graph.

    To develop and test such a subgraph, we recommend that you define explicit metadata in the Debug Input (or Debug Output) section of the subgraph. These metadata will make the subgraph valid for testing it.

    Typical use-case is a generic filter graph that performs filtering on specific (or user defined) fields, and copies all other fields. In the example below, all edges of the subgraph body are set to be auto-propagated. When the subgraph is used in a parent graph, the customers metadata are propagated through the subgraph.

    Generic subgraph not defining explicit metadata in its body

    Figure 44.16. Generic subgraph not defining explicit metadata in its body


    Metadata propagate through the Subgraph component

    Figure 44.17. Metadata propagate through the Subgraph component


    For details on metadata propagation, see also Auto-propagated Metadata.