Chapter 44. Developing Subgraphs
There are two ways how to create a subgraph.
Wrapping
Subgraph wrapping is a way to convert a section of existing graph into a subgraph. Wizard will let you copy additional graph elements (metadata, connections, lookups) from a parental graph to subgraph.
Wrapping in Steps
Select components you would like to move into a new subgraph.
Figure 44.1. Original graph without subgraphs
Use the right mouse button and choose Wrap As Subgraph.
Figure 44.2. Wrapping components into a subgraph
Enter the name of the subgraph file (
*.sgrf) and order of its input and output ports.
Figure 44.3. Wrapping Subgraph Wizard
A new Subgraph component replaced the wrapped components in the parent graph.
Figure 44.4. CloverDX Graph with the Subgraph Component
Continue with Making Subgraph Configurable.
Creating from Scratch
A new subgraph can be created from scratch. It has an initial structure - it contains a debug input component, SubgraphInput and SubgraphOutput components and a sample of the subgraph body. The initial structure is a template to help you design the subgraph.
Choose in the main menu → → .
 |
Figure 44.5. A new Subgraph
Design the subgraph body - implement the subgraph's logic in the central body section of the subgraph, using components, other subgraphs, etc.
Connect the subgraph body with the SubgraphInput and SubgraphOutput components.
Continue with Making Subgraph Configurable.
Making Subgraph Configurable
| Optional Ports |
| Color of Subgraph |
| Icon of Subgraph |
Each attribute of a component in a subgraph can be exposed as an attribute of corresponding subgraph component using public parameter. It allows you to develop more generic subgraphs.
Example 44.1. Using public parameter
You have a subgraph filtering and aggregating records. You need to use the subgraph on several places but with different filter expression. Export the filter expression as a public parameter and let the user of the subgraph to set it up per subgraph component.
Exporting an Attribute as Subgraph Parameter
To export an attribute of a component as a parameter of subgraph, choose the attribute of a component of subgraph and use the Export as subgraph parameter button.
Figure 44.6. Export as subgraph parameter button
The following window opens, where you can set the parameter properties.
Figure 44.7. Public parameter appeared as a subgraph component attribute
The public parameter then appears as a subgraph component attribute under its respective group of properties.
One public parameter can be used in more components of the same subgraph. For example two Filter components can share the filter expression exported from a subgraph component as a public parameter.
Using Existing Public Graph Parameter
Any existing public graph parameter can be used as an attribute value of components in a subgraph.
To use an existing public parameter as a value of an attribute of a component choose the attribute and use the Use parameter as value button.
Figure 44.8. Use parameter as value button
Optional Ports
Input and output ports of a subgraph can be marked as optional. It lets you create a component with ports that do not require a connected edge.
It can be set up in Outline within a subgraph. Right click the port in Outline and choose the corresponding option.
Figure 44.9. Setting up an Optional Port
You can set up optional ports from Context menu in the subgraph editor too. Move the mouse cursor on the optional port and right click to open the Context menu.
This way is available only if there is an edge connected to the port.
Figure 44.10. Setting up an Optional Port in Graph Editor
There are three options:
required
optional - edge receives zero records
optional - edge is removed
You can use Optional ports to conditionally enable or disable a component within subgraph. See Enable/Disable Component.
Required
If a port of a subgraph is marked as required, an edge has to be connected to the port of a subgraph component.
For example, the first output port of Filter is required and the port of a subgraph will work in the same way.
Optional - Edge Receives Zero Records
An edge does not have to be connected to a port of the subgraph component. If an edge is not connected to a port of the subgraph component, the subgraph itself assumes the port to be connected and zero records to be received through the port.
This is similar to any input port of SimpleGather.
This is useful, for example, in the case of merging input data streams within a subgraph.
Optional - Edge Is Removed
An edge does not have to be connected to a port of the subgraph component. If an edge is not connected to a port of the subgraph component, it works like there is no edge connected to the port within the subgraph.
This is similar to optional input ports of readers as it changes behavior of the component.
This case is useful,for example, in the case of wrapping reader with an optional port in a subgraph.
Optional - Edge Discards All Records
An edge does not have to be connected to an output port of the subgraph component. If an edge is connected to an output port of the subgraph component, records are sent out to the port from of the subgraph component. If an edge is not connected to an output port of the subgraph component, records to be sent out to the port are silently discarded within the subgraph.
It is similar to the second output port of Filter.
This is useful, if you convert records to several output formats using a subgraph and let the user of the subgraph to choose one or more output format to use.
Color of Subgraph
You can set up color for the subgraph component arising from the subgraph. The subgraph can be assigned to the category of components (Readers, Writers, etc.). The subgraph component will have the color of the assigned category.
To set up the category, right click on the subgraph component, select Open subgraph, and in the Properties Tab select → .
Icon of Subgraph
Subgraph can have own icon assigned. Three sizes of icons can be defined:
Small (16x16)
Medium (32x32)
Large (64x64)
To define the path to the icon, right click on the subgraph component, select Open Subgraph and go to Subgraph section on the Properties.
Suggested place for subgraph icons is in ${PROJECT}/icons.
As an icon, .png and .gif files can be used.
Continue with Developing and Testing Subgraphs.
Developing and Testing Subgraphs
Subgraph can be launched and tested as standalone, without being run from a parent graph. You can run and debug it the same as an ordinary graph.
Useful tips:
You can run the subgraph as an ordinary graph via the Run As item in toolbar or right-click the context menu.
Debugging tools such as View Data can be used as in an ordinary graph.
You can use all the ordinary graph elements in the subgraph - connections, metadata, lookup tables, phases, etc.
Prepare test data in the Debug Input section of the subgraph - components that produce data to input ports of SubgraphInput are executed only when running the subgraph as standalone, not when it's used from a parent graph. These components are used to generate testing and development data for the subgraph for easy development and testing without the need to use it in the parent graph.
Parameterize subgraph components using public parameters if necessary.
Note, that subgraph elements, like graph parameters, connections, etc., are independent of graph elements of parent graph. There is no connection between them like inheritance. If you need some elements of parent subgraph, map it using the Input mapping or Output mapping attributes.
Filling Required Parameters
If a subgraph contains required parameters, you are asked to fill them in using Dialog for Filling Required Parameters before a graph run. This way, you can easily test the subgraph with various values of required parameters.
Dialog opens just before a graph (subgraph) is run from Designer if the graph has any required parameters.
It shows only required parameters.
Values of parameters are shown in the dialog. If no value is defined for the parameter, it is prefilled with a value that was used last time the graph was run.
Figure 44.11. Dialog for Filling Required Parameters
Metadata Propagation
You can use Auto-propagated Metadata in the same way as in the graphs. Metadata can be propagated between a parent graph and subgraph.
Subgraph Providing Metadata
A subgraph can define explicit metadata in its definition and propagate them to the SubgraphOutput component. When the subgraph is used in a parent graph, these metadata will be propagated via subgraph's output edge to the parent graph.
Typical use-case is a reader subgraph that not only reads a data source,
but also provides metadata of the data source (e.g. orders).
In the example below, we define explicit metadata on the output of
SpreadsheetDataReader component for records containing orders.
Metadata on the output of the Filter
component are set to be auto-propagated, which propagates the
orders metadata to the output of the subgraph (as defined by SubgraphOutput).
When using such a subgraph in a parent graph, the orders
metadata are auto-propagated on the output of the subgraph.
Figure 44.12. Subgraph providing metadata
Figure 44.13. Metadata propagated from Subgraph component
Subgraph Requiring Metadata
Subgraph can require specific metadata when used in a parent graph by defining explicit metadata on the outputs of SubgraphInput. When the subgraph is used, its input metadata in the parent graph must match the metadata defined inside the subgraph.
Typical use-case is a writer subgraph that requires some specific
metadata (e.g. customers) to store records in a service.
In the example below, we explicitly define customers
metadata on the output of the SubgraphInput.
When the subgraph is used, its input metadata in the parent graph
must match the customers metadata.
Figure 44.14. Subgraph explicitly defines input metadata for customers
Figure 44.15. Using subgraph with matching metadata
Metadata Acquired from Parent
Subgraph can be quite generic and not specify any explicit metadata, only use auto-propagated metadata. The subgraph will acquire metadata from its parent graph.
To develop and test such a subgraph, we recommend that you define explicit metadata in the Debug Input (or Debug Output) section of the subgraph. These metadata will make the subgraph valid for testing it.
Typical use-case is a generic filter graph that performs filtering on specific (or user defined) fields, and copies all other fields. In the example below, all edges of the subgraph body are set to be auto-propagated. When the subgraph is used in a parent graph, the customers metadata are propagated through the subgraph.
 |
Figure 44.16. Generic subgraph not defining explicit metadata in its body
Figure 44.17. Metadata propagate through the Subgraph component
For details on metadata propagation, see also Auto-propagated Metadata.