Common properties of Transformers

Development > Component reference > Transformers > Common properties of Transformers

Transformers have both input and output ports. They can:

put together more data flows with the same metadata (Concatenate, SimpleGather and Merge);
remove duplicate records (Dedup);
filter data records (Filter and EmailFilter);
create samples from input records (DataSampler), sort data records (ExtSort, FastSort and SortWithinGroups);
multiply existing data flow (SimpleCopy);
split one data flow into more data flows (Partition at all, but optionally also Dedup, Filter and Map);
intersect two data flows (even with different metadata on inputs) (DataIntersection), aggregate data information (Aggregate);
and perform much more complicated transformations of data flows (Map, Denormalizer, Normalizer, Rollup and XSLTransformer).

Metadata can be propagated through some of these transformers, whereas the same is not possible in such components that transform data flows in a more complicated manner. You must have the output metadata defined prior to configuring these components.

Some of these transformers use transformations that have been described above. For detailed information about how transformation should be defined, see Defining transformations.

Some Transformers can have a transformation attribute defined, it may be optional or required. For information about transformation templates for transformations written in CTL, see CTL templates for Transformers.
Some Transformers can have a transformation attribute defined, it may be optional or required. For information about transformation interfaces that must be implemented in transformations written in Java, see: Java interfaces for Transformers.

Below is an overview of all Transformers:

Table 69. Transformers comparison
Component	Same input metadata	Sorted inputs	Inputs	Outputs	Java	CTL	Auto-propagated metadata
Aggregate	-	⨯	1	1	⨯	⨯	⨯
Concatenate	✓	⨯	1-n	1	⨯	⨯	✓
DataIntersection	⨯	✓	2	3	✓	✓	✓
DataSampler	-	⨯	1	n	⨯	⨯	✓
Dedup	-	✓	1	1-2	⨯	⨯	✓
Denormalizer	-	⨯	1	1	✓	✓	⨯
ExtSort	-	⨯	1	1-n	⨯	⨯	✓
FastSort	-	⨯	1	1-n	⨯	⨯	✓
Filter	-	⨯	1	1-2	⨯	⨯	✓
LoadBalancingPartition	-	⨯	1	1-n	⨯	⨯	✓
Merge	✓	✓	2-n	1	⨯	⨯	✓
MetaPivot	-	⨯	1	1	⨯	⨯	✓
Normalizer	-	⨯	1	1	✓	✓	⨯
Partition	-	⨯	1	1-n	^[1]	^[1]	✓
Pivot	-	⨯	1	1	✓	✓	⨯
Map	-	⨯	1	1-n	✓	✓	✓
Rollup	-	⨯	1	1-n	✓	✓	⨯
SimpleCopy	-	⨯	1	1-n	⨯	⨯	✓
SimpleGather	✓	⨯	1-n	1	⨯	⨯	✓
SortWithinGroups	-	✓	1	1-n	⨯	⨯	✓
XSLTransformer	-	⨯	0-1	0-2	⨯	⨯	⨯

¹	Partition can use either a transformation or two other attributes (Ranges or Partition key). The transformation must be defined unless one of these is specified.

{{{ highlightedName }}}

Common properties of Transformers