Version

    Part V. Graphs

    Table of Contents

    30. Components
    Adding Components
    Finding Components
    Edit Component Dialog
    Commands
    Attributes
    Enable/Disable Component
    Enabling Component
    Disabling Component
    Enabling by Graph Parameter
    Enabling by Connected Input Port
    Disable as Trash
    Passing Data Through Disabled Component
    Common Properties of Components
    Component Name
    Phases
    Component Allocation
    Specific Attribute Types
    Time Intervals
    Group Key
    Sort Key
    Metadata Templates
    31. Edges
    Connecting Components with Edges
    Types of Edges
    Assigning Metadata to Edges
    Colors of Edges
    Debugging Edges
    Selecting Debug Data
    Viewing Debug Data
    Turning Off Debug
    Edge Memory Allocation
    32. Metadata
    Records and Fields
    Record Types
    Data Types in Metadata
    Data Formats
    Locale and Locale Sensitivity
    Time Zone
    Autofilling Functions
    Metadata Types
    Internal Metadata
    External (Shared) Metadata
    SQL Query Metadata
    Reading Metadata from Special Sources
    Auto-propagated Metadata
    Priorities of Metadata
    Propagation of SQL Query Metadata
    Creating Metadata
    Extracting Metadata from a Flat File
    Extracting Metadata from an XLS(X) File
    Extracting Metadata from a Database
    Extracting Metadata from a DBase File
    Extracting Metadata from Salesforce
    SQL Query Metadata
    User Defined Metadata
    Merging Existing Metadata
    Creating Database Table from Metadata and Database Connection
    Metadata Editor
    Basics of Metadata Editor
    Record Pane
    Field Name vs. Label vs. Description
    Details Pane
    Changing and Defining Delimiters
    Changing Record Delimiter
    Changing Default (Field) Delimiter
    Defining Non-Default Delimiter for a Field
    Editing Metadata in the Source Code
    Multivalue Fields
    Lists and Maps Support in Components
    Joining on Lists and Maps (Comparison Rules)
    33. Connections
    Database Connections
    Internal Database Connections
    External (Shared) Database Connections
    Database Connection Properties
    Encryption of Access Password
    Browsing Database and Extracting Metadata from Database Tables
    Windows Authentication on Microsoft SQL Server
    Hive Connection
    Troubleshooting
    JMS Connections
    Internal JMS Connections
    External (Shared) JMS Connections
    Edit JMS Connection Wizard
    Encrypting the Authentication Password
    QuickBase Connections
    Hadoop connection
    Connecting to YARN
    Libraries Needed for Hadoop
    Kerberos Authentication for Hadoop
    MongoDB connection
    Salesforce connection
    Creating Salesforce Connection
    Important Details
    34. Lookup Tables
    Lookup Tables in Cluster Environment
    Internal Lookup Tables
    Creating Internal Lookup Tables
    Externalizing Internal Lookup Tables
    Exporting Internal Lookup Tables
    External (Shared) Lookup Tables
    Creating External (Shared) Lookup Tables
    Linking External (Shared) Lookup Tables
    Internalizing External (Shared) Lookup Tables
    Types of Lookup Tables
    Simple Lookup Table
    Database Lookup Table
    Range Lookup Table
    Persistent Lookup Table
    Aspell Lookup Table
    35. Sequences
    Persistent Sequences
    Non Persistent Sequences
    Internal Sequences
    Creating Internal Sequences
    Externalizing Internal Sequences
    Exporting Internal Sequences
    External (Shared) Sequences
    Creating External (Shared) Sequences
    Linking External (Shared) Sequences
    Internalizing External (Shared) Sequences
    Editing a Sequence
    Sequences in Cluster Environment
    36. Parameters
    Internal Parameters
    Externalizing Internal Parameters
    External (Shared) Parameters
    Creating External Parameters
    Linking External Parameters
    Internalizing External (Shared) Parameters
    XML Schema of External Parameters
    Graph Parameter Editor
    Secure Graph Parameters
    Parameters with CTL Expressions (Dynamic Parameters)
    Environment Variables
    Canonicalizing File Paths
    Using Parameters
    37. Internal/External Graph Elements
    Internal Graph Elements
    External (Shared) Graph Elements
    Working with Graph Elements
    Advantages of External (Shared) Graph Elements
    Advantages of Internal Graph Elements
    Changing Form of Graph Elements
    38. Dictionary
    Creating a Dictionary
    Using a Dictionary in Graphs
    Accessing Dictionary from Readers and Writers
    Accessing Dictionary with Java
    Accessing Dictionary with CTL2
    39. Notes in Graphs
    Placing Notes into Graph
    Resizing Notes
    Editing Notes
    Formatted Text
    Links from Notes
    Folding Notes
    Notes Properties
    Compatibility
    40. Transformations
    Defining Transformations
    Components Allowing Transformation
    Java or CTL
    Internal or External Definition
    Return Values of Transformations
    Error Actions and Error Log (deprecated since 3.0)
    Transform Editor
    Common Java Interfaces
    41. Data Partitioning (Parallel Running)
    42. Data Partitioning in Cluster
    High Availability
    Scalability
    Transformation Requests
    Parallel Data Processing
    Graph Allocation Examples
    Example of Distributed Execution
    Details of the Example Transformation Design
    Scalability of the Example Transformation
    Remote Edges