Version

    Common Properties of Readers

    • Readers allow you to specify the location of input data.

      See examples of the File URL attribute for reading from local and remote files, through proxy, input port and dictionary in Supported File URL Formats for Readers.

    • Readers allow you to view the source data. See Viewing Data on Readers.

    • Readers can read data from the input port. E.g. you can read URLs of files to be read. See Input Port Reading.

    • Readers can read only the new records. See Incremental Reading.

    • Readers can skip specific number of initial records or set limit on number of records to be read. See Selecting Input Records.

    • Readers allow you to configure a policy related to parsing incomplete or invalid data record. See Data Policy.

    • Some readers can log information about errors.

    • XML-reading components allow you to configure the parser. See XML Features.

    • In some Readers, a transformation can be or must be defined. For information about transformation templates for transformations written in CTL see:

      CTL Templates for Readers

    • Similarly, for information about transformation interfaces that must be implemented in transformations written in Java see:

      Java Interfaces for Readers

    Overview of Readers

    Table 55.1. Readers Comparison

    ComponentData sourceInput portsOutput portsEach to all outputs[1]Different to different outputs[2]TransformationTransf. req.JavaCTLAuto-propagated metadata
    CloverDataReaderCloverDX binary file01-n✓⨯⨯⨯⨯⨯⨯
    ComplexDataReaderflat file11-n⨯✓✓✓✓✓⨯
    DatabaseReaderdatabase01-n✓⨯⨯⨯⨯⨯⨯
    DataGeneratornone01-n⨯✓✓✓✓✓⨯
    DBFDataReaderdBase file0-11-n✓⨯⨯⨯⨯⨯⨯
    EmailReaderemail messages01--✓⨯✓⨯⨯
    FlatFileReaderflat file0-11-2⨯✓⨯⨯⨯⨯⨯
    HadoopReaderHadoop sequence file01⨯⨯⨯⨯⨯⨯⨯
    JavaBeanReaderdictionary01-n⨯✓⨯⨯⨯⨯ 
    JMSReaderjms messages01--✓⨯✓⨯⨯
    JSONExtractJSON file0-11-n⨯✓⨯⨯⨯⨯⨯
    JSONReaderJSON file0-11-n⨯✓⨯⨯⨯⨯⨯
    KafkaCommit 0-10⨯⨯✓⨯⨯✓⨯
    KafkaReaderKafka cluster01⨯⨯✓⨯⨯✓⨯
    LDAPReaderLDAP directory tree01-n⨯⨯⨯⨯⨯⨯⨯
    MongoDBReaderdatabase0-11-2⨯⨯✓✓⨯✓✓
    MultiLevelReaderflat file11-n⨯✓✓✓✓⨯⨯
    ParallelReaderflat file01⨯⨯⨯⨯⨯⨯✓
    QuickBaseRecordReaderQuickBase0-11-2⨯⨯⨯⨯⨯⨯⨯
    QuickBaseQueryReaderQuickBase01⨯⨯⨯⨯⨯⨯⨯
    SalesforceBulkReaderSalesforce01⨯⨯✓⨯⨯✓⨯
    SalesforceReaderSalesforce01⨯⨯✓⨯⨯✓⨯
    SpreadsheetDataReaderXLS(X) file0-11-2⨯⨯⨯⨯⨯⨯⨯
    UniversalDataReaderflat file0-11-2⨯✓⨯⨯⨯⨯⨯
    XMLExtractXML file0-11-n⨯✓⨯⨯⨯⨯⨯
    XMLReaderXML file0-11-n⨯✓⨯⨯⨯⨯⨯
    XMLXPathReaderXML file0-11-n⨯✓⨯⨯⨯⨯⨯

    [1] The component sends each data record to all of the connected output ports.

    [2] The component sends different data records to different output ports using return values of the transformation (DataGenerator and MultiLevelReader). For more information, see Return Values of Transformations. XMLExtract, XMLReader and XMLXPathReader send data to ports as defined in their Mapping or Mapping URL attribute.