DBCatalogExport reads the schemas, tables, and columns of a database and writes them to a JSON file, the database catalog. The data lineage of graphs uses the catalog to resolve the columns that SQL queries of DatabaseReader components read.

Reading the catalog of a large database takes a long time. The component therefore runs once and writes the result to a file, and the lineage reads the file instead of the database.

The component has no input port.

DBCatalogExport propagates metadata to its output port. The output port receives one record per exported schema, followed by one record with the totals. A schema without tables and views gets no record and does not count in the totals. In the totals record, the catalog and schema fields are empty.

The catalog file contains the schemas, tables, and views of the database, their columns with types and comments, and the primary and foreign keys. The header of the file says when the catalog was read, which connection it was read from, and which schema patterns limited it. The header stores the database product and version and the JDBC URL of the connection. The component removes the user name and the password from the URL before it writes the file.

The component does not add a file extension. We recommend the .dbcatalog.json suffix, or .dbcatalog.json.gz for a compressed file. The lineage reads plain, GZIP-compressed, and ZIP-packed catalog files.

When the graph is aborted, the component stops the export between two metadata queries and writes no file.

At the end of the export, the component writes the number of schemas, tables, views, and columns and the duration to the graph log.