Enterprise docs

Collector configurations by data source

The prerequisites and parameters for running a catalog collector depend on the source you want to catalog. The tables below provide a quick reference to which parameters are applicable to each DWCC collector source. All DWCC collectors use the same CLI formats and basic parameters. dbt, which is a non-DWCC collector, uses a different CLI format and different parameters. It is documented fully in its own article.

Note

dbt is a non-DWCC-collector source and it only requires --account

Below is a table which lays out the required parameters for each of our DWCC collectors:

required_params.png

The collector sources listed on the top of the table are identified by color.

  • standard_JDBC.png - Standard JDBC collectors.

  • JDBC_variant.png - JDBC collectors which have more or fewer parameters than the standard ones do.

  • All other colors are purely for convenience in identifying which collector sources have which unique parameters.

The parameters down the left side are used to indicate either a type of parameter, or to link unique parameters with their sources in the top row.

  • Standard_req_JDBC_params.png - Indicates the most common parameter options, and the only truly required options for most JDBC sources.

  • One-of-two_option_req_params.png - Shows our recommendation for which option to use from an either-or pair of required options. (E.g., either --account or --base is required, and we recommend using --account.)

  • Special.png - Special parameters are labeled this color. For parameters that are required, this lable indicates parameters that are required but should use a default value. For example, --output parameter is not a required option unless the collector is running in a Docker container. All of our DWCC collectors run in Docker containers, so the --output parameter is required and it must match the output specified in the corresponding Docker ----mount statement. We use --mount type=bind,source=/tmp,target=/dwcc-output in our sample script so the --output option should always be -o "/dwcc-output".

The following table presents all the optional parameters for each collector:

optional_params.png

The assignment of colors in this table is the same as in the previous table for required parameters with one addition:

  • advanced_params.png - Indicates advanced parameters that should be avoided unless you have been advised to use them by your customer success representative.

Complete documentation for each collector--including its prerequisites and instructions for running it from a CLI--can be found in the Integrations section of the documentation.

In this section: