About the Fivetran collector
Use this collector to discover Fivetran connectors, jobs, groups, sources, and destinations and perform impact analysis to understand how changes upstream to Fivetran connectors impact downstream systems.
Important
The Fivetran collector can be run in the Cloud or on-premise using Docker or Jar files.
Note
The latest version of the Collector is 2.247. To view the release notes for this version and all previous versions, please go here.
What is cataloged
Object | Information cataloged |
|---|---|
Groups | Name, Identifier, Creation date |
Connectors | Identifier, Succeeded at datetime, Failed at datetime, Created at datetime, Sync frequency, Name, Version, Daily sync time, Schedule type, Setup state, Update state, Pause state, Sync state, Connector type, Connected by user identifier |
Connector Jobs | Run state (Failed time, Success time) |
Destination | Name, Setup status, Destination type, Identifier, Region |
Data source | Data source configuration properties, as configuration key and value pairs. |
Database tabular source | Database name, Server, Port, JDBC URL |
Non-database tabular source | Datasource ID, Name |
Database schema | Name |
Destination database | Database name, Server, Port, JDBC URL |
Columns | Name |
Relationships between objects
By default, the harvested metadata includes catalog pages for the following resource types. Each catalog page has a relationship to the other related resource types. If the metadata presentation for this data source has been customized with the help of the data.world Solutions team, you may see other resource pages and relationships.
Resource page | Relationship |
|---|---|
Group | Destinations within group, Connectors within group |
Connector | Source associated to connector, group that contains this connector, connector job which was initiated by this connector. |
Connector Job | Connector |
Fivetran Data Source | Tables that this Fivetran Data Source connects to via Source (file or database), Connector that connects to this Fivetran Data Source |
Data Source | Tables within data source. For non-database sources, tabular sources connected via data source |
Destination | Tables within destination, group that contains this destination |
Table | Column within the table |
Column | Table containing the column |
Lineage for Fivetran
The Fivetran collector identifies the source columns that destination columns sourced their data from (via copy / load operation).
The collector catalogs all database sources and destinations. Cross-system column-level lineage is currently supported for PostgreSQL, SQL Server, and Snowflake.
Object | Lineage available |
|---|---|
Source database columns and tables | Downstream destinations that fields source their data from. |
Supported cross-system lineage
The currently supported data sources/connectors for cross-system lineage are:
PostgreSQL
SQL Server
Important
While other data sources are not formally supported, running the collector for those sources may still enable you to view cross-system lineage between Fivetran and these sources.
The currently supported databases destinations for cross-system lineage are:
Snowflake
SQL Server
Authentication supported
The Fivetran collector supports the following method for authentication:
Fivetran API key: The owner of the API key needs read access to destinations and connectors. See the Fivetran documentation for more information.
You will need the API Key and API secret information while running the collector.