Skip to main content

Catalog collector release notes

Important

Published versions of collectors are available as a docker image and a JAR file.

Release version 2.193

Details about the release

Table 1.

Item

Details

Release version

2.193

Release date

15 March, 2024

Docker image ID

Jar file



New features and changes

Bug fixes

  • The Azure Data factory collector is updated to correctly handle a situation that previously caused the collector to stop, due to the format of the information returned from the ADF APIs.

Release version 2.192

Details about the release

Table 2.

Item

Details

Release version

2.192

Release date

12 March, 2024

Docker image ID

Jar file



New features and changes

  • Amazon S3 collector: The collector now offers the options, --include-object and --exclude-object. These options allow you to select which objects should be included or excluded from the harvesting process.

  • Databricks collector: The collector now harvests Databricks tags for database, schema, table, view, and column as as key-value pairs. The collector also harvests tags for clusters and jobs, replacing the existing ClusterTag and JobTag resource types.

Release version 2.191

Details about the release

Table 3.

Item

Details

Release version

2.191

Release date

7 March, 2024

Docked image ID

Jar file



New features and changes

  • All collectors: The --dry-run option is now available for all collectors. This option allows you do a test run for the collectors to validate that the collector can authenticate to the specified source system. If specified, the collector does not actually harvest any metadata, but just checks the connection parameters provided by the user and reports success or failure at connecting.

Bug fixes

  • Teradata collector: The collector is updated to correctly parse view SQL syntax for extracting lineage metadata. It also now includes improved logging of any errors encountered during lineage harvesting.

  • BigQuery collector: The collector now properly handles fully qualified table names that include dashes (-).

Release version 2.190

Details about the release

Table 4.

Item

Details

Release version

2.190

Release date

5 March, 2024

Docked image ID

Jar file



New features and changes

  • Snowflake, Teradata and Netezza collectors: In the harvested metadata, the owner of resources are now correctly referenced as owner objects. Earlier they were referenced as string text.

Bug fixes

  • The Teradata collector now correctly manages variations in database cases within SQL statements while gathering lineage metadata.

Release version 2.189

Details about the release

Table 5.

Item

Details

Release version

2.189

Release date

24 February, 2024

Docker image ID

JAR file



New features and changes

  • The Tableau collector now captures all sub-projects when you specify certain projects to catalog. Additionally, it enables users to exclude specific projects using the --tableau-exclude-project parameter. Any sub-projects under an excluded project are also automatically excluded.

Release version 2.188

Details about the release

Table 6.

Item

Details

Release version

2.1288

Release date

23 February, 2024

Docker image ID

JAR file



New features and changes

  • The Information Schema Catalog Collector now collects descriptions from both tables and columns, if they are present in the source.

  • The Snowflake collector now harvests comments from Snowflake databases, schemas, and views (as resource description).

  • The Teradata collector has been enhanced to better parse view SQL definitions that use specific Teradata syntax elements, particularly when extracting lineage from views.

Bug fixes

  • BigQuery collector:

    • Fixed issues with handling identifiers with hyphens ( -).

    • Fixed issues with harvesting lineage when a view refers to columns in a separate database.

Release version 2.187

Details about the release

Table 7.

Item

Details

Release version

2.187

Release date

20 February, 2024

Docker image ID

JAR file



New features and changes

  • Netezza collector: A new and improved collector is now available for Netezza.

  • Oracle collector: The collector now harvest definitions for view, function and stored procedure.

Release version 2.186

Details about the release

Table 8.

Item

Details

Release version

2.186

Release date

14 February, 2024

Docker image ID

JAR file



New features and changes

  • The following collectors now harvest all databases in a single collector run when the --database parameter is not specified. 

    The collectors also support a new parameter --exclude-database to exclude specific databases from metadata collection:

    • Databricks

    • DB2

    • MySQL

    • Oracle

    • PostgreSQL

    • Redshift

    • SQL Server

    • Snowflake

    • Teradata

Bug fixes

  • Databricks collector: The collector properly handles malformed task responses.

  • Power BI collector: The collector properly handles harvesting lineage relationships from Power BI data sources when parameters are used in place of the Snowflake Warehouse value.

  • For the following collectors, the behavior of the --include-information-schema option is changed. Now, if you use this option in the command without the --all-schemas option, the system will generate a warning to alert you about the missing parameter.

    • Databricks

    • DB2

    • Oracle

    • PostgreSQL

    • Redshift

    • SQL Server

    • Snowflake

Release version 2.185

Details about the release

Table 9.

Item

Details

Release version

2.185

Release date

9 February, 2024

Docker image ID

JAR file



Bug fixes

  • Fixed an issue that was causing database collectors to run into error state.

Release version 2.184

Details about the release

Table 10.

Item

Details

Release version

2.184

Release date

7 February, 2024

Docker image ID

JAR file



Bug fixes

  • Azure Data Lake Storage Gen2 collector: Fixed an issue that previously prevented the collector from running successfully on machines using amd64 processor.

  • Microsoft SQL Server collector now properly harvests views from Azure Synapse Analytics.

Release version 2.183

Details about the release

Table 11.

Item

Details

Release version

2.183

Release date

1 February, 2024

Docker image ID

JAR file



Bug fixes

  • Tableau collector: The collector is updated to properly harvest usage data in newer versions of Tableau Server.

  • Azure Data Lake Storage Gen2 Collector: Fixed an authentication issue in the collector that resulted in failures to initialize a channel.

  • Snowflake collector: The collector now properly harvests lineage between function and source table if the source table is in the cataloged schema.

Release version 2.182

Details about the release

Table 12.

Item

Details

Release version

2.182

Release date

30 January, 2024

Docker image ID

JAR file



New features and changes

  • All collectors: In addition to being available as Docker Images, collectors are now also accessible as JAR files. Follow these instructions to run collectors using JAR files.

  • The following collectors now harvest all versions of overloaded function and stored procedure resources, each as its own resource:

    • Db2

    • MS SQL Server

    • Netezza

    • Oracle

    • PostgreSQL

    • Redshift

    • Snowflake

    • Teradata

Bug fixes

  • Teradata and MySQL collectors: The following schema options have been removed for these collectors: --all-schemas, --include-information-schema, and --schema.

Release version 2.181

Details about the release

Table 13.

Item

Details

Release version

2.181

Release date

22 January, 2024

Docker image ID

  • arm64: 55898dd6bee4c8760f2f242467887298b10afebef6a4e7b21022b8dbd50d6595

  • amd64: 3583b8ca098d37f47efcb815934e8e58b3b9bf774b0c03101e367908957b964a



New feature and changes:

  • The Snowflake collector now harvests Data Metric Functions, their associations to tables and observed metrics.

Release version 2.180

Details about the release

Table 14.

Item

Details

Release version

2.180

Release date

17 January, 2024

Docker image ID

  • arm64: bd1c31006bdccb9dfc55849999fb80a25b0602dc3e6233444b4c36e06ececc9a

  • amd64: 9ea00c32bf8d5b214b20e98bce0fd11e7b15673d61f2bbc3da13fbd804ff9bac



New features and changes

  • Snowflake collector harvests allowed tag values from Snowflake.

Bug fixes

  • Oracle collector properly harvests Column descriptions from Oracle Data Dictionary tables.

Release version 2.179

Details about the release

Table 15.

Item

Details

Release version

2.179

Release date

10 January, 2024

Docker image ID

  • arm64: d80d17e87ce7925c9ef46ff1fee577940e73b0479e19414f4f0266e3da2f7f99

  • amd64: cd2a2d0ae59a44ebf519399acda91772cd62e3bac04341a51c93abbb2a34c6f9



New features and changes

  • The latest tag for docker images has been removed and is not available for use going forward.

    What does this change mean for users using the latest tag?

    • If you were using the latest tag, you can continue to use the image with the latest tag. However, we recommend all users update their docker run command to use an explicit version.

    • If you make a change to your local docker environment (such as removing the latest image), then your collector run will not work. You will need to update the run command to use a specific version. You can open a support ticket for assistance on updating the command.

  • Athena, Snowflake, SQL Server, DB2 collectors now harvest basic metadata for materialized views (name, description if available).

  • The Postgres collector now collector harvests materialized view with name, description, and view SQL definition (DDL) and column-level lineage.

Bug fixes

  • All collectors: Environment variables referenced in collector config (YAML) files can now have values containing backslashes and dollar signs.

Release version 2.178

Details about the release

Table 16.

Item

Details

Release version

2.178

Release date

5 January, 2024

Docker image ID

  • arm64: 3d05719236c2838e9693bd6db37455728763daf52458f28d026b4d5c28c1d518

  • amd64: fa7f73cb70c10d8fe6cf8ec882a72afa54e26386491f6eec92c96a1090005833



New features and changes

  • The Snowflake collector now harvests the External URL for Snowsight for tables and views.

  • The dbt Cloud collector now includes --dbt-cloud-host option to enable interaction with dbt static access URLs.

Bug fixes

  • Databricks collector: Addressed an issue related to correctly forming IRIs for tables under certain circumstances. This was previously causing duplicate tables and databases to be cataloged and non-existent tables to be referenced by columns.

  • The Tableau collector now properly handles a scenario when the Tableau instance has no databases defined.

Release version 2.177

Details about the release

Table 17.

Item

Details

Release version

2.177

Release date

22 December, 2023

Docker image ID

  • amd64: 3fd446534e173b1773d11d7afdb8dbf9256afa21798e81e79e35928f694afda7

  • arm64: 3ca604807c9c2829cc527db1d6b6f43093e1cb1870b8f34c8d3f29d7f1513436



Bug fixes

  • dbt Core and dbt Cloud collectors now catalog the dbt product version.

  • Tableau collector properly handles columns with missing names.

  • Monte Carlo collector:

    • The collector now correctly associates views with incidents, rectifying previous issues caused by missing details for certain incident types and subtypes.

    • The collector has improved log messages when relating tables to incidents.

Release version 2.176

Details about the release

Table 18.

Item

Details

Release version

2.176

Release date

20 December, 2023

Docker image ID

  • arm64: 795a717210ad6fd9bfc66335cc6a178e479604f5e7232ae8343248910cb28b57

  • amd64: f72b137b0b47b533a3714e9b358c5cdd0919d931fab4c2dd68155bf334010a03



Bug fixes

  • Teradata collector: Information was missing while harvesting funtions from Teradata.

  • dbt Cloud and dbt Core collectors: Information was missing while harvesting test results from dbt Cloud and dbt Core.

Release version 2.175

Details about the release

Important

This release was for internal improvements and has no customer impacting changes.

Table 19.

Item

Details

Release version

2.175

Release date

19 December, 2023



Release version 2.174

Details about the release

Table 20.

Item

Details

Release version

2.174

Release date

18 December, 2023

Docker image ID

  • amd64: 8253823ee1192c842b373baa89cc92f653925f9c31bc66a2e8570b254c412120

  • arm64: 7f8275ab1eedb57275818b807ed8412be3c644fdbfdd7d436739bfbd5b2ae287



New features and changes

  • The following two new collectors are now available:

  • The following collectors now harvest Schema resources from the source:

    • Databricks, PostgresSQL, SQL Server, Db2, Redshift, Generic JDBC Collector, Denodo, Dremio, Infor ION, Oracle, Salesforce, SQL Anywhere, Athena, MySQL, Snowflake, Teradata, Presto, Vertica

  • dbt Cloud and dbt Core collectors now harvest following additional metadata: test results (failed, warning, success), last test run timestamp, test name, test arguments and type of dbt test.

Bug fixes

  • Teradata collector:  Information was missing while harvesting triggers from Teradata.

Release version 2.173

Details about the release

Table 21.

Item

Details

Release version

2.173

Release date

12 December, 2023

Docker image ID

  • amd64: 209d4c7a184fde3357548700fcd8b7fd88ccf5baea2d4ac0ad71c7060f2ed30d

  • arm64: 4d75d8c96a24955f77c2c35da4c1d01aa8ececdbbbdbcd4192eec0713219e2c9



New features and changes

  • dbt cloud and dbt core collectors now harvests metadata for Columns defined within Models and Sources

  • The Power BI collector now automatically filter out workspaces named My workspace or PersonalWorkspace <User> when the --all-workspaces-and-apps parameter is used. However, if you wish to include these workspaces in the catalog, you can use the --include-user-workspace option.

Release version 2.172

Details about the release

Important

This release was for internal improvements and has no customer impacting changes.

Table 22.

Item

Details

Release version

2.172

Release date

12 December, 2023



Release version 2.171

Details about this release

Table 23.

Item

Details

Release version

2.171

Release date

6 December, 2023

Docker image ID

  • amd64: 16194c9acaa97741b17dd14525968d3a4ee6afd5babe8b0d4cf32763de6b4c0d

  • arm64: f7eae1d3f25c88eea85c6353e902baeb5e6645440494c382546a627cda5873e3



New features and changes

  • Monte Carlo collector: The Monte Carlo collector is enhanced to automatically retry harvesting from Monte Carlo in case of API failure.

Bug fixes

  • All collectors: If errors occur while running the collectors using the YAML file, the collectors will now return a not successful exit status.

Release notes for previous versions