Skip to main content

Catalog collector release notes

Important

Published versions of collectors are available as a docker image and a JAR file.

Release version 2.235

Details about the release

Table 1.

Item

Details

Release version

2.235

Release date

September 10, 2024

Docker image ID

Jar file



New features and changes

  • Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors: The collectors now support harvesting decimal digits metadata for columns.

Release version 2.234

Details about the release

Table 2.

Item

Details

Release version

2.234

Release date

September 6, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:aba0a42c93f9ed4dc73f4290f4d1c917f6f2d2ad92cb21bc94fcbf6b18fc5af4

    • amd64: sha256:bbfb88b4665d0b1a72b54f600e829a16e104db9e4640b1b74bbb90d970a98601

Jar file



Bug fixes

  • Generic JDBC, Denodo, MySQL, SQL Server collectors: Resolved arithmetic overflow error when converting expressions to data type bigint.

  • SSIS collector: Added database location information when creating a database asset.

Release version 2.233

Details about the release

Table 3.

Item

Details

Release version

2.233

Release date

August 30, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:3efdd72b32787bce83c35d0fbb02760f485144b4e3c4084d0cc7f5caa569072c

    • amd64: sha256:d220ee98fd2cfe6edccefcf528f424dc65525239f0a5d4b6b62379a64d5b4d78

Jar file



New features and changes

  • Tableau collector: The collector now supports filtering descendant projects.

Bug fixes

  • Salesforce collector: The collector now properly handles null responses from the Salesforce API.

  • Snowflake collector: Fixed an issue where column nodes were not being copied to the interleaved Snowflake graph.

  • Tableau collector: Fixed an issue with missing owner users for various resources.

Release version 2.232

Details about the release

Table 4.

Item

Details

Release version

2.232

Release date

August 27, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:00dd206dd9d825a378db70b96609c9ff29569c9b30cc04dcada8e1a2673220f6

    • amd64: sha256:5bbc8fe2677e3425ed1b80fa59876327fbb24da0db94377645564c8c4d47bf5f

Jar file



New features and changes

  • Power BI and Power BI Gov collectors: Added new relationships from app report to workspace report.

Bug fixes

  • Salesforce Collector: Implemented a fix to avoid exceptions when custom object field metadata contains null values.

  • Denodo Collector: Resolved an issue with harvesting view SQL.

Release version 2.231

Details about the release

Table 5.

Item

Details

Release version

2.231

Release date

August 14, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:1149dec23147e01dd9ad3b92849801cc698e5055efe0ccd218adb4d7a2cc3498

    • amd64: sha256:600c532cf0c39afc00f34f00c8d78d1cc073be5026ce535e0f692acdcaae257a

Jar file



New features and changes

  • Power BI Service and Power BI Gov collectors:

    • The collectors now support parsing parameters, measures and column expressions when expression parsing is enabled.

    • Added functionality to get the correct case for database, schema, table, and column names when database credentials are provided for a source.

  • Power BI Gov collector: The collector now supports harvesting of user workspaces using the --include-user-workspace parameter.

  • Databricks collector: The collector now supports harvesting data from ADLS (Azure Data Lake Storage) Gen2 external location.

  • dbt Core collector: Added support for dbt projects using the dbt-sqlserver adapter.

Release version 2.230

Details about the release

Table 6.

Item

Details

Release version

2.230

Release date

August 6, 2024

Docker image

Jar file



Bug fixes

  • SQL Server Reporting Services (SSRS) and Power BI Report Server (PBIRS) collectors: Fixed an issue that occurred while fetching information on data sources for a report.

Release version 2.228

Details about the release

Table 7.

Item

Details

Release version

2.228

Release date

August 5, 2024

Docker image

Jar file



Bug fixes:

  • SQL Server Reporting Services (SSRS) and Power BI Report Server (PBIRS) collectors: Fixed an issue that caused API requests to return an HTTP 404 error.

Release version 2.227

Details about the release

Table 8.

Item

Details

Release version

2.227

Release date

July 31, 2024

Docker image ID

Jar file



New features and changes:

  • SQL Server collector: If an error occurs while fetching columns from the database by schema, the collector now attempts to fetch columns by table instead.

Bug fixes:

  • SQL Server Reporting Services (SSRS) and Power BI Report Server (PBIRS) collectors: Fixed an issue when collector resources were not returned by the API.

Release version 2.226

Details about the release

Table 9.

Item

Details

Release version

2.226

Release date

July, 31, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:f55a432cbf6aa6d260cafffc528371e76d3bdeb94c19ef4131c1bfec189448b1

    • amd64: sha256:351dd8559fbc460de302d9bd87bcae815244f92ecbbca651aa403db97be5110b

Jar file



Bug fixes:

  • SQL Server Reporting Services (SSRS) and Power BI Report Server (PBIRS) collectors: Fixed an issue where the collector would terminate abnormally if the SSRS API returned no data under certain circumstances.

  • Power BI Service and Power BI Gov Collectors: The collectors now correctly handled case mismatches in source column names when resolving SQL statements for lineage.

Release version 2.225

Details about the release

Table 10.

Item

Details

Release version

2.225

Release date

July 30, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:5506911e638a5b7493395bcce874837cf3fb19bdfe864cfab1b501996f115b4f

    • amd64: sha256:cbdeed21a29be42cf423c8114de30f180b7ce67225f422d91c2f73a56d6de686

Jar file



Bug fixes

  • SQL Server Reporting Services (SSRS) and Power BI Report Server (PBIRS) collectors: Fixed an issue that caused the collector to terminate unexpectedly when encountering Linked Reports with names containing non-alphanumeric characters.

Release version 2.224

Details about the release

Table 11.

Item

Details

Release version

2.224

Release date

July 30, 2024

Docker image ID

Jar file



New features and changes

  • Oracle collector: Enabled caching for primary keys and foreign keys, and reduced the number of queries used to gather table and column extended metadata, resulting in improved collector run time.

  • SQL Server Reporting Services (SSRS) and Power BI Report Server (PBIRS) collectors: Item path is now harvested for report, data source, and dataset titles.

Bug fixes

  • SQL Server Reporting Services (SSRS) and Power BI Report Server (PBIRS) collectors: Resolved an issue with NTLM Authentication.

Release version 2.223

Details about the release

Table 12.

Item

Details

Release version

2.223

Release date

July 29, 2024

Docker image ID

Jar file



New features and changes

  • Power BI Service and Power BI Gov collectors: The collectors now support TNS connection strings in lineage parsing for Oracles sources if HOST and SID are specified. For example, (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=SERVER_NAME)(PORT=1521))(CONNECT_DATA=(SID=KOSTEST))).

  • SQL Server Reporting Services and Power BI Report Service collectors: The collectors now support authentication using NTLM.

  • Amazon S3 collector: The collector now harvests objects that begin with a prefix.

  • Salesforce collector: The collector now harvests metadata for Objects, Fields, Dashboards, and Reports. It also supports OAuth authentication instead of Basic authentication. You must complete the new pre-requisite tasks to use OAuth authentication.

  • Tableau collector: Enhanced resiliency for Tableau GraphQL query execution.

Bug fixes

  • Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors:

    • The collectors now properly handle SQL parsing for lineage, ensuring newline characters \r do not disrupt SQL parsing.

    • Fixed an issue with the usage of variable names in stored procedures.

  • Power BI Service and Power BI Gov collectors: Fixed an issue with handling parameters that are defined in the tables section of Semantic Models, allowing for successful parsing of source information for tables using those parameters.

  • Snowflake collector: The collector now appropriately handles date time parsing for the timestamp NTZ format.

Release version 2.222

Details about the release

Table 13.

Item

Details

Release version

2.222

Release date

July 23, 2024

Docker image ID

Jar file



New features and changes

  • Power BI Service and Power BI Gov collectors: The collectors now support Denodo sources in Power BI column-level lineage parsing.

  • Denodo collector: The collector now harvests column-level lineage.

  • SQL Server Integration Services (SSIS) collector: Added a new --jdbc-property parameter. This allows you to provide authentication details for NTLM Authentication type.

  • dbt Core and dbt Cloud collectors: The collectors now harvests model columns from catalog.json and manifest.json database objects.

Bug fixes

  • Power BI collector: The collector now properly handle scenarios where columns are renamed in Power BI that resulted in duplicate columns in source tables.

  • Azure Data Factory collector: The collector now properly performs Date transformation when the time zone is not available as ZoneID.

  • Azure Data Lake Storage Gen2 collector:

    • Updated the collector to remove redundant permission-related relationships.

    • Fixed an issue with the IRIs creation for collector resources by using correct terms.

Release version 2.221

Details about the release

Table 14.

Item

Details

Release version

2.221

Release date

July 15, 2024

Docker image ID

Jar file



New features and changes

  • Power BI Gov collector: The collector now harvests preview images for Power BI reports. Add the new parameter --image-collection to your command/YAML file to use this new feature.

Bug fixes

  • Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors:

    • Fixed an issue where placing a comment directly after a keyword without a space was sometimes causing parsing issues.

    • Fixed an issue with parsing CREATE VIEW statements where parentheses were being incorrectly removed during the SQL pre-processing.

    • Proper error messages are now logged when users run the collectors with the --dry-run option without specifying a single database or with multiple databases.

  • Snowflake collector: Resolved an issue where the collector was cataloging an incorrect database when the user had a default namespace set in Snowflake.

  • Databricks collector: Fixed an issue where the collector output files uploads were failing due to spaces in IRIs.

  • QuickSight Collector: Fixed an AwsAccountId null error while listing resources using pagination, which was causing issues in cataloging all the specified resources.

  • Azure Data Factory collector:

    • Resolved an issue with truncated paginated results.

    • Fixed an issue with the title of global parameters by correctly using the parameter name.

Release version 2.220

Details about the release

Table 15.

Item

Details

Release version

2.220

Release date

July 10, 2024

Docker image ID

Jar file



New features and changes

  • Oracle collector: Added support for lineage when the SELECT statement contains synonyms. This enhancement fixes lineage tracking between Oracle and Power BI when synonyms are used.

  • Power BI collector: The collector now harvests preview images for Power BI reports. Add the new parameter --image-collection to your command/YAML file to use this new feature.

Bug fixes

  • Power BI and Power BI Gov collectors:

    • Fixed an issue with parameter value replacement in expressions when the parameter contains a $ symbol.

    • Fixed an issue where Power BI reports failed to process when the page name is null.

Release version 2.219

Details about the release

Table 16.

Item

Details

Release version

2.219

Release date

July 8, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:a9f524581a769ade5b7c01a887a7dbd90510a94ce6a37e115fc7a6a7e40f0557

    • amd64: sha256:f10deae54bbf33be7a40352ae048dd74d3981698dabceae748d513250dcf800a

Jar file



New features and changes

  • All collectors: Each catalog resource in the catalog output file now contains information about the collector that harvested the resource. This information is available only in the catalog file and can be used in SPARQL automations.

Bug fixes

  • Power BI and Power BI Gov collectors: The collectors now properly handle scenarios when they run into API request limits. A new parameter Disable max requests wait (--disable-max-requests-wait) is added for handling these scenarios.

  • Azure Data Lake Storage Gen2 collector: Resolved an issue where certain ACL information missing in the Azure Data Lake Storage API response caused errors in the collector.

  • Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors: Harvesting of column-level lineage from views now supports view definitions containing unaliased subselects.

Release version 2.218

Details about the release

Table 17.

Item

Details

Release version

2.218

Release date

July 1, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:93c8ebbd04553ad2123d944efca841dccc44b06dacdd0bd9683bbd7fe41c282e

    • amd64: sha256:d078c72ae238ab02d0a1e8cc003509eba463ed78f97cf2835073ee586d9fc741

Jar file



New features and changes

  • Power BI and Power BI Gov collectors: The collectors now support parsing data source expressions for Power BI tables where the source connection information is defined as a parameter. This means that if Power BI users specify data source connection information in a parameter and use that parameter in place of the source in the expression, the collectors will correctly parse and resolve the expression/lineage.

  • Oracle collector: The collector now harvests from DBA_ views if the credential used to execute the collector lacks permissions for information schema views.

  • dbt Core collector: The collector now harvest database objects and intra-database lineage from dbt projects and artifacts that use Azure Synapse as a backend.

  • All collectors: Collectors now verify that the user-requested upload location exists with proper permissions before execution and issue a warning if a problem exists.

  • Databricks collector: The collector no longer supports Databricks-managed password authentication. If you used this method of authentication, you must change the authentication to personal access token. For details, see "Preparing Databricks for collectors".

Bug fixes

  • SQL Server collector:

    • Fixed an issue where large values for column statistics produced an arithmetic overflow.

    • Resolved a problem where view definitions that include the TOP() expression were not properly handled when harvesting column-level lineage for views.

  • Power BI and Power BI Gov collectors: Fixed an issue where logging operations were causing an exception if certain Power BI objects were null.

  • Tableau collector: Fixed an issue where certain Tableau projects were not fully cataloged.

Release version 2.216

Details about the release

Important

This release was for internal improvements and has no customer impacting changes.

Table 18.

Item

Details

Release version

2.216

Release date

June 26, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:52406bc14a26061e473987b281f3c354592b811af99c7fc8cf86833cffc9e819

    • amd64: sha256:52a347688c2e68806bf82a284e5a0ee8c9dfbda5f970f24ab69037e8025e3e8f

Jar file



Release version 2.215

Details about the release

Table 19.

Item

Details

Release version

2.215

Release date

June 26, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:e8bd4f7ba78527cea800acb3a03a1440b283f1fdd81097ad19a7a8b9a5362d9c

    • amd64: sha256:9018f2d0a0b734e5bbd858e1ba5a00b88b56982da6b9c0f690f6fff901016630

Jar file



New features and changes

  • Power BI Gov Collector:

    • The collector now supports harvesting of all workspaces and apps using the --all-workspaces-and-apps parameter.

    • Added the ability to disable lineage harvesting using the --disable-expression-lineage parameter.

Release version 2.214

Important

This release was for internal improvements and has no customer impacting changes.

Details about the release

Table 20.

Item

Details

Release version

2.214

Release date

June 25, 2024

Docker image ID

Jar file



Release version 2.213

Details about the release

Table 21.

Item

Details

Release version

2.213

Release date

June 25, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:5c57df9b1285ce0f663955d0f90ad8c7c6e90acc110538473065d418a31ad2c9

    • amd64: sha256:20bf229751e3657c960ecc68a484b2df5be094215bc6e6163f206c8dcb9f0dba

Jar file



New features and changes

  • Azure Data Factory collector: The collector now harvests Expressions for table names, schema names, file names.

  • new collector for SQL Server Integration Services (SSIS) is now available in private preview. If you would like access to this collector, please contact your Customer Success Director.

Bug fixes

  • Power BI and Power BI Gov collectors: The collectors now correctly harvest lineage for column types.

Release version 2.212

Details about the release

Table 22.

Item

Details

Release version

2.212

Release date

June 21, 2024

Docker image ID

  • Link to download the Docker image: https://hub.docker.com/r/datadotworld/dwcc/tags

    • arm64: sha256:7880afc402b3db8f45a31bb90f565d417feb0c2fbfcc78715854da24c594cfc6

    • amd64: sha256:6218bc7e03a1d5b3195d64625ee4525f7739d59bbcb920afb95d719e8b90fe6a

Jar file



New features and changes:

  • Snowflake collector: The collector now harvests metadata for functions and stored procedures from the snowflake.account_usage views when the metadata is unavailable from the information_schema of the database.

  • Power BI and Power BI gov collectors now catalog:

    • Dataset table expression

    • Description for the workspace, app, and dataset

Bug fixes:

  • ADF collector: Fixed an issue with datetime parse errors while harvesting triggers.

Release version 2.211

Details about the release

Table 23.

Item

Details

Release version

2.211

Release date

June 15, 2024

Docker image ID

Jar file



New features and changes

  • Power BI and Power BI gov collectors: The collectors now support lineage for Oracle database objects.

Bug fixes

  • Power BI and Power BI gov collectors: Resolved an issue with collecting child resources for apps when using service principal authentication.

  • Snowflake and Oracle collectors: Fixed an issue where the system now correctly does not harvest function lineage when users enable the Disable lineage collection (--disable-lineage-collection) option.

  • Oracle collector: Fixed an issue with harvesting database columns of LONG type.

Release version 2.210

Details about the release

Table 24.

Item

Details

Release version

2.210

Release date

June 7, 2024

Docker image ID

Jar file



New features and changes

  • Power BI and Power BI Gov collectors:

    • Added a new feature that provides support to parse SQL statements within table expressions, enabling column-level lineage harvesting. To use this feature, you need to use the --datasource-mapping-file to specify the credentials. These credentials allow the collector to link lineage to the database sources.

    • The collector now harvests measures.

  • Databricks collector: The collector now harvests table and column tags by schema.

Bug fixes

  • Snowflake collector was unable to harvest lineage if the SQL statement included a dash in the column aliases.

  • Snowflake, Teradata, Netezza collectors: Fixed an issue that occurred because of insufficient information while harvesting agent resources for functions and procedures.

  • SQL Server collector: Fixed an issue that occurred while parsing view queries where columns have dashes in their names.

Release version 2.209

Details about the release

Table 25.

Item

Details

Release version

2.209

Release date

June 2, 2024

Docker image ID

Jar file



New features and changes

  • Databricks collector: The collector now harvests table and column lineage from system tables. To use this feature, you need to set new permissions for the collector.

Bug fixes

  • Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server collectors: Resolved a problem concerning column statistics when an aggregate statistic has a zero value.

  • Tableau collector: Resolved an issue to correctly associate lineage with the appropriate parent project.

  • Sigma collector: Resolved an issue which occurred when a dataset referred to in the lineage was not available among the harvested datasets.

  • Snowflake collector: Fixed an issue associated with external URLs containing special characters.

Release notes for previous versions